KR20230158349A - Signal processing device of optical coherence tomography system - Google Patents

Abstract

The present invention relates to a signal processing device for an optical coherence tomography system that integrates deep learning inference, and includes a converter that converts and outputs an analog optical coherence phase difference electrical signal into digital data in the form of a stream, and a digital output of the converter at a constant level. An FFT processing unit that collects numbers in units, performs high-speed discrete Fourier transformation and outputs them, and stores the FFT processing results output from the FFT processing unit at regular intervals as row-level FFT processing results, and separately stores m*n (m and Max pooling or average pooling is performed on the convolution operation output of a certain size, which is created by convolution operation by multiplying and accumulating the FFT processing result of size (n is row and column) by kernel information. It is characterized by including a deep learning processing unit that does.

Description

Signal processing device of optical coherence tomography system}

The present invention relates to an optical coherence tomography system, and particularly to a signal processing device for an optical coherence tomography system incorporating deep learning inference.

The optical coherence tomography (OCT) system irradiates low coherence light, close to natural light, to multiple scattering materials such as living organisms and detects the light reflected from the materials to obtain tomographic images of living organisms. As a system, it is continuously being researched and developed due to its advantages of having a simpler structure than MRI or CT and being able to provide higher resolution than an ultrasound imager. It is especially widely used in the medical field as well as for detecting defects in light-transmitting materials.

The configuration of a general optical coherence tomography system that processes image signals to restore tomographic images and further processes the restored images to extract necessary information is shown in Figure 1.

As shown in Figure 1, a typical optical coherence tomography system converts the optical coherence phase difference electrical signal output through the optical system 10 and the detection system 20 into digital information through an ADC (Analog-to-Digital Converter) 30. After that, image information is created through Fast Fourier Transformation (hereinafter referred to as FFT) (40). This image information is temporarily stored in the buffer memory 50 and transmitted to the computer 70 through the Direct Memory Access Controller (DMAC) 60 and the system bus, where it is processed and displayed as a displayable tomographic image.

In some cases, the computer 70 processes the tomography image to extract (or detect) additional information, such as finding outliers, and uses it for useful applications. As a specific example, it is possible to detect defects in a product by detecting foreign substances, detecting scratches, and finding cracks. In some cases, it is possible to install and apply an artificial intelligence inference program to the computer 70 as shown in FIG. 1 to enhance the results of image processing, but in this case, because the amount of tomography image data obtained is large, The computer 70 requires a high-performance processor, and furthermore, there is the difficulty of using a general-purpose graphics controller (GP-GPU).

Republic of Korea Patent Publication No. 10-2021-0016861

Accordingly, the present invention is an invention created to solve the above-mentioned shortcomings, and the main purpose of the present invention is to reduce the computational burden on the computer constituting the optical coherence tomography system and improve the data processing speed in hardware. The purpose is to provide a signal processing device for a shooting system.

A signal processing device for an optical coherence tomography system according to an embodiment of the present invention to achieve the above-described object,

A converter (ADC) that converts and outputs an analog optical interference phase difference electrical signal into stream-type digital data,

An FFT processing unit that collects the digital output of the converter in a certain number of units, performs high-speed discrete Fourier transformation, and outputs the digital output,

The FFT processing results output from the FFT processing unit at certain times are stored separately as FFT processing results in row units, and kernel information is added to the separately stored FFT processing results of size m*n (m and n are rows and columns). It is characterized by including a deep learning processor that performs Max pooling or average pooling on the convolution operation output of a certain size, which is created by convolution operation in a multiplication and accumulation manner.

Furthermore, in the signal processing device of the optical coherence tomography system including the above-described configuration, the deep learning processing unit

A line buffer unit for convolution that stores FFT processing results output from the FFT processing unit at regular intervals in different line buffers;

A convolution processing unit that outputs a convolution operation by multiplying and accumulating row-level FFT processing results stored in each of the different line buffers by kernel information;

A line buffer unit for pooling that stores the output of the convolution operation in different line buffers,

A pooling processing unit that processes the row-level convolution operation output results stored in each of the line buffers constituting the pooling line buffer unit through max pooling or average pooling and outputs the results to a direct memory access controller. is another feature.

A signal processing device for an optical coherence tomography system according to another embodiment of the present invention includes a converter that converts and outputs an analog optical coherence phase difference electrical signal into digital data in the form of a stream;

An FFT processing unit that collects the digital output of the converter in a certain number of units, performs high-speed discrete Fourier transformation, and outputs the digital output,

A line buffer unit for convolution that stores FFT processing results output from the FFT processing unit at regular intervals in different line buffers;

A convolution processing unit that outputs a convolution operation by multiplying and accumulating row-level FFT processing results stored in each of the different line buffers by kernel information;

A line buffer unit for pooling that stores the output of the convolution operation in different line buffers,

A pooling processing unit that processes the row-level convolution operation output results stored in each of the line buffers constituting the pooling line buffer unit through max pooling or average pooling and outputs the results to a direct memory access controller. It is characterized by .

In the signal processing device of this optical coherence tomography system, the line buffer unit for convolution,

a plurality of line buffers for storing the FFT processing results in row units;

Characterized in that it includes a multiplexer that connects the output terminal of the FFT processing unit with one of the plurality of line buffers in a round-robin manner at regular intervals,

The pooling line buffer unit includes a plurality of pooling line buffers for storing the convolution operation output in row units;

Another feature includes a multiplexer that connects the output terminal of the convolution processor with one of the plurality of line buffers for pooling in a round-robin manner at regular intervals.

According to the above-described means of solving the technical problem, the present invention sequentially converts the optical coherence phase difference electrical signal output through the optical system and detection system constituting the optical coherence tomography system into digital information and FFT processing, and produces the result (image information ) is processed through a deep learning inferred convolutional neural network and then transmitted to the computer system. By designing the signal processing device of the optical coherence tomography system, an artificial intelligence inference program was installed and operated to detect defects in the object, etc. Compared to computer systems, it has the advantage of reducing the computational burden on computers and improving data processing speed, and has the advantage of supporting the function of detecting defects in objects even with low-specification computer systems.

1 is an exemplary configuration diagram of a general optical coherence tomography system.
Figure 2 is an exemplary block configuration diagram of a signal processing device for an optical coherence tomography system according to an embodiment of the present invention.
Figure 3 is an exemplary block configuration diagram of a signal processing device for an optical coherence tomography system according to another embodiment of the present invention.
Figure 4 is an example of the surrounding configuration of the convolution processing unit 133 for convolution according to an embodiment of the present invention.
Figure 5 is an exemplary peripheral configuration diagram of the pooling processing unit 137 according to an embodiment of the present invention.
Figure 6 is a diagram for explaining the form in which FFT processing results are output in an optical coherence tomography system.
7A and 7B are diagrams for explaining an example of convolution.
Figures 8 and 9 are example diagrams of line buffers to further explain the convolution process according to an embodiment of the present invention.
Figure 10 is a detailed configuration example of the deep learning processing unit 130 to which the activation function is applied.

DETAILED DESCRIPTION OF THE INVENTION The detailed description of the invention described below refers to the accompanying drawings, which show as illustrations specific embodiments in which the invention may be practiced in order to make clear the objectives, technical solutions and advantages of the invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

Also, throughout the detailed description and claims of the present invention, the word 'comprise' and its variations are not intended to exclude other technical features, attachments, components or steps. Other objects, advantages and features of the invention will appear to those skilled in the art, partly from this description and partly from practice of the invention. The examples and drawings below are provided by way of example and are not intended to limit the invention. Moreover, the present invention encompasses all possible combinations of the embodiments shown herein. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described.

In this specification, unless otherwise indicated or clearly contradictory to the context, items referred to in the singular include plural unless the context otherwise requires. In addition, in describing the present invention, known configurations or functions, for example, the configuration and structure of the optical system constituting an optical coherence tomography system (OCT) are already widely known, so detailed descriptions thereof will be omitted.

First, Figure 2 illustrates a block configuration diagram of a signal processing device of an optical coherence tomography system according to an embodiment of the present invention, and Figure 3 is a block diagram of another signal processing device of an optical coherence tomography system according to an embodiment of the present invention. This is an example of a block configuration diagram. Furthermore, Figure 4 illustrates the peripheral configuration of the convolution processor 133 for convolution according to an embodiment of the present invention, and Figure 5 illustrates the peripheral configuration diagram of the pooling processor 137 according to an embodiment of the present invention. will be.

As shown in FIG. 2, the optical coherence tomography system according to an embodiment of the present invention streams an analog optical coherence phase difference electrical signal output through the optical system 10 and the detection system 20 described in FIG. A converter (ADC, 110) that converts and outputs digital data in the form of a stream,

An FFT processing unit 120 that collects the digital output of the converter 110 in units of a certain number (for example, 1024) and outputs the high-speed discrete Fourier transform;

The FFT processing results output from the FFT processing unit 120 at certain times (e.g., the FFT performance time) are separately stored in the internal line buffer memory as row-level FFT processing results, and m*n are separately stored. (m and n are natural numbers as rows and columns) Max pooling (Max) is performed on the convolution operation output of a certain size, which is created by convolution operation by multiplying and accumulating the FFT processing result of the size by the kernel (filter) information. It includes a deep learning processing unit 130 that performs pooling or average pooling processing and outputs it to DMAC 140.

In particular, the deep learning processing unit 130 includes a convolution line buffer unit 131 that stores the FFT processing results output from the FFT processing unit 120 at regular intervals in different line buffers;

A convolution processor 133 that outputs a convolution operation by multiplying and accumulating row-level FFT processing results stored in each of the different line buffers by kernel information;

A line buffer unit 135 for pooling that stores the convolution operation output in different line buffers,

Max pooling or average pooling is performed on the row-level convolution operation output results stored in each of the line buffers constituting the pooling line buffer unit 135, and a direct memory access controller (DMAC, 140) ) and a pooling processing unit 137 that outputs the output.

As shown in FIG. 4, the line buffer unit for convolution 131, which constitutes the deep learning processing unit 130, includes a plurality of line buffers 131b, 131c, and 131d for storing FFT processing results in row units,

It includes a multiplexer (MUX, 131a) that connects the output terminal of the FFT processing unit 120 with one of the plurality of line buffers 131b, 131c, and 131d in a round-robin manner at regular intervals.

Meanwhile, the line buffer unit for pooling 135, which constitutes the deep learning processing unit 130, includes a plurality of line buffers 137b, 137c for pooling to store the output of the convolution operation in row units, as shown in FIG. 5. 137d) and,

It includes a multiplexer (MUX, 137a) that connects the output terminal of the convolution processor 133 with one of the plurality of line buffers 137b, 137c, and 137d for pooling in a round-robin manner at regular intervals.

As discussed above, the signal processing device of the optical coherence tomography system according to an embodiment of the present invention is a signal processing device that integrates a deep learning processing unit 130 to apply deep learning inference in processing optical coherence phase difference electrical signals. It is.

In particular, a convolution neural network (CNN) is a type of deep learning neural network, and since it is common to use multiple layers, it is used as a convolution and pooling processor as shown in Figure 3. The signal processing device of the optical coherence imaging system may be configured with a multi-layer structure in which one or more deep learning processing units (130, 230) are connected in series. In particular, interoperability is possible by using an interface through a line buffer.

Hereinafter, the operation of signal processing by integrating deep learning inference into the signal processing device of the optical coherence tomography system will be explained in detail.

Figure 6 is a diagram for explaining the form in which FFT processing results are output in an optical coherence tomography system, Figures 7A and 7B are diagrams for explaining an example of convolution, and Figures 8 and 9 are diagrams of the present invention. FIG. 10 illustrates a line buffer for further explaining the convolution process according to the embodiment, and FIG. 10 illustrates a detailed configuration diagram of the deep learning processing unit 130 to which the activation function is applied.

First, the optical coherence tomography system according to an embodiment of the present invention uses a convolutional neural network (CNN), which basically uses convolution and pooling among deep neural networks, to reduce the computational burden on the computer and improve hardware data processing speed. Integrated and implemented in the processing device.

로 표시)을 유지하게 된다.Referring to FIG. 6, first, the analog optical interference phase difference electrical signal output through the optical system 10 and the detection system 20 is converted into digital data in the form of a stream through the ADC 110 and transmitted to the FFT processing unit. It is output as (120). The FFT processing unit 120 collects and processes a certain number of consecutive data (for example, 1024 samples) for FFT. The FFT processing result is output equal to the number of continuous values input. In addition, the FFT output is an interval equal to the time for performing the FFT, as shown in FIG.

(marked with) is maintained.

As shown in Figures 7a and 7b, a convolutional neural network (CNN) accumulates input data (usually image data, called a feature map) by multiplying it by a filter (or kernel) of a certain size for each element. In the example of Figure 7a, the size of the filter (kernel) is 3x3, so the input data is convolutioned 3x3 to create the result. Figure 7a shows an example of stride 1, and Figure 7b shows an example of stride 2.

For reference, since the padding applied in convolution must be applied as a matter of course, detailed explanation about it will be omitted. In addition, since it is also common to perform a non-linear function or activation function after convolution, the description thereof will also be omitted. Like ReLU (Rectified Linear Unit), if the input value is less than 0, all values are treated as 0 and all values higher than that are output as is, which can be applied relatively simply, as shown in FIG. 10.

Meanwhile, in the examples of FIGS. 7A and 7B, the input data corresponds to the FFT result value (FFT processing result) in FIG. 6, and one row of the input data corresponds to the result of performing the FFT. Therefore, since multiple FFT results must be prepared to apply convolution, the line buffers 131b, 131c, 131d,... and the MUX 131a are applied at the front of the convolution processing unit 133 shown in FIG. 4. This constitutes a signal processing device.

)을 유지하므로, 이 시간 동안 합성곱을 수행한 후 출력되는 새로운 FFT 처리결과(T3 시점)는 도 8의 위 그림에서와 같이 라인 버퍼4에 저장된다. 이러한 경우 도 8의 아래 그림에서와 같이 라인 버퍼의 번호를 변경함으로써 새로운 3x3 합성곱을 적용할 수 있다. 즉, 라인 버퍼를 라운드 로빈(round robin) 방식으로 관리하여 합성곱에 바로 적용할 수 있도록 한다. 스트라이드는 라인 버퍼에서 데이터를 읽을 때 주소를 1 이상 증가하는 방식으로 적용 가능하고, 패딩의 경우는 값이 없거나 정해진 값을 라인 버퍼에 미리 채우는 방식으로 적용 가능하다.Figure 8 is a diagram to explain that FFT processing results output from the FFT processing unit 120 at different times (T0, T1, T2,...) are stored in different line buffers 1, 2, 3, and 4. In other words, if three FFT processing results output at different times are sequentially stored in line buffers 1, 2, and 3 through the MUX 131a, convolution is performed on these three lines (assuming a kernel 3x3). You can. As mentioned earlier, the FFT output is an interval equal to the time for performing the FFT (

) is maintained, so the new FFT processing result (at time T3) output after performing convolution during this time is stored in line buffer 4 as shown in the upper figure of FIG. 8. In this case, a new 3x3 convolution can be applied by changing the number of the line buffer as shown below in FIG. 8. In other words, the line buffer is managed in a round robin manner so that it can be directly applied to convolution. Stride can be applied by increasing the address by 1 or more when reading data from the line buffer, and padding can be applied by filling the line buffer with no value or a set value in advance.

The convolution performed in the convolution processing unit 133 is multiplication and accumulation (MAC) of line buffer information and kernel (filter) information, so it can be processed as shown in FIG. 9.

Meanwhile, if we explain in detail about pooling, which is a representative function constituting a convolutional neural network (CNN), as shown in FIG. 5, the convolution result is transmitted to different line buffers 137b, 137c through the MUX 137a. 137d). The pooling processing unit 137 processes the convolution operation output of a certain size stored in the line buffers 137b, 137c, and 137d through max pooling or average pooling and outputs the output. Even in this case, the multiple line buffers 137b, 137c, and 137d can be managed in a round-robin manner and configured to output information required for pooling. Pooling requires only kernel size information, so there is no need to store the value separately. For reference, the convolution result of a convolutional neural network (CNN) typically applies an activation function to the output. Accordingly, an example of applying ReLU (Retified Linear Unit) is shown in Figure 10.

As described above, in the present invention, the optical interference phase difference electrical signal output through the optical system 10 and the detection system 20 is sequentially converted into digital information and FFT processed, and the result (image information) is used for deep learning inference. By designing the signal processing device of the optical coherence tomography system to be processed through a convolutional neural network and then transmitted to the computer system, the computer system compared to the previous computer system that installed and operated an artificial intelligence inference program to detect defects in the object, etc. There is an advantage in reducing the computational burden and improving data processing speed, and in supporting the function of detecting defects in an object even with a low-specification computer system.

The present invention has been described above with reference to the embodiments shown in the drawings, but these are merely illustrative examples, and those skilled in the art will understand that various modifications and equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined only by the appended claims.

Claims (8)

a converter that converts and outputs an analog optical interference phase difference electrical signal into stream-type digital data;
an FFT processing unit that collects the digital output of the converter in a certain number of units and performs high-speed discrete Fourier transformation on the digital output;
The FFT processing results output from the FFT processing unit at certain times are stored separately as FFT processing results in row units, and kernel information is added to the separately stored FFT processing results of size m*n (m and n are rows and columns). An optical coherence tomography comprising a deep learning processing unit that performs Max pooling or average pooling on the convolution operation output of a certain size, which is created by convolution operation by multiplying and accumulating. Signal processing device of a shooting system. The signal processing device of an optical coherence tomography system according to claim 1, wherein one or more deep learning processing units are connected in series to have a multi-layer structure. The method of claim 1, wherein the deep learning processing unit,
a line buffer unit for convolution that stores FFT processing results output from the FFT processing unit at regular intervals in different line buffers;
a convolution processor that outputs a convolution operation by multiplying and accumulating row-level FFT processing results stored in each of the different line buffers by kernel information;
a line buffer unit for pooling that stores the output of the convolution operation in different line buffers;
A pooling processing unit that processes the row-level convolution operation output results stored in each of the line buffers constituting the pooling line buffer unit through max pooling or average pooling and outputs the results to a direct memory access controller. A signal processing device for an optical coherence tomography system, characterized in that: The method of claim 3, wherein the line buffer unit for convolution,
a plurality of line buffers for storing the FFT processing results in row units;
A signal processing device for an optical coherence tomography system, comprising: a multiplexer connecting the output terminal of the FFT processing unit to one of the plurality of line buffers in a round-robin manner at regular intervals. The method according to claim 3, wherein the line buffer unit for pooling,
a plurality of line buffers for pooling to store the convolution operation output in row units;
A signal processing device for an optical coherence tomography system comprising a multiplexer that connects the output terminal of the convolution processor to one of the plurality of line buffers for pooling in a round-robin manner at regular intervals. An optical coherence tomography system including a converter that converts and outputs an analog-type optical coherence phase difference electrical signal into stream-type digital data, and an FFT processor that collects the digital output of the converter in a certain number of units and outputs the high-speed discrete Fourier transform. In a signal processing device,
a line buffer unit for convolution that stores FFT processing results output from the FFT processing unit at regular intervals in different line buffers;
a convolution processor that outputs a convolution operation by multiplying and accumulating row-level FFT processing results stored in each of the different line buffers by kernel information;
a line buffer unit for pooling that stores the output of the convolution operation in different line buffers;
A pooling processing unit that processes the row-level convolution operation output results stored in each of the line buffers constituting the pooling line buffer unit through max pooling or average pooling and outputs the results to a direct memory access controller. A signal processing device for an optical coherence tomography system, characterized in that: The method of claim 6, wherein the line buffer unit for convolution,
a plurality of line buffers for storing the FFT processing results in row units;
A signal processing device for an optical coherence tomography system, comprising: a multiplexer connecting the output terminal of the FFT processing unit to one of the plurality of line buffers in a round-robin manner at regular intervals. The method according to claim 6 or 7, wherein the line buffer unit for pooling,
a plurality of line buffers for pooling to store the convolution operation output in row units;
A signal processing device for an optical coherence tomography system comprising a multiplexer that connects the output terminal of the convolution processor to one of the plurality of line buffers for pooling in a round-robin manner at regular intervals.

Images