WO2021095826A1 - Image processing device, image processing method, and program - Google Patents

Abstract

According to the present invention, an electric field estimation unit converts an optical coherence tomographic signal representing the state of a sample into a spatial frequency domain and generates electric field data indicating the electric field on a pupil surface corresponding to the reciprocal space of the sample, a mask unit acts on the electric field data with a mask indicating the distribution of the transmission characteristics of the electric field on the pupil surface and generates mask electric field data, and a conversion unit converts the mask electric field data into a spatial region to generate a mask optical coherence tomographic signal. The present embodiment may be any of an image processing apparatus, an image processing method, and a program.

Description

Image processing equipment, image processing methods and programs

The present invention relates to an image processing apparatus, an image processing method and a program.
The present application claims priority with respect to Japanese Patent Application No. 2019-206436 filed in Japan on November 14, 2019, the contents of which are incorporated herein by reference.

Optical coherence tomography (OCT) is a technique for acquiring tomographic images of samples (mainly living organisms) using the coherence of light. According to OCT, it is possible to acquire an image showing not only the surface of a sample but also the internal structure thereof with high spatial resolution. Conventionally, OCT has been put into practical use for retinal diagnosis in ophthalmology. In addition, visualization and quantification of tissue structures below optical resolution and microfiber structure characteristics (for example, statistical properties of directionality and size) using observed tissues such as cultured tissues and ex vivo (ex vivo) samples as samples. It is used for conversion.

When a sample is photographed from a plurality of directions using OCT, an image having a different signal intensity pattern is obtained for each imaging direction. The change in the signal intensity pattern is considered to be due to the fine structure of the tissue constituting the sample. Therefore, as shown in Non-Patent Document 1, a method of indirectly visualizing the characteristics of the fine structure of a tissue by performing multi-directional OCT measurement has been attempted.

However, since the method described in Non-Patent Document 1 realizes multi-directional observation using hardware, the configuration of the device becomes complicated and it is difficult to realize economically. In addition, the measurement time tends to be long because the measurement is performed a plurality of times. Therefore, it is not always realistic to apply it to a living sample. In addition, since the number of measurements is limited, there are restrictions on the measurement target, such as being unsuitable for quantitative observation that requires a large number of measurements.

The present invention has been made in view of the above circumstances, and the subject of the present invention is, for example, to eliminate or relax restrictions on observation such as observation direction, measurement time, number of measurements, measurement target, etc., and more easily sample. It is an object of the present invention to provide an image processing apparatus, an image processing method and a program capable of adjusting the spatial conditions related to the observation of the above.

(1) The present invention has been made to solve the above problems, and one aspect of the present invention is to convert an optical interference tom signal representing the state of a sample into a spatial frequency region and convert it into the reverse space of the sample. An electric field estimation unit that generates electric field data indicating an electric field on the corresponding pupil surface, a mask unit that generates mask electric field data by acting on the electric field data with a mask indicating the passage characteristic distribution of the electric field on the pupil surface, and the above. It is an image processing device including a conversion unit that converts mask electric field data into a spatial region and generates a mask light interference tom signal.

(2) Another aspect of the present invention is the image processing apparatus of (1), and the passing characteristic distribution may indicate a spatial frequency band of an azimuth angle passing through the electric field in the pupil plane.

(3) Another aspect of the present invention is the image processing apparatus of (1) or (2), wherein the passing characteristic distribution indicates a spatial frequency band of a moving diameter passing through the electric field in the pupil plane. May be good.

(4) Another aspect of the present invention is the image processing apparatus of (1) or (2), in which the mask may indicate the presence or absence of passage of the electric field for each sample point corresponding to the spatial frequency. ..

(5) Another aspect of the present invention is the image processing apparatus of (4), wherein the mask is within a predetermined range from the boundary between a passing region that passes through the electric field and a blocking region that does not pass through the electric field. It may have a mask value that changes monotonically between sample points.

(6) Another aspect of the present invention is the image processing apparatus according to any one of (1) to (5), which is a synthesis unit that generates a mask image showing the structure of the sample based on the mask light interference tom signal. May be provided.

(7) Another aspect of the present invention is the image processing apparatus according to any one of (1) to (6), which is based on each of M mask optical interference tomographic signals (M is an integer of 2 or more). The mask unit includes a compositing unit that synthesizes mask images with different display modes to generate a composite image, and the mask unit acts on the electric field data with M masks having different passage characteristic distributions to generate M mask electric field data. The conversion unit may convert the M mask electric field data into a spatial region to generate the M mask optical interference tomographic signals.

(8) Another aspect of the present invention is the image processing apparatus of (7), and the hues of the colors representing the M mask images may be different as the display mode.

(9) Another aspect of the present invention is the image processing apparatus of (7), and the color in which the colors representing the M mask images are combined among the M as the display mode is achromatic. There may be.

(10) Another aspect of the present invention is a method in an image processing apparatus, in which an optical interference tom signal representing a sample state is converted into a spatial frequency region to generate an electric field in the pupil surface corresponding to the reverse space of the sample. An electric field estimation process that generates the electric field data shown, a mask process that generates masked electric field data by acting on the electric field data with a mask showing the passage characteristic distribution of the electric field on the pupil surface, and the masked electric field data in a spatial region. It is an image processing method having a conversion process of converting to generate a masked optical interference tomographic signal.

(11) Another aspect of the present invention is an electric field in which a computer of an image processing apparatus converts an optical interference tom signal representing the state of a sample into a spatial frequency region to indicate an electric field in the pupil surface corresponding to the reverse space of the sample. The electric field estimation procedure for generating data, the mask procedure for generating mask electric field data by acting on the electric field data with a mask showing the distribution of the passing characteristics of the electric field on the pupil surface, and the mask electric field data being converted into a spatial region. This is a program for executing a conversion procedure for generating a masked optical interference tomographic signal.

According to the present invention, the spatial conditions related to the observation of the sample can be adjusted more easily. For example, it is possible to virtually realize the adjustment of the observation direction and the multi-directional observation without the provision or adjustment of the optical component for acquiring the reflected light from each direction.

It is a block diagram which shows an example of the optical interference tomography which concerns on this embodiment. It is a block diagram which shows the functional structure example of the signal processing apparatus which concerns on this embodiment. It is explanatory drawing which shows an example of the image processing which concerns on this embodiment. It is explanatory drawing which shows the other example of the image processing which concerns on this embodiment. It is explanatory drawing which shows the example of the synthesis processing which concerns on this embodiment. It is a figure which shows the 1st example of the composite image which concerns on this embodiment. It is a figure which shows the 2nd example of the composite image which concerns on this embodiment. It is a figure which shows the 3rd example of the composite image which concerns on this embodiment. It is a figure which shows the 4th example of the composite image which concerns on this embodiment. It is a figure which shows the 5th example of the composite image which concerns on this embodiment. It is a figure which shows the 6th example of the composite image which concerns on this embodiment. It is a figure which shows the example of the cross-section OCT image which concerns on this embodiment. It is a figure which shows the 1st example of the composite image with respect to the cross-section OCT image which concerns on this embodiment. It is a figure which shows the 2nd example of the composite image with respect to the cross-section OCT image which concerns on this embodiment. It is a figure which shows another example of the directional mask which concerns on this embodiment. It is a figure which shows the 1st example of the principal direction dependence of a signal strength. It is a figure which shows the 2nd example of the principal direction dependence of a signal strength. It is a figure which shows an example of a sample model. It is a figure which shows the 7th example of the composite image which concerns on this embodiment. It is a figure which shows the 8th example of the composite image which concerns on this embodiment. It is a figure which shows the 9th example of the composite image which concerns on this embodiment. It is a figure which shows the tenth example of the composite image which concerns on this embodiment. It is a figure which shows the estimation example of the tissue structure.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram showing an example of an optical interference tomographic meter 1 according to the present embodiment.
The optical coherence tomography 1 constitutes an observation system for observing the state of a sample using OCT.
The optical interference tomometer 1 irradiates the sample Sm with light, and acquires and acquires the interference light generated by interfering the reflected light reflected from the sample Sm with the reference light reflected by the reference mirror 40 (described later). It is a device that generates an image showing the surface of the sample Sm and the state inside the sample Sm from the interference light.
The object to be observed as the sample Sm may be, for example, a human or animal living body or a non-living body. The living body may be a fundus, a blood vessel, a tooth, a subcutaneous tissue, or the like. The non-living organism may be an artificial structure such as an electronic part or a mechanical part, a natural structure such as a stone or a mineral, or a substance having no specific shape.

The optical interference tomometer 1 includes a light source 10, a beam splitter 20, collimators 30a, 30b, 50a, 50b, a reference mirror 40, a galvanometer mirror 60a, 60b, a spectroscope 70, and an image processing device 100. Among these components, the beam splitter 20, the collimators 30a, 30b, 50a, 50b, the reference mirror 40, the galvanometer mirrors 60a, 60b, and the spectroscope 70 constitute an optical system called an interferometer. The interferometer illustrated in FIG. 1 is a Michelson interferometer including an optical fiber F. More specifically, the light source 10, the spectroscope 70, the collimator 30a, and the collimator 50a are each connected to the beam splitter by using an optical fiber F. The optical fiber F has a transmission band including a wavelength band of light emitted from the light source 10.

In the optical coherence tomography 1, it is a Fourier domain OCT (FD-OCT: Fourier-Domain OCT). In the optical coherence tomography 1, any of Spectral Domain OCT (SD-OCT: Spectral-Domain OCT) and Wavelength Sweep OCT (SS-OCT: Swept-Source OCT), which are classified as FD-OCT, is adopted. You may.

The light source 10 is a wavelength sweep light source such as a very high frequency pulse laser and SLD (Superluminescent Diode). The light source 10 irradiates probe light having, for example, a near-infrared wavelength (for example, 800 to 1000 nm) and having low coherence. The light emitted from the light source 10 is guided inside the optical fiber F and incident on the beam splitter 20.

The beam splitter 20 separates the incident light into light that is guided toward the collimator 30a (hereinafter, reference light) and light that is guided toward the collimator 50a (hereinafter, measurement light). The beam splitter 20 is, for example, a cube beam splitter.

The collimator 30a changes the reference light guided from the beam splitter 20 into parallel light, and emits the parallel light toward the collimator 30b.
The collimator 30b collects the parallel light incident from the collimator 30a and emits the collected reference light toward the reference mirror 40. The collimator 30b incidents the reference light reflected by the reference mirror 40, converts it into parallel light, and emits the converted parallel light toward the collimator 30a.
The collimator 30a collects the parallel light incident from the collimator 30b and guides it toward the beam splitter 20.

On the other hand, the collimator 50a converts the measurement light guided from the beam splitter 20 into parallel light, and emits the converted parallel light toward the galvanometer mirror 60a. On the surfaces of the galvanometer mirrors 60a and 60b, the parallel light incident from the collimator 50a is reflected and emitted toward the collimator 50b, respectively. The collimator 50b collects the parallel light incident from the collimator 50a via the galvanometer mirrors 60a and 60b, and irradiates the sample Sm with the collected measurement light. The measurement light applied to the sample Sm is reflected by the reflecting surface of the sample Sm and incident on the collimator 50b. The reflecting surface is not limited to, for example, the boundary surface between the sample Sm and the surrounding environment (for example, the atmosphere) of the sample Sm, and can be a boundary surface that separates materials or structures having different refractive indexes inside the sample Sm. Hereinafter, the light reflected on the reflecting surface of the sample Sm and incident on the collimator 50b is referred to as reflected light.

The collimator 50b emits the incident reflected light toward the galvanometer mirror 60b. It is reflected on the surfaces of the galvanometer mirrors 60b and 60a, respectively, and emitted toward the collimator 50a. The collimator 50a collects the parallel light incident from the collimator 50a via the galvanometer mirrors 60a and 60b, and guides the collected reflected light toward the beam splitter 20.
The beam splitter 20 guides the reference light reflected by the reference mirror 40 and the reflected light reflected by the sample Sm to the spectroscope 70 via the optical fiber F.

The spectroscope 70 includes a diffraction grating and a light receiving element inside the spectroscope 70. The diffraction grating disperses the reference light and the reflected light guided from the beam splitter 20. The separated reference light and reflected light interfere with each other and become interference light that interferes with each other. The light receiving element is arranged on the imaging surface to which the interference light is irradiated. The light receiving element detects the emitted interference light and generates a signal based on the detected interference light (hereinafter referred to as a detection signal). The light receiving element outputs the generated detection signal to the image processing device 100.

The image processing apparatus 100 acquires an OCT signal representing the state of the sample Sm from the detection signal input from the spectroscope 70. The image processing apparatus 100 sequentially changes the observation points in a predetermined order, and sequentially accumulates the detection signals acquired at the individual observation points or for each of a plurality of observation points forming a predetermined unit, and determines the predetermined ones. Generates an OCT signal in the observation area.
The image processing device 100 converts the generated OCT signal into a spatial frequency domain, and estimates the electric field of the reflected light on the pupil surface corresponding to the reciprocal space of the sample Sm as the pupil surface electric field. The image processing device 100 acts on the electric field data indicating the electric field in which the mask indicating the passing characteristic distribution of the electric field on the pupil surface is estimated to acquire the mask electric field data. The image processing device 100 converts the acquired mask electric field data into a spatial region to generate a mask OCT signal. The image processing device 100 generates a mask OCT image based on the generated mask OCT signal.

The image processing apparatus 100 generates M mask electric field data by acting on the electric field data with M masks having different passage characteristic distributions (M is an integer of 2 or more), and the generated M mask electric field data. May be converted into spatial regions to generate M mask images. The image processing apparatus 100 may generate a composite image by synthesizing the generated M mask images with different display modes (for example, colors).

Next, a functional configuration example of the image processing device 100 according to the present embodiment will be described.
FIG. 2 is a block diagram showing a functional configuration example of the image processing device 100 according to the present embodiment.
The image processing device 100 includes a control unit 110 and a storage unit 190. A part or all the functions of the control unit 110 are realized as a computer including a processor such as a CPU (Central Processing Unit), for example. The processor reads a program stored in the storage unit 190 in advance, performs a process instructed by a command described in the read program, and performs its function. In the present application, performing a process instructed by a command described in a program may be referred to as executing a program, executing a program, or the like. A part or all of the control unit 110 is not limited to general-purpose hardware such as a processor, and may be configured to include dedicated hardware such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

The control unit 110 includes an optical system control unit 120, a detection signal acquisition unit 130, an electric field estimation unit 140, a mask unit 150, a conversion unit 160, an image composition unit 170, and an output processing unit 180.

The optical system control unit 120 drives a drive mechanism that changes the positions of the galvanometer mirrors 60a and 60b, and scans the sample point which is the observation point of the sample Sm. The sample points of the sample Sm are scanned in each of the depth direction of the sample Sm and the direction intersecting the depth direction (for example, the direction parallel to the front surface of the sample Sm). In the following description, the depth direction of the sample may be referred to as the z direction in the three-dimensional Cartesian coordinate system, and the first and second directions orthogonal to the z direction may be referred to as the x direction and the y direction, respectively. Signal acquisition in the depth direction of the sample is called A-scan, and signal acquisition in the direction intersecting that direction (for example, x-direction or y-direction) is called B-scan (B-scan). Sometimes called scan).

The detection signal acquisition unit 130 sequentially acquires detection signals from the spectroscope 70. The detection signal acquisition unit 130 sets signal values indicating the intensity distribution of the reflected light in the depth direction of the sample Sm based on the acquired detection signal for each sample point arranged at predetermined intervals in the three-dimensional space where the sample Sm exists. To get to. The depth direction of the sample Sm corresponds to the incident direction (z direction) of the measurement light. The detection signal acquisition unit 130 repeats the process of acquiring the intensity distribution of the reflected light for each observation point changed by signal acquisition along the plane intersecting in the z direction (for example, the xy plane). The intensity distribution of the acquired reflected light is based on the distribution of the refractive index of the sample Sm in the depth direction. As a result, the detection signal acquisition unit 130 acquires data representing the state of the sample Sm in the observable three-dimensional region (hereinafter, observable region) as a three-dimensional OCT signal (also referred to as an OCT volume). can do. The detection signal acquisition unit 130 stores the three-dimensional OCT signal in the storage unit 190. The planes intersecting in the z direction do not necessarily have to be orthogonal to the z direction and may not be parallel to the z direction. Further, in the two-dimensional planes intersecting in the z direction, the individual sample points do not necessarily have to be arranged on each lattice point of the orthogonal lattice. The individual sample points may be arranged, for example, on each grid point of the diagonal grid or may be arranged aperiodically.
In the above description, signal acquisition involving a change in the target sample point or batch signal acquisition for a plurality of sample points is referred to as "scan", but it is not necessarily a member constituting the optical system (for example, a collimator). It may also include cases without scanning driven by 50b, sample Sm) or detector. In that case, the detection signal acquisition unit 130 is responsible for the function related to signal acquisition, and is omitted in the optical system control unit 120. For example, when performing an A-scan at a certain point on the xy plane using FD-OCT as an OCT method, the detection signal acquisition unit 130 converts reflected light having different frequency components depending on the depth of the sample Sm. Based on the interference light, it is detected as a detection signal. The detection signal acquisition unit 130 can Fourier transform the detection signal to calculate the conversion coefficient for each frequency, and determine the conversion coefficient of the depth associated with the frequency as the signal value at that depth. Further, when scanning by driving in the x-direction or the y-direction is not required, the optical system control unit 120 may be omitted.

The detection signal acquisition unit 130 refers to a portion of the acquired three-dimensional OCT signal that represents the intensity distribution of reflected light in the two-dimensional plane to be observed (hereinafter, the observation target plane) as a two-dimensional OCT signal (hereinafter, simply referred to as an OCT signal). It may be called). The observation target plane is, for example, the front surface of the sample Sm. The front surface is a plane (that is, an xy plane) orthogonal to the depth direction of the sample Sm, and is also called an en face plane. The detection signal acquisition unit 130 stores the extracted OCT signal in the storage unit 190.
The detection signal acquisition unit 130 may select, for example, a two-dimensional plane (for example, the xy plane) of the depth (that is, z coordinate) indicated by the control signal input from the output processing unit 180 as the observation target plane. ..
In this embodiment, it is desirable that the distance between adjacent sample points in the observable region is equal to or less than the spatial resolution of the optical system.

The electric field estimation unit 140 reads the OCT signal stored in the storage unit 190, performs a two-dimensional Fourier transform on the read OCT signal in the spatial domain, and generates data in the spatial frequency domain. The data in the spatial frequency domain obtained by the Fourier transform corresponds to the data (hereinafter, electric field data) indicating the electric field (hereinafter, pupillary electric field) in the pupil plane (pupil plane). That is, the electric field of the reflected light projected on the pupil surface is estimated by performing a two-dimensional Fourier transform on the OCT signal related to the observation target plane. Further, in general, the conversion coefficient for each frequency sample point (frequency bin) converted into the spatial frequency domain is a complex number, and is represented by its absolute value and argument. Therefore, in the electric field data, the amplitude intensity and the phase are represented by the absolute value and the declination of each conversion coefficient. The electric field estimation unit 140 stores the generated data in the spatial frequency region as electric field data in the storage unit 190.

As illustrated in FIG. 3, the pupil plane Pp corresponds to the back-focal plane corresponding to the reciprocal space of the sample Sm. That is, the pupil surface Pp is a surface arranged on the opposite side of the sample Sm from the objective lens Ol, and the reflected light diffused from the sample Sm and parallelized by the objective lens Ol is virtually projected. This is the surface. The entire band of the spatial frequency converted into the spatial frequency domain corresponds to the observation region of the electric field to be observed on the pupil surface. The intersection of the optical axis of the objective lens Ol and the pupil surface corresponds to the origin of the spatial frequency. Spatial frequencies in the spatial frequency domain correspond to sample points in the observation domain. The sample Sm is installed at a position where it intersects with the optical axis of the objective lens Ol.

Returning to FIG. 2, the mask unit 150 reads out the electric field data stored in the storage unit 190, applies a mask to the read electric field data, and displays the masked electric field data indicating the masked electric field. Generate. The mask is numerical data showing the passing characteristic distribution of the electric field of the reflected light in the spatial frequency domain. As the mask, a numerical value indicating the passing characteristics of the electric field is shown as a mask value for each frequency sample point corresponding to each spatial frequency. For example, the mask value for each frequency sample point takes one of two values, 1 and 0, and 1 and 0 indicate the presence or absence of passage of an electric field, respectively. Therefore, the distribution of frequency sample points with the mask value set to 1 indicates the pass band of the spatial frequency domain passing through the electric field in the reciprocal space of the sample Sm. In another aspect, the passband represents a virtual aperture on the pupil plane through which the reflected light passes. On the other hand, the distribution of frequency sample points with the mask value set to 0 indicates the cutoff band in the spatial frequency region that does not pass through the electric field in the reciprocal space of the sample Sm. In another aspect, the blocking band represents a virtual shield on the pupil surface that does not allow reflected light to pass through.

When the mask is applied, the mask unit 150 multiplies the electric field value for each frequency sample point indicated by the electric field data by the mask value of the frequency sample. The mask unit 150 generates data having the multiplication value obtained by multiplication as the mask electric field value for each frequency sample point as the mask electric field data. The masked electric field data is used to generate an OCT image that reflects spatial characteristics (for example, directivity) corresponding to the passage characteristic distribution, as will be described later. The mask unit 150 stores the generated mask electric field data in the storage unit 190. Specific examples of masks will be described later.

The conversion unit 160 reads out the mask electric field data stored in the storage unit 190, performs a two-dimensional inverse Fourier transform on the read mask electric field data in the spatial frequency domain, and generates a mask OCT signal in the spatial domain. The conversion unit 160 stores the generated mask OCT signal in the spatial region in the storage unit 190.
When M masks having different electric field passing characteristic distributions in the spatial frequency region are set, the mask unit 150 acts on each of the M masks on the read electric field data, and M A number of masked electric field data may be generated. In that case, the conversion unit 160 converts each of the M mask electric field data in the spatial frequency domain into a mask OCT signal in the spatial domain.

The image synthesizing unit 170 reads out the mask OCT signal stored in the storage unit 190, and uses a predetermined conversion function for the signal value for each sample point in the observation target plane indicated by the read mask OCT signal to obtain the brightness for each pixel. Convert to a value. The converted luminance value takes a value within a value range that can be expressed by the bit depth of each pixel. The image synthesizing unit 170 generates mask image data having a luminance value converted for each sample point. The generated mask image data shows a mask image that reflects the spatial characteristics corresponding to the passage characteristic distribution.
When the number of masks used for the mask of the pupillary electric field is one, the image synthesizing unit 170 uses the generated mask image data as output image data according to the control signal input from the output processing unit 180. Output to the display (not shown).

When the mask OCT signals corresponding to the M masks are acquired, the image synthesizing unit 170 generates a mask image having a different display mode (for example, color) for each of the M mask OCT image signals. ..
The image synthesizing unit 170 synthesizes the generated M mask images to generate composite image data indicating one composite image. For example, when generating a color composite image using an RGB color system, the image synthesis unit 170 determines the average value of the pixel values for each pixel among the M mask images as the pixel value of the composite image. .. In the RGB color system, the hue (chromaticity) is determined by the ratio of the pixel values of the pixels representing the primary colors red, green, and blue for each pixel block in which the shade is set. The image composition unit 170 outputs the generated composite image data to the display unit as output image data according to the control signal input from the output processing unit 180.
The image synthesizing unit 170 may store the output image data in the storage unit 190 according to the control signal input from the output processing unit 180.

The output processing unit 180 controls the generation or output of output image data indicating an OCT image based on an operation signal input from an operation input unit (not shown).
The operation input unit may include, for example, a button, a knob, a dial, a mouse, a joystick, and other members that accept user operations and generate operation signals according to the received operations. The operation input unit may be an input interface that receives an operation signal wirelessly or by wire from another device (for example, a portable device such as a remote controller).

The operation signal indicates, for example, the necessity of displaying or storing the OCT image, the observation target plane as the observation target area, the spatial frequency characteristic of the mask, and the like as parameters. The output processing unit 180 may display a parameter that can be set by operation, a parameter setting, and a setting screen for guiding the parameter that can be set on the display unit, and configure a user interface related to the display of the OCT image. Good.
For example, when an operation signal indicating the necessity of displaying an OCT image is input, the output processing unit 180 outputs a control signal indicating the necessity of displaying the OCT image to the image synthesis unit 170. The image synthesizing unit 170 outputs output image data to the display unit when a control signal indicating display necessity is input from the output processing unit 180, and outputs an output image when a control signal indicating display / rejection is input from the output processing unit 180. Do not output data to the display.

When the operation signal indicating the observation target plane is input, the output processing unit 180 outputs the control signal indicating the observation target plane to the detection signal acquisition unit 130. The detection signal acquisition unit 130 outputs a portion of the three-dimensional OCT signal related to the observation target plane indicated by the control signal input from the output processing unit 180 as an OCT signal. The observation target plane is defined, for example, as parameters such as the depth from the surface of the sample Sm, the observation direction, and the area of the observation area.
When an operation signal indicating the spatial frequency characteristic of the mask is input, the output processing unit 180 outputs a control signal indicating the spatial frequency characteristic to the mask unit 150. The output processing unit 180 may set the spatial frequency characteristics of each of the plurality of masks based on the input operation signal, and output a control signal indicating the set spatial frequency characteristics to the mask unit 150. The mask unit 150 sets a mask having spatial frequency characteristics indicated by a control signal input from the output processing unit 180.

In addition to the above program, the storage unit 190 stores various data used for processing executed by the control unit 110 and various data acquired by the control unit 110.
The storage unit 190 includes, for example, a non-volatile (non-temporary) storage medium such as a ROM (Read Only Memory), a flash memory, and an HDD (Hard Disk Drive). The storage unit 190 includes, for example, a volatile storage medium such as a RAM (Random Access Memory) and a register.

(Image processing)
Next, an example of image processing according to the present embodiment will be described. FIG. 3 is an explanatory diagram showing an example of image processing according to the present embodiment. The image processing exemplified in FIG. 3 is an application example to directional imaging of an OCT image. In directional imaging, a mask having directivity in the spatial frequency domain (hereinafter, directional mask) is used. The directional mask has a mask value with a dependency on a spatial frequency (hereinafter, azimuth spatial frequency) corresponding to an azimuth from the origin in a two-dimensional spatial frequency domain.

(Step S01) The electric field estimation unit 140 performs a two-dimensional Fourier transform on the OCT signal indicating the front OCT image Oi01 of the spatial region acquired by the detection signal acquisition unit 130, and performs a two-dimensional Fourier transform on the pupil in the spatial frequency region indicating the pupil surface electric field Pe01. Generate surface electric field data. The front OCT image Oi01 is an OCT image with the observation target plane as the front.
(Step S02) The mask unit 150 acts on the pupillary surface electric field data generated by the electric field estimation unit 140 as an example of a virtual mask with the directional mask Dm01 to generate mask electric field data indicating the mask electric field Ie01.
In the directional mask Dm01, the spatial frequency (hereinafter referred to as the radial spatial frequency) corresponding to the distance (driving diameter) from the origin is within a predetermined band, and the azimuth angle spatial frequency is −π / 2 to 0. The half-circle band up to π / 2 is used as the pass band, and the other bands are used as the cutoff band.

(Step S03) The conversion unit 160 performs a two-dimensional inverse Fourier transform on the mask electric field data generated by the mask unit 150 to generate a mask OCT signal in the spatial region.
The image synthesizing unit 170 converts the signal value for each sample point indicated by the mask OCT signal into a pixel value for each pixel, and generates mask image data having the pixel value obtained by the conversion.
The image composition unit 170 outputs the generated mask image data as output image data to a display unit (not shown).

As described above, by acting the directional mask Dm01 on the pupillary surface electric field Pe01, the pupillary surface in the spatial frequency band within the range of the bandwidth π centered on the azimuth spatial frequency 0 (rad) (main direction). The electric field is passed through, and the pupillary electric field in other bands is cut off. As a result, an OCT image having directivity with the azimuth spatial frequency 0 (rad) as the main direction can be obtained by the obtained mask electric field Ie01.

Further, by making the spatial frequency characteristic of the mask variable according to the operation, the user can arbitrarily adjust the directivity of the OCT image by using the acquired OCT signal without driving or adjusting the optical system. Can be done. For example, the output processing unit 180 can set one or both of the azimuth space frequency band and the radial space frequency band as the pass band through which the electric field is passed based on the operation signal input from the operation input unit (not shown). And. The mask unit 150 sets a mask value at a frequency sample point in the pass band set by the output processing unit 180 as 1, a mask value related to other frequency sample points as 0, and sets a directional mask indicating the set mask value. Should be generated.

FIG. 4 is an explanatory example showing another example of image processing according to the present embodiment. An application example to virtual multi-directional imaging will be described with reference to FIG.
In multidirectional imaging, in step S02 (FIG. 3), the mask unit 150 generates mask electric field data using M directional masks having different directivities in the spatial frequency domain with respect to a common pupillary electric field. To do. In the example shown in FIG. 4, three directional masks Dm01-Dm03 are used. The pass bands of the directional masks Dm01 to Dm03 are set in 2π / 3 cycles and do not overlap each other in the spatial frequency domain.

In step S03 (FIG. 3), the conversion unit 160 performs a two-dimensional inverse Fourier transform on each of the generated M mask electric field data to generate a mask OCT signal in the spatial region. The image synthesizing unit 170 converts each of the generated M mask OCT signals in the spatial region into a mask OCT image.
The image synthesizing unit 170 generates output image data indicating composite images displayed in different colors for the converted M mask OCT images, and outputs the generated output image data to the display unit. Therefore, a composite image in which OCT images having different directivities are multiplexed with different colors is displayed.
Here, the output processing unit 180 sets each of the M masks (M is an integer of 2 or more) as a pass band through which the electric field is passed, based on the operation signal input from the operation input unit (not shown). One or both of the azimuth space frequency band and the radial space frequency band may be set. In that case, the mask unit 150 sets the mask value of the frequency sample points in the pass band set by the output processing unit 180 to 1 for each mask, and sets the mask value of the other frequency sample points to 0.

Next, an example of compositing processing for mask OCT images (hereinafter, directivity images) to which M different directivities are given will be described.
FIG. 5 is an explanatory diagram showing an example of the synthesis process according to the present embodiment.
(Step S11) The image synthesizing unit 170 colors M directional images with different colors to generate colored image data indicating the colored images. In the example shown in FIG. 5, the image synthesizing unit 170 colors the two directional images Di11 and Di12 with different colors (for example, red and green) to generate colored images Ci11 and Ci12. In FIG. 5, each color is represented by an upward-sloping diagonal line and a downward-sloping diagonal line. The directional images Di11 and Di12 are OCT images generated by using directional masks having directivity 1 and 2, respectively. Directivity 1 indicates a pass band having an azimuth angle of 7 / 6π as a main direction and a bandwidth of π. Directivity 2 indicates a pass band having an azimuth angle of -5 / 6π as a main direction and a bandwidth of π.

(Step S12) The image synthesizing unit 170 synthesizes the colored images indicated by the M colored image data to generate the composite image data indicating the composite image. In the example shown in FIG. 5, the colored colors of the colored images Ci11 and Ci12 are mixed for each part. The composite image Si11 represents a mixed color obtained by mixing each part. The diagonally shaded portion indicates a mixed color obtained by mixing the colored colors in the colored images Ci11 and Ci12, respectively. The upward-sloping oblique line portion and the downward-sloping oblique line portion indicate a portion colored in the colored images Ci11 and Ci12, respectively, but not colored in the colored images Ci12 and Ci11.

Therefore, the user who comes into contact with the displayed composite image can intuitively grasp the distribution of the state of the sample corresponding to the directivity in the observation area of the sample by the hue. By using hue as the display mode, it is possible to express a finer difference in directivity distribution than when distinguishing by other types of display modes, for example, a pattern such as shading. Hue generally means a hue represented by a component of a specific remarkable wavelength in image light, and is one of the three attributes of color. It is desirable that the hues of the colors to be colored differ as much as possible in the color space between the M mask OCT images. When M is 2, the color of one mask OCT image may be a complementary color to the color of the other mask OCT image. For example, when the color of one mask OCT image is red, the color of the other mask OCT image may be green. When M is 3, for example, the colors of the three mask OCT images may be the three primary colors of red, green, and blue.

Further, the color obtained by simply mixing the colors of the M mask OCT images among the M masks may be set to be achromatic. An achromatic color is a desaturated color, that is, white, black, or gray of various densities. When M is 2, if the color of one mask OCT image is a complementary color to the color of the other mask OCT image, mixing the respective colors results in an achromatic color. When M is 3, when the colors of the three mask OCT images are the three primary colors of red, green, and blue, mixing the respective colors results in an achromatic color.
As a result, the difference in brightness between different directivities is expressed depending on the color of the composite image. Therefore, the user who comes into contact with the composite image can easily recognize that the portion where a specific color is prominently represented has a tissue structure corresponding to the directivity associated with the color.

The image synthesizing unit 170 may generate average image data having an average value obtained by averaging the pixel values indicated by each of the M mask OCT image data as pixel values. Then, the image synthesizing unit 170 generates average colored image data showing a new colored image (hereinafter, average colored image) colored with a color different from any of the M colored images with respect to the average image data.
The image synthesizing unit 170 synthesizes the colored image shown by the M colored image data and the average colored image shown by the average colored image data to generate the composite image data showing the composite image. For example, when M is 2, the image synthesizing unit 170 sets the colors of the two colored images to red and green, and sets the color of the average colored image to blue.
As a result, the parts in which the states corresponding to the respective directivities are prominent due to the colors remarkably displayed in the composite image are represented by comparing their average values.

The compositing mode related to the generation of the compositing image is not limited to this. The composite mode refers to a combination of spatial frequency characteristics of the mask used for generation for each of the mask OCT images (for example, directional images) used for generating the composite image.
6A-6F are diagrams showing an example of a composite image for each composite mode according to the present embodiment. An example is taken when the front OCT image Oi01 is used as a common OCT image between FIGS. 6A and 6F in generating the composite image.

The low-frequency image Li11 shown in FIG. 6A is a composite image generated by using the composite mode 11. In the synthesis mode 11, the azimuth angle 7 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the passband having the bandwidth π, the azimuth angle -5 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a low frequency range from frequency 1 to frequency 2 as a passing band and other radial spatial frequencies as a cutoff band. Frequency 1 is a predetermined radial space frequency that is sufficiently close to 0 or 0. The frequency 2 is a predetermined radial space frequency higher than the frequency 1 and lower than the upper limit of the radial space frequency. The upper limit of the radial space frequency is determined by the sampling theorem based on the sample point spacing of the OCT signal. Since the low-frequency image Li11 is generated by using the low-frequency component of the pupillary electric field, it is configured to include a pattern that is coarser and has an unclear outline.

The high-frequency image Hi11 shown in FIG. 6B is a composite image generated by using the composite mode 12. In the synthesis mode 12, the azimuth angle 7 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the pass band having the bandwidth π, the azimuth angle -5 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a high frequency range from frequency 2 to frequency 3 as a passing band and other radial spatial frequencies as a cutoff band. The frequency 3 is a predetermined low radial space frequency that is higher than the frequency 2 and equal to or less than the upper limit of the radial space frequency. Since the high-frequency image Hi11 is generated by using the high-frequency component of the pupillary electric field, it is configured to include a relatively fine pattern.

The composite image Si11 shown in FIG. 6C is a composite image generated by using the composite mode 13. In the synthesis mode 13, the azimuth angle 7 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the passband having the bandwidth π, the azimuth angle -5 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic are not provided with band limitation in the radial spatial frequency region. That is, the synthesis mode 13 is the same as the synthesis mode including the directivity 1 and 2 illustrated in FIG. 5, and has both the pass band of the synthesis mode 11 and the pass band of the synthesis mode 12.

The low-frequency image Li12 shown in FIG. 6D is a composite image generated by using the composite mode 14. In the synthesis mode 14, the azimuth angle of 5 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the passband having the bandwidth of π, the azimuth angle of -7 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a low frequency range from frequency 1 to frequency 2 as a passing band and other radial spatial frequencies as a cutoff band.
Since the low-frequency image Li12 is generated by using a mask whose main direction of the pass band is different from that of the low-frequency image Li11, it represents a pattern different from that of the low-frequency image Li11. The low-frequency image Li12 shows a pattern that is relatively coarse and has an unclear outline.

The high-frequency image Hi12 shown in FIG. 6E is a composite image generated by using the composite mode 15. In the synthesis mode 15, the azimuth angle of 5 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the passband having the bandwidth of π, the azimuth angle of -7 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a high frequency range from frequency 2 to frequency 3 as a passing band and other radial spatial frequencies as a cutoff band. The high-frequency image Hi12 represents a pattern more similar to the low-frequency image Li12 having a common passband azimuthal spatial frequency than the high-frequency image Hi11 having a common passband spatial frequency of the mask used for generation. .. The high-frequency image Hi12 shows a finer pattern than the low-frequency image Li12.

The composite image Si12 shown in FIG. 6F is a composite image generated by using the composite mode 16. In the synthesis mode 16, the azimuth angle 7 / 6π is the main direction, and the first spatial frequency characteristic showing the directionalness indicating the passband having the bandwidth π, the azimuth angle -5 / 6π is the main direction, and the bandwidth is It is composed of a second spatial frequency characteristic having a directivity indicating a pass band of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic are not provided with band limitation in the radial spatial frequency region. That is, the synthesis mode 16 has both the pass band of the synthesis mode 14 and the pass band of the synthesis mode 15.

Note that, in the directivity patterns shown in FIGS. 5 and 6A to 6F, a case where a part or all of the pass bands related to each of the plurality of masks do not overlap is taken as an example, but the present invention is not limited to this. A part of the pass band may overlap between at least two masks among the plurality of masks. Further, the entire pass band of each of the plurality of masks does not necessarily cover the entire spatial frequency band.

In addition, FIG. 3 to FIG. 6F exemplify the case where the observation target plane is a front surface perpendicular to the depth direction of the sample Sm, but if it is a surface that crosses a part or all of the observation target area, this may be used. Not limited.
Therefore, the output processing unit 180 may be able to set the direction of the observation target plane by the user's operation. More specifically, an operation signal indicating the direction of the observation target plane is input from the operation input unit (not shown), and a control signal indicating the direction of the observation target plane indicated by the operation signal is output to the detection signal acquisition unit 130. .. The detection signal acquisition unit 130 determines the observation target plane in the direction indicated by the control signal input from the output processing unit 180.

7A-7C show a display example when the observation target plane is a cross section parallel to the depth direction of the sample Sm.
The cross-section OCT image Cs21 exemplified in FIG. 7A is an OCT image showing the state of the sample Sm in the cross-section parallel to the depth direction. The cross-sectional OCT image Cs21 represents a pattern different from the front OCT image Oi01.
The composite image Si21 exemplified in FIG. 7B is a composite image generated by using the composite mode 13 with respect to the OCT signal related to the cross-sectional OCT image Cs21. The diagonally shaded portion indicates a portion commonly extracted by using a mask having a first directivity and a second directivity constituting the synthesis mode 13. The upward-sloping diagonal line portion and the downward-sloping diagonally-lined portion are extracted by the mask having the first directivity and the second directivity, respectively, but the mask having the second directivity and the first directivity. Indicates the part that is not extracted. The uncolored portion indicates a portion that is not extracted by either the mask having the first directivity or the second directivity.
The composite image Si22 exemplified in FIG. 7C is a composite image generated by using the composite mode 16 with respect to the OCT signal related to the cross-sectional OCT image Cs21. The composite image Si22 represents a pattern different from that of the composite image Si21.

Next, another example of the directional mask will be described. The directional mask may have a narrower bandwidth with azimuth spatial frequencies. The composite mode of the directional mask Dm31 illustrated in FIG. 8 has two pass bands. The bandwidths of the two passbands are π / 9, respectively. However, the main directions of the respective pass bands are azimuth angles π / 2 and −π / 2, respectively, and are directed in opposite directions. By using a directional mask with a narrow bandwidth, it is possible to extract the components of the azimuth spatial frequency around the main direction with the main direction as the target direction. Such a directional mask can be applied to the analysis of the structure of a sample.

The mask unit 150 rotates, for example, the main direction of the passing band in the direction of the azimuth at minute intervals (for example, 1 ° to 5 °), and each of the rotated main directions is directional to the pupillary electric field data. The mask is applied to generate mask electric field data. The conversion unit 160 performs a two-dimensional inverse Fourier transform on the generated individual mask electric field data to generate a mask OCT signal in the spatial region. Even if the control unit 110 includes an analysis unit (not shown) that analyzes the mask OCT signal for each rotated main direction or the change characteristic of the intensity of a part or all of the mask image data based on the mask OCT signal. Good.

FIGS. 9 and 10 exemplify the main direction dependence of the signal intensity obtained using the bovine Achilles tendon and the chicken breast muscle as samples, respectively. The vertical axis and the horizontal axis indicate the signal strength and the main direction, respectively. As the signal strength, the total pixel strength of the mask image data was used. The pixel strength corresponds to the total value of the signal values for each pixel. 9 and 10 show that the signal strength depends on the main direction and the signal strength has a significant maximum value, respectively. In the examples shown in FIGS. 9 and 10, the main directions in which the maximum values are taken are 152 ° and 116 °, respectively. Further, the maximum value of the signal strength is about four times the average value. This suggests that the tissue structure used as a sample has a high periodicity with respect to the main direction in which the signal intensity reaches the maximum value.

This is also supported by the following simulation. In the simulation, it was assumed that a light ray from the z direction was incident on the sample model Sp shown in FIG. The surface of the sample model Sp is a triangular wave whose height in the z direction does not depend on the y direction and has a constant period in the x direction. Then, the OCT volume is set for each of the cases where the z coordinate of the sample model Sp is farther (Δz> 0) and closer (Δz <0) than the reference point (the focal point of the optical system, which corresponds to the focal point of the collimator 50b in FIG. 1). Synthesized. A directional mask was applied to the pupillary electric field data based on the OCT image signals of the front image and the cross-sectional image obtained from each OCT volume to generate mask electric field data for each pass band. However, as the directivity mask, a synthesis mode 32 (FIG. 12A) having a passing region having a main direction of π and a bandwidth of π and a passing region having a main direction of 0 and a bandwidth of π was used. Then, a two-dimensional inverse Fourier transform was performed on the mask electric field data generated for each pass band, and the mask OCT signal in the spatial region was used to generate OCT image data showing each one composite image.

12A and 12B show a composite image for the front image and a composite image for the cross-sectional image generated on the assumption that Δz> 0, respectively. Both FIGS. 12A and 12B show patterns according to the period of height in the x direction. However, in FIG. 12A, the brightness is constant in the y direction, and there is a portion in the x direction in which the brightness is higher than the periphery in the negative direction and the positive direction from the center of each cycle. The negative and positive directions from the center of each cycle correspond to the passbands whose main directions are π and 0, respectively. In FIG. 12B, a portion of the surface of the sample model Sp that is proportional to the height in the z direction has a portion having a higher brightness than the surroundings. 12C and 12D show a composite image for the front image and a composite image for the cross-sectional image generated on the assumption that Δz <0, respectively. The patterns of the composite images shown in FIGS. 12C and 12D are symmetrical with the patterns of the composite images shown in FIGS. 12A and 12B, respectively.

Therefore, the height of the interface between the bovine Achilles tendon and the chicken pectoralis major used as samples in the examples of FIGS. 9 and 10 is high in one direction (x direction) intersecting the height direction (z direction), respectively. It is confirmed that the structure has periodicity and is less dependent on other directions (y direction) (see FIG. 13). Then, the image of the surface inclined in the negative direction of the period and the image of the surface inclined in the positive direction are associated with the main directions of the individual pass bands, and are displayed in the composite image in different patterns. Will be done.

In the above example, the case where the number M of one directional mask is mainly one or two has been described, but the present invention is not limited to this. The number M of the masks may be 3 or more, for example, 60, 120, 360, and the like. As the number of masks increases, the passband bandwidth for each mask can be finer. The bandwidth is not limited to the azimuth space frequency, and may be fixed or variable with respect to the azimuth space frequency or the combination of the azimuth space frequency and the azimuth space frequency, and may be further subdivided. The number of masks may be large enough that the individual passbands can be considered spatially continuous. The passbands of each of the M masks do not necessarily have to be periodically, regularly or exhaustively distributed in the spatial frequency domain, and may be randomly or intermittently distributed.

In the above example, the mask value for each spatial frequency corresponding to each frequency sample point constituting the mask is mainly either 0 or 1, but the case is not limited to this. The mask value may be any real number. This makes it possible to set the degree of passage of the pupillary electric field for each spatial frequency in more detail. For example, if the mask value in the pass band is larger than the mask value in the cutoff band, the dependence of the mask OCT signal by the mask can be emphasized or relaxed rather than being limited to 1 and 0, respectively. In addition, by setting the mask value so that it changes monotonically between the sample points corresponding to the spatial frequency within a predetermined range from the boundary between the pass band and the cutoff band, the mask according to the spatial frequency around the boundary is set. The change in value can be more gradual. Therefore, it is possible to alleviate an abnormal spatial change in the abnormal brightness of the mask OCT image due to aliasing noise that may occur in the mask OCT signal.

The mask value for each spatial frequency is not limited to a real number but may be a complex number. The mask value may be, for example, a value obtained by further dividing the above mask value by a coefficient (complex number) indicating the phase and amplitude for each spatial frequency representing the aberration generated in the optical system and the sample to be measured. By using such a mask value, aberrations generated in the optical system and the sample to be measured are compensated.
The generation of mask electric field data and thus the mask OCT signal using the mask described above can be realized by complex number calculation in the spatial frequency domain. In other words, it cannot be achieved by existing hardware such as optical components or image processing that handles real luminance values or color signal values.

The control unit 110 may further include an analysis unit (not shown) that analyzes the spatial frequency characteristics of the OCT signal. For example, the analysis unit causes the processing of steps S01 to S03 to be performed using a mask having an azimuthal spatial frequency band having a predetermined bandwidth as a pass band, calculates the power of the generated mask OCT signal, and processes the processing target. The process of changing the main direction of the pass band may be repeated. As a result, the analysis unit can acquire the azimuth spatial frequency dependence of the power and specify the azimuth spatial frequency at which the power is maximum, maximum, minimum, or minimum. The analysis of the signal strength illustrated in FIGS. 9 and 10 using the directional mask illustrated in FIG. 8 corresponds to the analysis example of the azimuth spatial frequency.
The analysis unit may calculate the power of the mask OCT signal instead of the azimuth space frequency band or for each pass band determined by the azimuth space frequency band together with the azimuth space frequency band. As a result, the analysis unit can acquire the spatial frequency dependence of the power and specify the spatial frequency at which the power becomes the maximum, the maximum, the minimum, or the minimum.
The analysis unit may execute the process of calculating the power for each part of the observation target plane indicated by the OCT signal, instead of the entire plane. The analysis unit can acquire the spatial frequency dependence of power for each part, and can specify the spatial frequency at which the power is maximum, maximum, minimum, or minimum for each part. Further, the analysis unit can determine a range of power for each spatial frequency that can be taken for each site, and can identify a site whose spatial frequency dependence is more remarkable than a predetermined range.
The output processing unit 180 outputs information indicating at least one of the powers calculated by the analysis unit, the specified spatial frequency, or the part to the display unit or other equipment according to the operation signal input from the operation input unit. You may.

As described above, the image processing apparatus 100 according to the above-described embodiment converts an optical interference tom signal representing the state of the sample into a spatial frequency domain and shows an electric field in the pupil surface corresponding to the reciprocal space of the sample. The electric field estimation unit 140 for generating data is provided. Further, the image processing apparatus 100 acts on the electric field data with a mask showing the distribution of the passing characteristics of the electric field on the pupil surface to generate the mask electric field data, and converts the mask electric field data into a spatial region to convert the mask light into a space region. It includes a conversion unit 160 that generates an interference fault signal.
With this configuration, by acting a mask on the electric field on the pupil surface, it is possible to acquire a mask optical tomography signal that reflects the passage characteristic distribution indicated by the mask. Further, the image synthesizing unit 170 can acquire a masked optical tomographic image by converting the signal value of the acquired masked optical tomographic signal into a luminance value. Therefore, even if the optical system is not actually equipped with various optical components, the distribution of the passing characteristics of the electric field according to the spatial conditions related to the observation of the sample can be virtually adjusted by the action of the mask.

Further, the passage characteristic distribution of the mask may indicate the spatial frequency band of the azimuth angle passing through the electric field on the pupil surface.
With this configuration, the spatial frequency band of the azimuth angle as the pass band of the electric field is indicated as the pass characteristic distribution of the mask, so that the observation of the sample having the directivity of the observation direction according to the pass band is realized. Further, by acquiring a mask image showing the structure of the sample based on the mask, it is possible to eliminate or relax the restriction on the observation direction and visualize the anisotropy of the fine structure of the structure constituting the sample. ..

Further, the passage characteristic distribution of the mask may indicate the spatial frequency band of the moving diameter passing through the electric field on the pupil surface.
With this configuration, the spatial frequency band of the moving diameter, which is the pass band of the electric field, is indicated as the pass characteristic distribution of the mask, so that the observation of the sample having the spatial resolution according to the pass band is realized.

Further, the mask may indicate the presence / absence of passage of an electric field for each sample point corresponding to the spatial frequency.
With this configuration, the spatial frequency band through which the pupillary electric field passes is defined as digital data, so that the calculation related to the action of the mask can be simplified.

Further, the image processing apparatus 100 according to the present embodiment may include an image synthesizing unit 170 that generates a composite image by synthesizing mask images based on each of the M masked optical interference tomographic signals in different display modes. Good. However, the mask unit 150 acts on the electric field data with M masks having different passage characteristic distributions to acquire the M mask electric field data. The conversion unit 160 converts each of the M masked electric field data into a spatial region to generate M masked optical interference tomographic signals.
With this configuration, the user who comes into contact with the composite image can intuitively identify each mask image based on the display mode appearing in the composite image. By comparing the individual mask images, the user can intuitively grasp the dependence of the sample state due to the passage characteristic distribution (for example, directivity).

Further, as a display mode, the hues of the colors representing the M mask images may be different.
With this configuration, the user can intuitively identify each mask image based on the color appearing in the composite image, and easily grasp the part where the state of the sample corresponding to the passage characteristic distribution (for example, directivity) appears. be able to.

Further, the color in which the colors representing the M mask images are combined between the M mask images is achromatic.
With this configuration, the user can identify the part where the state of the sample corresponding to the passage characteristic distribution (for example, directivity) corresponding to the color appearing in the composite image appears more prominently than other transmission characteristic distributions, and the passage depends on the shading. It is possible to recognize the state of the sample regardless of the characteristic distribution.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to do.

For example, in the above description, the case where the image processing device 100 is a part of the optical interference tomographic meter 1 is taken as an example, but the present invention is not limited to this. The image processing device 100 may be a single device that is independent of the optical interference tomometer 1 and does not have an optical system. In that case, the optical system control unit 120 may be omitted in the control unit 110 of the image processing device 100. The detection signal acquisition unit 130 is not limited to the optical system, and may acquire a detection signal or an OCT signal from another device such as a data storage device or a PC by wire or wirelessly, for example, via a network.

Further, the image processing device 100 may include the above-mentioned operation input unit and display unit, or one or both of them may be omitted.
In the control unit 110 of the image processing device 100, one or both of the image composition unit 170 and the output processing unit 180 may be omitted. When the image composition unit 170 is omitted, the conversion unit 160 may output the generated mask OCT signal to another device such as a data storage device, a PC, or another image processing device. The device as the output destination has the same function as the image synthesizing unit 170, that is, generates output image data based on the mask OCT signal input from the image processing device 100, and displays an image based on the generated output image data. It may have a function.
The color system adopted by the image composition unit 170 is not necessarily limited to the RGB color system, and may be another color system, for example, a YCbCr color system.
The display mode of each of the M mask images is not limited to hue and gradation, and other methods such as patterns such as halftone dots, grids, and diagonal lines, brightness such as blinking, and time change of hue may be used.

When a part of the image processing device 100 in the above-described embodiment, for example, all or part of the control unit 110 is realized by a computer, a program for realizing this control function is recorded on a computer-readable recording medium. Then, the program recorded on the recording medium may be read into a computer system and executed. The "computer system" referred to here is a computer system built in the image processing device 100, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Furthermore, a "computer-readable recording medium" is a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Further, a part or all of the image processing device 100 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image processing apparatus 100 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

1 ... Optical interference tomography, 10 ... Light source, 20 ... Beam splitter, 30a, 30b, 50a, 50b ... Collimator, 40 ... Reference mirror, 60a, 60b ... Galvano mirror, 70 ... Spectrometer, 100 ... Image processing device, 110 ... Control unit, 120 ... Optical system control unit, 130 ... Detection signal acquisition unit, 140 ... Electric field estimation unit, 150 ... Mask unit, 160 ... Conversion unit, 170 ... Image synthesis unit, 180 ... Output processing unit, 190 ... Storage unit

Claims (11)

1. An electric field estimation unit that converts an optical interference tom signal representing the state of a sample into a spatial frequency domain and generates electric field data indicating an electric field on the pupil surface corresponding to the reciprocal space of the sample.
   A mask unit that generates masked electric field data by acting on the electric field data with a mask showing the distribution of electric field passing characteristics on the pupil surface.
   An image processing apparatus including a conversion unit that converts the mask electric field data into a spatial region to generate a mask optical interference tom signal.
2. The image processing apparatus according to claim 1, wherein the passage characteristic distribution indicates a spatial frequency band of an azimuth angle passing through the electric field in the pupil surface.
3. The image processing apparatus according to claim 1 or 2, wherein the passing characteristic distribution indicates a spatial frequency band of a moving diameter passing through the electric field in the pupil surface.
4. The image processing apparatus according to claim 2 or 3, wherein the mask indicates the presence or absence of passage of the electric field for each sample point corresponding to the spatial frequency.
5. The image processing apparatus according to claim 4, wherein the mask has a mask value that changes monotonically between sample points within a predetermined range from the boundary between a passing region that passes through the electric field and a blocking region that does not pass through the electric field.
6. The image processing apparatus according to any one of claims 1 to 5, further comprising a synthesis unit that generates a mask image showing the structure of the sample based on the mask light interference tom signal.
7. A compositing unit is provided which synthesizes mask images based on each of M mask optical interference tomographic signals (M is an integer of 2 or more) in different display modes to generate a composite image.
   The mask unit obtains M mask electric field data by acting on the electric field data with M masks having different passage characteristic distributions.
   The image processing apparatus according to any one of claims 1 to 6, wherein the conversion unit converts each of the M mask electric field data into a spatial region to generate the M mask optical interference tomographic signals. ..
8. The image processing apparatus according to claim 7, wherein as the display mode, the hues of the colors representing the M mask images are different.
9. The image processing apparatus according to claim 8, wherein the color obtained by synthesizing the colors representing the M mask images as the display mode among the M mask images is an achromatic color.
10. It is a method in an image processing device.
    An electric field estimation process that converts an optical interference tom signal representing the state of a sample into a spatial frequency domain and generates electric field data indicating an electric field on the pupil surface corresponding to the reciprocal space of the sample.
    A mask process of generating masked electric field data by acting on the electric field data with a mask showing the distribution of electric field passing characteristics on the pupil surface.
    An image processing method comprising a conversion process of converting the masked electric field data into a spatial region to generate a masked optical interference tomographic signal.
11. To the computer of the image processing device
    An electric field estimation procedure that converts an optical interference tom signal representing the state of a sample into a spatial frequency domain and generates electric field data indicating an electric field on the pupil surface corresponding to the reciprocal space of the sample.
    A mask procedure for generating masked electric field data by acting on the electric field data with a mask showing the distribution of electric field passing characteristics on the pupil surface.
    A program for executing a conversion procedure for converting the masked electric field data into a spatial region to generate a masked optical interference tomographic signal.

Images