WO2022270291A1

WO2022270291A1 - Image processing device, and image processing method

Info

Publication number: WO2022270291A1
Application number: PCT/JP2022/022826
Authority: WO
Inventors: 巧実岡崎
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2021-06-21
Filing date: 2022-06-06
Publication date: 2022-12-29
Also published as: JP2023001783A

Abstract

An image processing device according to an embodiment of the present disclosure comprises a processor, and a storage device that stores data relating to an image. The processor: divides the image into a plurality of regions; subjects each of the plurality of regions to an orthogonal transformation; uses a plurality of orthogonal transformation coefficients calculated for each of the plurality of regions to calculate a representative value representing each region; and adjusts the sharpness of the image on the basis of a value calculated on the basis of at least one of the calculated representative values, and the plurality of orthogonal transformation coefficients.

Description

Image processing device and image processing method

The present disclosure relates to an image processing device and an image processing method.

A known technique is to sharpen an object in an image by means such as increasing the contrast. For example, Patent Literature 1 discloses a fundus image processing apparatus capable of automatically extracting a blood vessel region from a fundus image while maintaining the blood vessel fine shape such as the blood vessel diameter.

Japanese Unexamined Patent Application Publication No. 2018-23602

An object of the present disclosure is to provide an image processing device and an image processing method capable of emphasizing specific elements in image data compared to other elements.

One aspect of the present disclosure provides an image processing device that includes a processor and a storage device that stores image data. The processor divides the image into a plurality of regions, performs an orthogonal transform on each of the plurality of regions, and uses a plurality of orthogonal transform coefficients calculated for each of the plurality of regions to obtain a representative representative of each region. A value is calculated, and the sharpness of the image is adjusted based on the value calculated based on the calculated at least one or more representative values and the plurality of orthogonal transform coefficients.

Another aspect of the present disclosure provides an image processing method for performing image processing on image data stored in a storage device by a processor. This image processing method divides an image into a plurality of regions, performs an orthogonal transform on each of the plurality of regions, uses a plurality of orthogonal transform coefficients calculated for each of the plurality of regions, and converts each region into a The method includes calculating a representative value and adjusting the sharpness of the image based on a value calculated based on at least one or more calculated representative values and the plurality of orthogonal transform coefficients.

According to the image processing device and image processing method according to the present disclosure, it is possible to emphasize specific elements in image data compared to other elements.

1 is a block diagram showing a configuration example of an image processing device according to an embodiment of the present disclosure; FIG. 2 is a flow chart illustrating a procedure of image processing executed by a processor of the image processing apparatus of FIG. 1; Schematic diagram for explaining the DCT executed in step S7 of FIG. Flowchart illustrating the detailed procedure of the DCT coefficient enhancement process S8 in FIG. Schematic diagram for explaining the grouping process S81 in FIG. Schematic diagram for explaining normalization processing S82 in FIG. Graph illustrating magnification for first to fourth groups of DCT blocks

(Findings on which this disclosure is based)
In the conventional technology, when an image is generated by imaging an object covered by a shield using light having a specific wavelength, the object is not clear in the image due to attenuation and scattering of light by the shield. There is a problem to be Here, examples of the "object" are tissues such as blood vessels, bones and internal organs, and examples of the "shield" are tissues covering the object such as skin, fat and muscle. An example of "light having a specific wavelength" is light such as visible light, infrared light, ultraviolet light, and X-rays.

For example, when the human body is imaged using a camera, in the captured image, the blood vessels in the deep part are blurred compared to the blood vessels near the body surface, and the blood vessels may not be distinguishable.

In order to solve the above problems, the present inventors have developed an image processing apparatus and an image processing method that emphasize specific elements in image data compared to other elements so that the specific elements can be easily identified. Found it.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art. It is noted that Applicants provide the accompanying drawings and the following description for a full understanding of the present disclosure by those skilled in the art and are not intended to limit the claimed subject matter thereby. No.

(embodiment)
[1. composition]
FIG. 1 is a block diagram showing a configuration example of an image processing device 100 according to an embodiment of the present disclosure. The image processing apparatus 100 includes a processor 1 , a storage device 2 , an input interface (I/F) 3 and an output interface (I/F) 4 .

The processor 1 implements the functions of the image processing apparatus 100 by executing information processing. The processor 1 is composed of circuits such as a CPU, MPU, and FPGA. Such information processing is realized, for example, by the processor 1 executing a program 21 stored in the storage device 2 .

The storage device 2 is a recording medium for recording various information including the program 21 and data necessary for realizing the functions of the image processing device 100 . The storage device 2 is realized by, for example, a semiconductor storage device such as a flash memory, a solid state drive (SSD), a magnetic storage device such as a hard disk drive (HDD), or other recording media alone or in combination. The storage device 2 may include volatile memory such as SRAM and DRAM.

The input interface 3 is an interface circuit that connects the image processing device 100 and an external device in order to input information such as the image data 11 to the image processing device 100 . Such an external device is, for example, another information processing terminal (not shown) or a device such as a camera that acquires the image data 11 . The input interface 3 may be a communication circuit that performs data communication according to existing wired communication standards or wireless communication standards.

The output interface 4 is an interface circuit that connects the image processing device 100 and an external output device in order to output information from the image processing device 100 . Such an output device is for example the display 12 . The output interface 4 may be a communication circuit that is connected to the network 13 and performs data communication according to existing wired communication standards or wireless communication standards. The input interface 3 and output interface 4 may be realized by similar hardware.

[2. motion]
[2-1. Overall operation]
FIG. 2 is a flow chart illustrating the procedure of image processing executed by the processor 1 of the image processing apparatus 100 of FIG.

In step S1 of FIG. 2, the processor 1 acquires the image data 11. The image data 11 may be input to the processor 1 via the input interface 3 as shown in FIG. 1, or may be stored in the storage device 2 in advance.

In step S2 of FIG. 2, the processor 1 performs noise removal processing on the image represented by the image data 11. In this specification, the image represented by the image data 11 may also be expressed as "image 11" using the same reference numerals.

At step S3 of FIG. 2, the processor 1 performs a flattening process to increase the contrast of the image 11. FIG. For example, the processor 1 executes histogram equalization processing for widening the luminance value distribution for the luminance value of each pixel of the image 11 .
At step S4 in FIG. 2, the processor 1 executes gamma correction processing.
At step S5 in FIG. 2, the processor 1 performs other brightness correction processing. For example, the processor 1 determines each pixel value of the output image not only by using the corresponding input pixel value, but also by using the pixel values surrounding the corresponding input pixel. For example, the processor 1 identifies an object in the input image by comparing the luminance value of the input pixel with the average luminance value of pixels surrounding the input pixel, and makes the object clear in the output image. Correct the luminance value to

In step S6 of FIG. 2, the processor 1 performs resizing processing for adjusting at least one of the size of the image 11 and the size of the DCT block so that the width of the object such as the blood vessel is included in the DCT block, which will be described later. to run. Here, the “size” represents the dimension of the image represented by the image 11 , for example, the vertical and/or horizontal size of the image represented by the image 11 . For example, if the blood vessel is large compared to the DCT block and the width of the blood vessel in the image 11 is not contained within the DCT block, the processor 1 reduces the size of the image 11 so that the width of the blood vessel is contained within the DCT block. adjust. The ratio between the image size before resizing and the image size after resizing is determined by the processor 1 detecting the width of an object such as a blood vessel in the image 11, and based on the detection result, the size of the image 11, and the size of the DCT block. can be calculated by Alternatively, the ratio between the image size before resizing and the image size after resizing may be input to the processor 1 by the user via an input device such as a keyboard or touch panel and the input interface 3 .

2, the processor 1, as shown in FIG. 3, divides the image 11 processed in steps S2 to S6 of FIG. 2 by M pixels in the horizontal direction (x direction) and M pixels in the vertical direction (y direction). ) having a size of N pixels (herein referred to as “DCT blocks”) B _ij , and discrete cosine transform (DCT) is performed on each block. Here, M and N are integers of 2 or more, and M=N=8 in the example shown in FIG. DCT is an example of "orthogonal transform" in this disclosure. Multiple DCT blocks are an example of "multiple regions" in this disclosure. Also, i and j are integers equal to or greater than 0, and B _ij indicates the i-th DCT block in the horizontal direction (x direction) and the j-th DCT block in the vertical direction (y direction) in the image 11 .

By DCT, each block of image 11 is represented as a sum of base images with various frequency components. An example of 64 base images corresponding to each DCT coefficient F(u, v) is shown in the uv coordinate system indicated by the arrow of "DCT" in FIG. For example, in step S7, the processor 1 transforms the image data f(x, y) of each DCT block into image data having frequency components, that is, DCT coefficients F(u, v) by the following equation (1). do. The transformation basis is cos θ. A case where the DCT coefficient F(0,0) when u=v=0 is called a "DC component" and the DCT coefficient F(u,v) when u≠0 and v≠0 is called an "AC component" There is Note that Cu and Cv are constant terms in the formula (1). Formula (1) is merely an example, and the contents of constant terms and other specific parameter values can be arbitrarily determined according to the purpose of processing.

At step S8 in FIG. 2, the processor 1 executes DCT coefficient enhancement processing. For example, processor 1 multiplies frequency components F(u, v) of DCT coefficients in each DCT block that satisfy a specific condition by more than one. A predetermined value may be added instead of multiplying the magnification. A detailed procedure of the DCT coefficient enhancement processing S8 will be described later.

At step S9 in FIG. 2, the processor 1 performs an inverse discrete cosine transform (IDCT) on the DCT coefficients F(u, v) enhanced by the DCT coefficient enhancement processing S8.

At step S10 in FIG. 2, the processor 1 executes a reverse resizing process corresponding to the resizing process S6. Specifically, the image 11 after the IDCT processing S9 has the size resized in the resizing processing S6. size.

[2-2. DCT coefficient enhancement processing]
FIG. 4 is a flowchart illustrating a detailed procedure of the DCT coefficient enhancement processing S8 of FIG.

At step S81 in FIG. 4, the processor 1 groups the DCT coefficients in order of absolute value in each DCT block. FIG. 5 is a schematic diagram for explaining the grouping process S81 of FIG. The right diagram of FIG. ₅ is a schematic diagram showing the relationship between the magnitudes of the DCT coefficients of the DCT block B05 of the image 11 shown in the left diagram of FIG. The numbers in the right diagram of FIG. 5 indicate the order of magnitude of the absolute values of the DCT coefficients. For example, the absolute value of the DCT coefficient F ₀₅ (1, ₀ ) shown in the first row and second column of the right diagram of FIG. 9th largest in

For example, in the grouping process S81, the processor 1 classifies the DCT coefficients having the second to fourth magnitudes into the first group, and classifies the DCT coefficients having the fifth to eighth magnitudes into the second group. Then, the DCT coefficients having magnitudes of the 9th to 16th ranks are classified into the third group, and the DCT coefficients having magnitudes of the 17th rank and below are classified into the fourth group. In FIG. 5, the first group is indicated by horizontal line hatching, the second group is indicated by dot hatching, and the third group is indicated by diagonal line hatching. The DC component and the fourth group are not hatched. Also, description of the order of the smaller DCT coefficients, which do not belong to the fourth group, has been omitted. By the grouping process S81, each DCT block is grouped into a first group, a second group, . . .

At step S82 of FIG. 4, processor 1 determines the group with the largest absolute value of the DCT coefficients among all groups of all DCT blocks. Specifically, the processor 1 first obtains the sum r of the absolute values of the DCT coefficients belonging to the first group for each of all DCT blocks. The calculated sum r is an example of a "representative value" representing each DCT block. Processor 1 then determines the largest maximum representative value _{r_max} among the representative values r determined for all DCT blocks.

FIG. 6 is a schematic diagram for explaining step S82 in FIG. The upper diagram of FIG. 6 shows the representative value r in the first group of each DCT block of the image 11 . In the example shown in the upper diagram of FIG. 6, the maximum representative value r _max =224.

At step S82 of FIG. 4, processor 1 normalizes the DCT coefficients in each DCT block based on the maximum representative value _{r_max} . As an example of normalization, processor 1 divides the DCT coefficients in each DCT block by the _maximum representative value rmax. The lower diagram of FIG. 6 shows the normalized value _rn of the sum of the absolute values of the DCT coefficients in the first group of each DCT block.

At step S83 of FIG. 4, the processor 1, in each group of each DCT block,
A magnification _m is determined according to the normalization value rn calculated in the normalization processing S82. For example, the processor 1 sets the magnification _m ₁ for the first group, the magnification m 2 for the second group, the magnification m ₂ for the third group, and Determine the magnification m ₃ and the magnification m ₄ for the fourth group.

FIG. 7 is a graph illustrating such scaling factors m ₁ -m ₄ as a function of normalization value r _n . In FIG. 7, the solid line indicates _m1 , the dashed line indicates _m2 _, the dashed line indicates m3, and the dotted line indicates _m4 . The relationship between the normalized value r _n and the magnifications m ₁ to m ₄ as shown in the graph of FIG. 7 may be stored in advance in the storage device 2 as a table. In this case, the processor 1 refers to the relationship between the normalization value r _n and the magnifications m ₁ to m ₄ stored in the storage device 2, and calculates the magnifications m ₁ to m according to the magnitude of the normalization value r _n . ₄ is determined.

At step S84 in FIG. 4, the processor 1 multiplies the corresponding DCT coefficients by the scaling factors determined at step S83.

In the example shown in FIG. 7, the magnification m1 for the first group and the magnification _m2 for the second group are set to ₁ or more, and the magnification m3 for the _third group and the magnification m4 for the _fourth group are set to 1 or less. is set to For example, m ₁ >m ₂ and m ₃ >m ₄ . Thus, the image processing apparatus ₁₀₀ multiplies the DCT coefficients of the first group, which contribute greatly to the construction of the image 11, by the maximum magnification m1, and contributes to the construction of the image 11 to a lesser extent than the first group. Multiply the DCT coefficients of the second group by the maximum scale factor _m2 . On the other hand, the image processing apparatus 100 can regard the DCT coefficients of the third group and the fourth group, which contribute relatively little to the configuration of the image 11, as noise that does not contribute to the configuration of the image 11. attenuate by multiplying by a factor of . Further, the magnification m1 for the _first group and the magnification _m2 for the second group take maximum values in the range where the normalized value r _n is greater than 0 and less than 1. In this way, blocks with relatively few frequency components are maximized. Therefore, objects existing at different depths can be corrected to have similar appearances.

In this way, the image processing apparatus 100 can sharpen the object in the output image data by changing the magnification according to the degree of contribution to the configuration of the original image data. For example, in the original image data obtained by imaging a living body, even if deep blood vessels are unclear compared to blood vessels near the body surface, the image processing apparatus 100 can output , deep vessels or both.

[3. effects, etc.]
As described above, in the image processing apparatus 100, the processor 1 divides the image 11 into a plurality of DCT blocks and performs DCT on each of the plurality of DCT blocks. The processor 1 calculates a representative value representing each DCT block using a plurality of DCT coefficients calculated for each of the plurality of DCT blocks, and calculates a value calculated based on at least one or more of the calculated representative values. , and a plurality of DCT coefficients.

With this configuration, the image processing apparatus 100 can emphasize specific elements in the image 11 compared to other elements.

A processor 1 classifies a plurality of DCT coefficients into a plurality of groups based on absolute values, and for each classified group, a value calculated based on at least one or more calculated representative values, a plurality of DCT coefficients, and The sharpness of image 11 may be adjusted based on . For example, processor 1 may multiply each absolute value of a plurality of DCT coefficients in each of the sorted groups by different scale factors.

With this configuration, it is possible to determine the magnification for each group and emphasize each DCT coefficient of the DCT block according to the extent to which each group contributes to the configuration of the image 11 .

The processor 1 may determine the scaling factor in each of the multiple groups based on the absolute values of the DCT coefficients in each of the multiple groups.

With this configuration, each DCT coefficient of the DCT block can be emphasized according to the extent to which each group contributes to the configuration of the image 11.

The processor 1 selects a predetermined representative value from the representative values, normalizes the representative value representing each region based on the predetermined representative value, and converts the DCT block represented by the normalized representative value into a DCT block. The sharpness of the image may be adjusted based on the included DCT and the value according to the normalized representative value.

With this configuration, it is possible to set magnifications that can correspond to various images 11 .

Specifically, processor 1 may classify a plurality of DCT coefficients into a plurality of groups in descending order of absolute value (step S81). In this case, the processor 1 calculates the sum of the absolute values of the DCT coefficients in the group having the largest absolute value of the plurality of DCT coefficients among the plurality of groups as the maximum representative value r _max , and calculates the calculated maximum representative value r Based on _max , the DCT coefficients in each of the plurality of DCT blocks are normalized (step S82). Processor 1 multiplies the normalized DCT coefficients by a scale factor corresponding to their absolute values (steps S83 and S84).

The image processing device 100 may execute image sharpening processing for sharpening the object appearing in the image 11 . In such an image processing apparatus 100, the processor 1, before dividing the image 11 into a plurality of DCT blocks, processes the size and size of the image 11 so that the width of the object is the width contained within each DCT block. Adjust at least one of the sizes of the plurality of DCT blocks.

With this configuration, the width of the object fits within the DCT block, so the object can be sharpened within the DCT block.

(Other embodiments)
As described above, the above embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments in which modifications, replacements, additions, omissions, etc. are made as appropriate. Therefore, other embodiments will be exemplified below.

In the above embodiment, in the DCT coefficient enhancement process S8, the processor 1 executes the grouping process S81, and then performs the normalization process S82 to determine the magnification m (step S83). However, the present disclosure is not limited to this, and after executing the grouping process S81, if the representative value r of the DCT block is larger than a predetermined threshold, the processor 1 multiplies the relevant value by a factor larger than 1. The absolute value of the DCT coefficients may be increased. According to this configuration, the image processing apparatus 100 can emphasize only the DCT coefficients with relatively large absolute values among the plurality of DCT coefficients of the DCT block in the image 11 .

In the above embodiment, an example was explained in which the processor 1 determines the magnification m (step S83) in the DCT coefficient enhancement processing S8. However, the present disclosure is not limited to this, and after the processor 1 executes the grouping process S81, instead of steps S82 and S83 in FIG. 4, the user may manually determine the magnification m. At step S84, processor 1 multiplies the corresponding DCT coefficients by the user-determined scaling factor m. For example, a user determines the scale factor m using a graphical user interface (GUI). The user may adjust the magnification m using the GUI while checking the image 11 that has undergone the multiplication processing S84 of the magnification m and the IDCT processing S9 using the display 12 or the like.

In the above embodiment, image data 11 generated by imaging an object covered with a shield using light having a specific wavelength has been described. However, the image data 11 that can be processed by the image processing apparatus 100 is not limited to optically captured data, and may be ultrasound images, magnetic resonance imaging (MRI), or the like.

In the above embodiment, DCT was described as an example of orthogonal transform, but orthogonal transform is not limited to this, and may be discrete Fourier transform (DFT), for example. DFT includes Fast Fourier Transform (FFT).

The present disclosure is applicable to image processing technology.

1 Processor 2 Storage Device 3 Input Interface 4 Output Interface 11 Image Data 12 Display 13 Network 21 Program 100 Image Processing Apparatus

Claims

An image processing device comprising a processor and a storage device storing image data,
The processor
dividing the image into a plurality of regions;
performing an orthogonal transform on each of the plurality of regions;
calculating a representative value representing each region using a plurality of orthogonal transform coefficients calculated for each of the plurality of regions;
adjusting the sharpness of the image based on the values calculated based on the calculated at least one or more representative values and the plurality of orthogonal transform coefficients;
Image processing device.
The image processing device according to claim 1, wherein said processor increases absolute values of said plurality of orthogonal transform coefficients when said representative value in each of said plurality of regions is larger than a predetermined threshold.
The processor
classifying the plurality of orthogonal transform coefficients into a plurality of groups based on absolute values;
adjusting the sharpness of the image based on the values calculated based on the calculated at least one or more representative values and the plurality of orthogonal transform coefficients for each of the classified groups;
The image processing apparatus according to claim 1.
The processor
selecting a predetermined representative value from the representative values;
normalizing the representative value representing each region based on the predetermined representative value;
Adjusting the sharpness of the image based on the orthogonal transform coefficients included in the region represented by the normalized representative value and a value corresponding to the normalized representative value;
The image processing apparatus according to claim 1.
The processor
classifying the plurality of orthogonal transform coefficients into a plurality of groups in order according to the magnitude of the absolute value;
calculating, as the representative value, the sum of the absolute values of the plurality of orthogonal transform coefficients in the group having the largest absolute value of the plurality of orthogonal transform coefficients among the plurality of groups;
normalizing the orthogonal transform coefficients in each of the plurality of regions based on the calculated representative value;
multiplying the plurality of normalized orthogonal transform coefficients by a scale factor according to their absolute values;
The image processing apparatus according to claim 1.
The image processing device executes an image sharpening process for sharpening an object appearing in the image,
The processor adjusts at least one of the size of the image and the sizes of the plurality of regions such that the width of the object is contained within each region before dividing the image into regions. do,
The image processing apparatus according to claim 1.
An image processing method for performing image processing on image data stored in a storage device by a processor,
dividing the image into a plurality of regions;
performing an orthogonal transform on each of the plurality of regions;
calculating a representative value representing each region using a plurality of orthogonal transform coefficients calculated for each of the plurality of regions;
A value calculated based on the calculated at least one or more representative values, and adjusting the sharpness of the image based on the plurality of orthogonal transform coefficients.
Image processing method.
The image processing method according to claim 7, comprising increasing absolute values of the plurality of orthogonal transform coefficients when the representative value in each of the plurality of regions is greater than a predetermined threshold.
classifying the plurality of orthogonal transform coefficients into a plurality of groups based on absolute values;
Adjusting the sharpness of the image based on the values calculated based on the calculated at least one or more representative values and the plurality of orthogonal transform coefficients for each of the classified groups,
The image processing method according to claim 7.
selecting a predetermined representative value from the representative values;
normalizing the representative value representing each region based on the predetermined representative value;
Adjusting the sharpness of the image based on the orthogonal transform coefficients included in the region represented by the normalized representative value and the value corresponding to the normalized representative value,
The image processing method according to claim 7.
classifying the plurality of orthogonal transform coefficients into a plurality of groups in order according to the magnitude of the absolute value;
calculating, as the representative value, the sum of the absolute values of the plurality of orthogonal transform coefficients in the group having the largest absolute value of the plurality of orthogonal transform coefficients among the plurality of groups;
normalizing the orthogonal transform coefficients in each of the plurality of regions based on the calculated representative value;
Multiplying the plurality of normalized orthogonal transform coefficients by a scale factor according to their absolute values,
The image processing method according to claim 7.
The image processing method is an image sharpening processing method for sharpening an object appearing in the image,
adjusting at least one of the size of the image and the size of the plurality of regions so that the width of the object is contained within each region before dividing the image into regions. ,
The image processing method according to claim 7.