JP2019087126A - Image processing device, image processing method and image processing program - Google Patents

Abstract

To provide an image processing device capable of generating images in which minor changes are emphasized by reducing blur noises, and an image processing method and an image processing program.SOLUTION: The image processing device comprises: a generation unit that generates images of various resolution from original images; a detection unit that detects minute changes in the images of various resolution; a reliance unit that acquires the reliance level of the minute changes; an emphasis unit that performing emphasis processing on a multiplication result of a value representing the minute change and the reliance level; and a reconstitution unit that generates images the original images which have been subjected to the emphasis processing on the minute changes on the basis of multiplication result which has been subjected to the emphasis processing and the original images.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  The minute change in the image in which the subject in the real world is photographed may include an important message. For example, the difference in usage of muscles and joints of the athlete who is the subject is a minute change in the image in which the subject is photographed, and is one factor that determines the exercise performance of the athlete. In addition, the heartbeat, movement of the chest due to breathing, changes in the brightness of the body surface color, and minute changes in the vibration of the crane or engine may serve as judgment materials for detecting an abnormal state based on an image. . However, it is difficult for human vision to capture minute changes in the image.

  Video Magnification is a technique for visualizing minute changes in an image by detecting and enhancing the minute changes in the image. The image processing apparatus detects a change in the color or motion of the subject in the image using optical flow, Euler method or phase change, and detects a minute change in the subject in the image by applying a time series filter ( Non-patent documents 1, 2 and 3).

  However, it is difficult for the image processing apparatus to detect only a minute change of the subject when the subject moves large or quickly in the image. When a change other than a minute change of the subject is detected and emphasized, blur noise may occur in the image.

  In recent years, two methods have been proposed that can detect only minute changes in an object even when the object moves significantly in an image. The first method is a method of specifying a subject in an image, stabilizing a large movement, and detecting a minute change of the subject (see Non-Patent Document 4). The second method is a method of designing a time-series filter capable of detecting only a minute change by modeling a large motion in an image (see Non-Patent Document 5).

Ce Liu, Antonio Torralba, William T. Freeman, Fredo Durand, Edward H. Adelson. “Motion Magnification”. ACM Transactions on Graphics. Vol. 24. (2005) Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Fredo Durand, William Freeman. “Eulerian Video Magnification for Revealing Subtle Changes in the World”. SIGGRAPH. (2012) Neal Wadhwa, Michael Rubinstein, Fredo Durand, William T. Freeman. “Phase-based video motion processing”. ACM Transactions on Graphics. Vol. 32. (2013) Mohamed A. Elgharib, Mohamed Hefeeda, Fredo Durand, and William T. Freeman. “Video Magnification in Presence of Large Motions”. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 7. pp. 4119-4127 . (2015) Yichao Zhang, Silvia L. Pintea, and Jan C. van Gemert. “Video Acceleration Magnification”. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). (2017)

  In order to be able to easily apply video magnification to various images, it is necessary to detect only minute changes in an object in an image without using manual operation as in Non-Patent Document 3 . For that purpose, it is useful to design a time-series filter based on motion modeling as in Non-patent Document 4.

  However, in Non-Patent Document 3, a time-series filter is designed in which only slow and largely changing motion causing blur noise is modeled. Such a time-series filter generates a large blur noise in the image when there is a non-linear motion that rapidly and largely changes in a short time in the image. As described above, the conventional image processing apparatus may not be able to generate an image in which a minute change is emphasized by reducing blur noise.

  In view of the above circumstances, the present invention has an object of providing an image processing apparatus, an image processing method, and an image processing program capable of generating an image in which a minute change is emphasized by reducing blur noise. .

  One aspect of the present invention is a generation unit that generates an image of each resolution from an original image, a detection unit that detects a minute change in the image of each resolution, a reliability unit that acquires the reliability of the minute change, and the minute The minute change in the original image is emphasized based on the emphasizing unit which performs the emphasizing process on the multiplication result of the value representing the change and the reliability, and the multiplication result on which the emphasis process is performed and the original image. It is an image processing apparatus provided with the reconstruction part which produces | generates an image.

  One embodiment of the present invention is the image processing device described above, wherein the generation unit generates a luminance image, which is an image representing luminance, from the luminance image of the original image for each resolution, and the detection unit The minute change of the brightness in the brightness image is detected, the reliance unit acquires the reliability of the minute change of the brightness, and the emphasizing unit multiplies the reliability result by the value representing the minute change of the brightness. An emphasizing process is performed, and the reconstruction unit generates an image in which a slight change in the luminance in the original image is subjected to the emphasizing process based on the multiplication result subjected to the emphasizing process and the color image of the original image.

  One embodiment of the present invention is the image processing device described above, wherein the generation unit generates a phase image which is an image representing a phase from the phase image of the original image for each resolution and direction, and the detection unit The minute change of the phase in the phase image of the resolution is detected, the reliability unit obtains the reliability of the minute change of the phase, and the emphasis unit multiplies the reliability by a value representing the minute change of the phase. An emphasizing process is performed on the result, and the reconstruction unit generates an image in which an emphasizing process is performed on the minute change in the original image based on the multiplication result on which the emphasizing process is performed and the color image of the original image. Do.

  One embodiment of the present invention is the image processing apparatus described above, further comprising: an integration unit that integrates the reliability in a phase image of a plurality of resolutions for each direction, and the emphasis unit represents a minute change in the phase. Emphasis processing is performed on the multiplication result of the reliability integrated with the value.

  One aspect of the present invention is an image processing method executed by an image processing apparatus, which comprises: generating an image of each resolution from an original image; detecting a minute change in the image of each resolution; The step of acquiring the reliability, the step of performing enhancement processing on the multiplication result of the value representing the minute change and the reliability, the above-mentioned in the original image based on the multiplication result subjected to the enhancement processing and the original image And d) generating an image in which a slight change is enhanced.

  One aspect of the present invention is a computer that generates an image of each resolution from an original image, detects a minute change in the image of each resolution, obtains a reliability of the minute change, and the minute An image in which the minute change in the original image is subjected to the enhancement process based on the procedure of performing the enhancement process on the multiplication result of the value representing the change and the reliability, and the multiplication result subjected to the enhancement process and the original image And an image processing program for executing the steps of generating

  According to the present invention, blur noise can be reduced to generate an image in which a minute change is emphasized.

FIG. 1 is a diagram showing an example of the configuration of an image processing apparatus according to a first embodiment. 5 is a flowchart showing an example of the operation of the image processing apparatus in the first embodiment. It is a figure which shows the example of a structure of the image processing apparatus in 2nd Embodiment. It is a flow chart which shows an example of operation of an image processing device in a 2nd embodiment. It is a figure which shows the example of a structure of the image processing apparatus in 3rd Embodiment. It is a flowchart which shows the example of operation | movement of the image processing apparatus in 3rd Embodiment.

Embodiments of the present invention will be described in detail with reference to the drawings.
First Embodiment
FIG. 1 is a diagram showing an example of the configuration of the image processing apparatus 1a. The image processing apparatus 1a is an apparatus that executes image processing. The image processing apparatus 1a detects and emphasizes a minute change in the brightness of a subject in an image. The image processing apparatus 1a includes a generation unit 10a, a detection unit 11a, a reliance unit 12a, a multiplication unit 13a, an emphasis unit 14a, and a reconstruction unit 15a. The image processing apparatus 1a may further include a storage unit. The storage unit may be included in each functional unit.

  For example, a processor such as a CPU (Central Processing Unit) executes a program stored in the storage unit to execute a part or all of the functional units. Some or all of the functional units may be realized using hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit).

  The storage unit is, for example, a non-volatile recording medium (non-temporary recording medium) such as a flash memory or a hard disk drive (HDD). The storage unit may have, for example, a volatile recording medium such as a random access memory (RAM) or a register. The storage unit stores, for example, various data such as image data and mathematical expressions.

  The original image is a moving image composed of a plurality of frames. The original image includes luminance information and color information for each pixel. The generation unit 10a acquires a luminance image which is an image representing luminance information of the original image. The generation unit 10a generates a luminance image of each resolution from the luminance image of the original image. For example, the generation unit 10a generates a luminance image of each resolution for each frame by convoluting a Gaussian filter or Laplacian filter and then performing downsampling processing on each frame of the luminance image multiple times. The downsampling process is an image process for reducing the resolution based on the amount of downsample such as half. The change in luminance of the image at time t is expressed as Expression (1).

  Here, l represents the resolution. x represents the x coordinate in the frame of the luminance image. y represents the y-coordinate in the frame of the luminance image.

The detection unit 11a detects a minute change in luminance in the luminance image of each resolution generated by the generation unit 10a. The detection unit 11a detects a minute change in luminance in the luminance image of each resolution by convolving the time series filter H (t) with the value I ^l (x, y, t) representing the luminance change in the luminance image of each resolution. Do. The time-series filter H (t) is a time-series filter having a frequency response of minute changes that are emphasized. The value B ^l (x, y, t) representing a minute change in luminance is expressed as equation (2).

  The detection unit 11 a may detect a minute change in luminance by a method other than the method using the time series filter H (t).

The trust unit 12a outputs the reliability of the minute change of the luminance in the luminance image of each resolution generated by the generation unit 10a to the multiplication unit 13a. The reliability of the minute change in luminance is the degree of certainty that only a minute change in luminance occurs in a predetermined spatial region and time region in the image. The reliance unit 12a calculates the reliability of the minute change of the brightness for the value I ^l (x, y, t) representing the change of the brightness in the brightness image of each resolution.

The trust unit 12a calculates the reliability of the minute change in luminance using the jerk. The jerk is a differential value of acceleration and is a value corresponding to the third derivative of displacement. The confidence unit 12a applies the time series filter G (t) and the third derivative to the frequency band of the minute change emphasized among the values I ^l (x, y, t) representing the luminance change in the luminance image of each resolution. Use to calculate the jerk. The trust unit 12a calculates the jerk Jerk ^l (x, y, t) as shown in equation (3).

  The reliance unit 12a may calculate the jerk by a method other than the method using the time series filter G (t).

  The trust unit 12a calculates the absolute value of the jerk as expressed by equation (4).

  The trust unit 12a normalizes the jerk in the space and time directions, and inverts the value of the jerk as expressed by equation (5).

The trust unit 12a calculates the reliability C ^l (x, y, t) of the minute change as shown in Expression (6).

The trust unit 12a designs a time series filter for detecting only a minute change with high accuracy, based on the calculated reliability C ^l (x, y, t). The trust unit 12a corrects the reliability of the minute change of the luminance in the luminance image of each resolution. The reliance unit 12a corrects the reliability of the minute change in luminance as in Expression (7) based on the down sample amount λ and the hyper parameter β in the down sampling process when the luminance image of each resolution is generated. .

  In the luminance image, the degree of reliability of the space area and the time area with a high degree of jerk is reduced by assigning a value close to 0 by correction as shown in equation (7). The degree of reliability of the space area and the time area with small jerk in the luminance image is increased by assigning a value close to 1 by correction as shown in equation (7). That is, the reliability of the minute change in luminance is increased by being assigned a value close to 1 by the correction.

The multiplication unit 13a multiplies the output of the detection unit 11a by the output of the reliance unit 12a. That is, the multiplying unit 13a multiplies the reliability as in the equation (8) by the value B ^l (x, y, t) representing a minute change in luminance as in the equation (2). Thus, the multiplying unit 13a can detect only a minute change in luminance with high accuracy.

The emphasizing unit 14a obtains the multiplication result of the output of the detecting unit 11a and the output of the relieving unit 12a from the multiplying unit 13a. The emphasizing unit 14a performs an emphasizing process on the minute change of the luminance in the luminance image of each resolution which is the multiplication result. The emphasizing unit 14a multiplies an arbitrary emphasis rate α by a value representing a minute change detected with high accuracy as expressed by equation (8). The enhancement factor α may be common or different in the luminance image of each resolution. The emphasizing unit 14a emphasizes only the minute change by adding the multiplication result of the emphasis ratio α and the value I ^l (x, y, t) representing the change in luminance of the original image as in equation (9). Generate a value lI ^l (x, y, t) representing the change in luminance.

The reconstructing unit 15a generates an image in which the minute change in the luminance in the original image is subjected to the emphasizing process based on the minute change in the luminance in which the emphasizing process is performed on the luminance image of each resolution by the emphasizing unit 14a. That is, the reconstruction unit 15a generates the value ^ I ^l (x, y, t) representing the change in luminance shown in the equation (9) from 1 to L (L, as in the equation (10) By adding up to the number of resolution types of the luminance image) in the resolution direction, a luminance image in which only a minute change in luminance is emphasized in the luminance image of the original image is generated.

  The reconstruction unit 15a acquires a color image which is an image representing color information of the original image. The reconstruction unit 15a generates an image in which only the minute change in luminance is emphasized in the original image, by combining the luminance image in which only the minute change in luminance is emphasized and the color image of the original image.

Next, an example of the operation of the image processing apparatus 1a will be described.
FIG. 2 is a flowchart showing an example of the operation of the image processing apparatus 1a. The generation unit 10a generates a luminance image of each resolution from the luminance image of the original image (step S101). The detection unit 11a detects a minute change in luminance in the generated luminance image of each resolution (step S102). The trust unit 12a calculates the reliability of the minute change in luminance in the luminance image of each resolution generated by the generation unit 10a (step S103). The multiplication unit 13a multiplies the output of the detection unit 11a by the output of the reliance unit 12a (step S104).

  The emphasizing unit 14a multiplies the emphasizing rate α by the multiplication result of the output of the detecting unit 11a and the output of the relieving unit 12a. That is, the emphasizing unit 14a multiplies the value representing a minute change of the luminance in the luminance image of each resolution by the emphasis ratio α. The emphasizing unit 14a adds the multiplication result of the emphasis ratio to a value representing a change in luminance in the luminance image of the original image (step S105). The reconstructing unit 15a generates an image in which the minute change is emphasized in the original image by combining the addition result of the values indicating the luminance change and the color image of the original image (step S106).

  As described above, the image processing apparatus 1a according to the first embodiment generates the image of each resolution from the original image, the detection unit 11a that detects the minute change in the image of each resolution, and the reliability of the minute change In the original image, based on the reliance unit 12a for acquiring the degree, the emphasizing unit 14a for emphasizing processing on the multiplication result of the value representing the minute change and the reliability, the multiplication result on which the emphasizing processing is performed and the original image And a reconstruction unit 15a that generates an image in which a minute change is emphasized.

  The generation unit 10a generates a luminance image, which is an image representing luminance, for each resolution from the luminance image of the original image. The detection unit 11a detects a minute change in luminance in the luminance image of each resolution. The truster 12a acquires the reliability of the minute change in luminance. The emphasizing unit 14 a performs emphasizing processing on the multiplication result of the value representing the minute change of the luminance and the reliability. The reconstruction unit 15a generates an image in which the minute change in the luminance in the original image is subjected to the emphasizing process based on the multiplication result subjected to the emphasizing process and the color image of the original image. Thus, the image processing apparatus 1a according to the first embodiment can reduce blur noise and generate an image in which a minute change is emphasized.

Second Embodiment
The second embodiment is different from the first embodiment in that a change in luminance in a luminance image of each resolution is converted into motion information (amplitude information and phase information). In the second embodiment, only differences from the first embodiment will be described.

  FIG. 3 is a diagram showing an example of the configuration of the image processing apparatus 1b. The image processing device 1 b is a device that executes image processing. The image processing device 1b detects and emphasizes minute changes in the subject's motion in the image. The image processing apparatus 1b includes a generation unit 10b, a detection unit 11b, a reliance unit 12b, a multiplication unit 13b, an emphasis unit 14b, and a reconstruction unit 15b. The image processing apparatus 1b may further include a storage unit. The storage unit may be included in each functional unit.

  The generation unit 10 b acquires a luminance image of the original image. The generation unit 10 b converts the luminance change in the luminance image of the original image into amplitude information and phase information in a plurality of directions in the luminance image. Hereinafter, an image including phase information is referred to as a “phase image”. The generation unit 10 b generates a phase image of each resolution from the luminance image including the amplitude information and the phase information of the original image.

  The method by which the generation unit 10 b converts a change in luminance into amplitude information and phase information to generate a phase image of each resolution is not limited to a specific method. The generation unit 10 b converts a luminance change into amplitude information and phase information using a filter such as a CSF (Complex Steerable Filter), for example.

The generation unit 10 b applies CSF to the value I (x, y, t) representing the luminance change of the luminance image at the coordinates (x, y) of the frame of the phase image and the time t as shown in Expression (11). The generation unit 10b applies the CSF to the value I (x, y, t) representing the change in luminance of the luminance image to generate the value I (x, y, t) representing the change in luminance with the directivity θ at the resolution l. It is converted into a value A _θ ^l (x, y, t) representing an amplitude change and a value φ _θ ^l (x, y, t) representing a phase change. Here, _{θ θ} ^l represents CSF at resolution l and direction θ.

The detection unit 11 b detects a minute change in phase in the phase image of each resolution generated by the generation unit 10 b for each direction in the phase image. The detection unit 11 b convolutes the time series filter H (t) into the value φ _θ ^l (x, y, t) representing the phase change in each direction in the phase image of each resolution as in Expression (12), The minute change of the phase in the phase image of each resolution is detected. The value B ^l (x, y, t) representing a minute change in phase is expressed as in equation (12).

  Note that the detection unit 11b may detect a minute change in phase by a method other than the method using the time series filter H (t).

  The trust unit 12b outputs the reliability of the minute change of the phase in the phase image of each resolution generated by the generation unit 10b to the multiplication unit 13b for each direction in the phase image. Compared with Formula (4) to Formula (7), parameter (theta) showing a direction is added in Formula (11) and Formula (12). The reliability of the phase change for each direction in the phase image of each resolution is expressed as Expression (13).

  The confidence unit 12b corrects the reliability of the minute change of the phase as in Expression (13) using the down sample amount λ and the hyper parameter β in the down sampling process when the phase image of each resolution is generated. .

The multiplication unit 13 b multiplies the output of the detection unit 11 b by the output of the reliance unit 12 b. That is, the multiplication unit 13 b multiplies the reliability as in Expression (14) by the value B ^l (x, y, t) representing a minute change in phase. Thus, the multiplying unit 13b can detect only a minute change in the phase with high accuracy.

The emphasizing unit 14 b acquires the multiplication result of the output of the detecting unit 11 b and the output of the relieving unit 12 b. The emphasizing unit 14 b performs emphasizing processing on the minute change of the phase in each direction in the phase image of each resolution which is the multiplication result. The emphasizing unit 14 b multiplies a value representing a minute change detected with high accuracy as in Expression (14) by an arbitrary emphasis rate α. The emphasizing unit 14 b emphasizes only the minute change by adding the multiplication result of the emphasis ratio α and the value φ _θ ^l (x, y, t) representing the phase change of the original image as in equation (15). Generate values ^ φ _θ ^l (x, y, t) representing the phase change.

The reconstructing unit 15b generates an image in which the minute change in the phase in the original image is subjected to the enhancement process based on the minute change in the phase in which the enhancement process for each direction in the phase image of each resolution is performed by the emphasis unit 14b. Do. That is, the reconstruction unit 15a adds the values ^ φ _θ ^l (x, y, t) representing the phase change shown in equation (15) from 1 to L in the resolution direction for each direction in the phase image. By combining, a phase image in which only a small change in phase is emphasized in the phase image of the original image is generated.

  The reconstruction unit 15b applies the inverse filter of CSF to the phase image in which the minute change in phase is emphasized to obtain the phase image in which the minute change in phase is emphasized as the luminance of the original image in which the minute change in luminance is emphasized. Convert to an image.

  The reconstruction unit 15b acquires a color image of the original image. The reconstruction unit 15b generates an image in which only the minute change in motion is emphasized in the original image by combining the luminance image in which only the minute change in luminance is emphasized with the color image of the original image.

Next, an example of the operation of the image processing apparatus 1b will be described.
FIG. 4 is a flowchart showing an example of the operation of the image processing apparatus 1b. The generation unit 10 b converts the luminance change in the luminance image of the original image into amplitude information and phase information in a plurality of directions in the luminance image. The generation unit 10 b generates a phase image of each resolution from the luminance image of the original image (step S 201). The detection unit 11 b detects a slight change in phase in the generated phase image of each resolution for each direction in the phase image (step S 202). The reliance unit 12 b calculates the reliability of the minute change of the phase in the phase image of each resolution generated by the generation unit 10 b (step S 203). The multiplication unit 13b multiplies the output of the detection unit 11b by the output of the reliance unit 12b (step S204).

  The emphasizing unit 14 b multiplies the emphasizing rate α by the multiplication result of the output of the detecting unit 11 b and the output of the relieving unit 12 b. That is, the emphasizing unit 14b multiplies the value representing the minute change of the phase in the phase image of each resolution by the emphasis ratio α. The emphasizing unit 14b adds the multiplication result of the emphasis ratio to a value representing phase change in the phase image of the original image (step S205). The reconstruction unit 15b generates an image in which a minute change in the original image is emphasized by combining the addition result of the phase change information and the color image of the original image (step S206).

  As described above, the image processing apparatus 1b according to the second embodiment generates the image of each resolution from the original image, the detection unit 11b that detects the minute change in the image of each resolution, and the reliability of the minute change. In the original image, based on the reliance unit 12b for acquiring the degree, the emphasizing unit 14b for emphasizing processing on the multiplication result of the value representing a minute change and the reliability, and the multiplication result on which the emphasizing processing is performed and the original image And a reconstruction unit 15 b that generates an image in which a minute change is emphasized.

  The generation unit 10 b generates a phase image, which is an image representing a phase, from the phase image of the original image for each resolution and direction. The detection unit 11 b detects a minute change of the phase in the phase image of each resolution. The truster 12b acquires the reliability of the minute change of the phase. The emphasizing unit 14 b performs emphasizing processing on the multiplication result of the value representing the minute change in phase and the reliability. The reconstructing unit 15 b generates an image in which a slight change in the phase of the original image is subjected to the emphasis processing, based on the multiplication result subjected to the emphasis processing and the color image of the original image. As a result, the image processing apparatus 1b according to the second embodiment can reduce blur noise and generate an image in which a minute change is emphasized.

Third Embodiment
The third embodiment is different from the second embodiment in that the reliability of the minute change of the phase in the phase image of each resolution is integrated. In the third embodiment, only differences from the second embodiment will be described.

  FIG. 5 is a diagram showing an example of the configuration of the image processing apparatus 1c. The image processing device 1 c is a device that executes image processing. The image processing device 1c detects and emphasizes minute changes in the subject's motion in the image. The image processing apparatus 1c includes a generation unit 10b, a detection unit 11b, a reliance unit 12b, a multiplication unit 13b, an emphasis unit 14b, a reconstruction unit 15b, and an integration unit 16c. The image processing apparatus 1 c may further include a storage unit. The storage unit may be included in each functional unit.

  The integration unit 16c acquires the reliability of the phase image of each resolution from the reliance unit 12b. The integration unit 16c integrates the reliability in the phase image of a plurality of resolutions for each direction in the phase image. In the following, the resolution of the original image is written as 1. Also, the lowest resolution is denoted as L.

The integration unit 16c performs integration of the reliability for each direction in the phase image as in Expression (16). That is, the integrating unit 16 c multiplies the reliability in the phase image of all resolutions for all the generated phase images, thereby multiplying the reliability ^ C _θ (x, y, x) common to all the resolutions. t) Calculate ^{β / λ} for each direction θ.

The integration unit 16c may perform integration of the reliability for each direction in the phase image as in Expression (17). That is, the integrating unit 16 c calculates the weighted sum of the reliabilities in the phase images of all the resolutions for all the generated phase images, to thereby obtain the reliability ^ C _θ common to all the resolutions. x, y, t) ^{β / λ} may be calculated for each direction θ.

The integration unit 16c may perform integration of the reliability for each direction in the phase image as in Expression (18). That is, the integration unit 16 c multiplies the reliability for each direction with respect to the phase image of each resolution from resolution 1 to low resolution (1 + N) to obtain the multiplication result as the reliability for each direction θ in the phase image of resolution l. It is good also as degree ^ _Ctheta ^l (x, y, t) ^{beta / lambda} .

The integration unit 16c may perform integration of the reliability for each direction in the phase image as in Expression (19). That is, the integration unit 16 c calculates the weighted sum of the reliabilities for each direction with respect to the phase image of each resolution from the resolution 1 to the low resolution (l + N) to obtain the reliability for each direction θ in the phase image of the resolution l. It may be ^ C _θ ^l (x, y, t) ^{β / λ} .

  The integration unit 16 c can accurately detect a minute change in phase by calculating the equation (14) based on any of the degrees of reliability from the equations (16) to (19).

  FIG. 6 is a flowchart showing an example of the operation of the image processing apparatus 1c. The generation unit 10 b converts the luminance change in the luminance image of the original image into amplitude information and phase information in a plurality of directions in the luminance image. The generation unit 10b generates a phase image of each resolution from the luminance image of the original image (step S301). The detection unit 11 b detects a slight change in phase in the generated phase image of each resolution for each direction in the phase image (step S 302). The trust unit 12b calculates the reliability of the minute change of the phase in the phase image of each resolution generated by the generation unit 10b (step S303). The integration unit 16c integrates the reliability in the phase image of a plurality of resolutions for each direction in the phase image (step S304). The multiplication unit 13b multiplies the output of the detection unit 11b by the output of the integration unit 16c (step S305).

  The emphasizing unit 14 b multiplies the emphasizing rate α by the multiplication result of the output of the detecting unit 11 b and the output of the relieving unit 12 b. That is, the emphasizing unit 14b multiplies the value representing the minute change of the phase in the phase image of each resolution by the emphasis ratio α. The emphasizing unit 14b adds the multiplication result of the emphasis ratio to a value representing phase change in the phase image of the original image (step S306). The reconstruction unit 15b generates an image in which a minute change in the original image is emphasized by combining the addition result of the phase change information and the color image of the original image (step S307).

  As described above, the image processing apparatus 1c according to the third embodiment further includes the integration unit 16c as compared with the image processing apparatus 1b according to the second embodiment. The integration unit 16c integrates the degrees of reliability in phase images of a plurality of resolutions. The emphasizing unit 14 b performs emphasizing processing on the multiplication result of the integrated reliability and the value representing the minute change of the phase. Thus, the image processing apparatus 1c according to the third embodiment can reduce blur noise and generate an image in which a minute change is emphasized. The image processing apparatus 1c according to the third embodiment can detect a minute change in phase more accurately than the detection result according to the second embodiment.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

  1a, 1b, 1c ... image processing apparatus, 10a, 10b ... generation unit, 11a, 11b ... detection unit, 12a, 12b ... reliance unit, 13a, 13b ... multiplication unit, 14a, 14b ... emphasis unit, 15a, 15b ... re Component part, 16c ... integrated part

Claims (6)

A generation unit that generates an image of each resolution from the original image;
A detection unit that detects a minute change in an image of each resolution;
A trust unit for acquiring the reliability of the minute change;
An emphasizing unit that performs emphasizing processing on the multiplication result of the value representing the minute change and the reliability;
An image processing apparatus, comprising: a reconstruction unit that generates an image in which the minute change in the original image is subjected to the emphasis process based on the multiplication result subjected to the emphasis process and the original image. The generation unit generates a luminance image, which is an image representing luminance, for each resolution from the luminance image of the original image.
The detection unit detects a minute change in luminance in the luminance image of each resolution;
The reliability unit acquires the reliability of the minute change of the luminance,
The emphasizing unit performs emphasizing processing on the multiplication result of the value representing the minute change of the luminance and the reliability.
The said reconstruction part produces | generates the image by which the emphasis process was performed to the said minute change in the original image based on the multiplication result to which the emphasis process was performed, and the color image of an original image. Image processing device. The generation unit generates a phase image, which is an image representing a phase, from the phase image of the original image for each resolution and direction;
The detection unit detects a minute change in phase in a phase image of each resolution;
The reliability unit acquires the reliability of the minute change of the phase;
The emphasizing unit performs emphasizing processing on the multiplication result of the value representing the minute change of the phase and the reliability.
The said reconstruction part produces | generates the image by which the emphasis process was performed to the minute change of the said phase in an original image based on the multiplication result to which the emphasis process was performed, and the color image of an original image. Image processing device. The image processing apparatus further comprises an integration unit that integrates the reliability of phase images of multiple resolutions in each direction,
The image processing apparatus according to claim 3, wherein the emphasizing unit performs emphasizing processing on a multiplication result of the reliability integrated with the value representing the minute change of the phase. An image processing method performed by an image processing apparatus, comprising:
Generating an image of each resolution from the original image;
Detecting minute changes in the image of each resolution;
Obtaining the degree of reliability of the minute change;
Emphasizing the multiplication result of the value representing the minute change and the reliability;
Generating an image in which the minute change in the original image is subjected to the enhancement process based on the multiplication result subjected to the enhancement process and the original image. On the computer
Generating an image of each resolution from the original image;
A procedure for detecting minute changes in an image of each resolution;
Obtaining a degree of reliability of the minute change;
A procedure for emphasizing the multiplication result of the value representing the minute change and the reliability;
An image processing program for executing an operation of generating an image in which the minute change in the original image is subjected to the enhancement process based on the multiplication result subjected to the enhancement process and the original image.

Images