JP7357836B2 - Image processing device and image processing program - Google Patents

Description

The present invention relates to an image processing device and an image processing program.

When evaluating the health of a building immediately after a major earthquake occurs, it is important to perform a primary diagnosis to determine whether evacuation is necessary before conducting a detailed diagnosis of the building. In view of this, the inventors of the present invention have proposed in Patent Document 1 a system for diagnosing the health of a building that can perform a primary and simple diagnosis without relying on acceleration sensors or the like.

This system numerically evaluates the health of a building based on the rate of change in the building's natural frequency before and after an earthquake, which is obtained by analyzing video images taken of the building using a camera. can be converted into

When calculating the natural frequency of a target building, the natural frequency can be calculated by detecting minute vibration components in moving images using the techniques disclosed in Patent Documents 2 to 3 and Non-Patent Document 1. When calculating, if the imaging device itself exists in a microvibration environment, it is necessary to distinguish between the vibrations of the imaging device and the vibrations of the building.

If the time-frequency characteristics and spatial frequency characteristics of the vibration of the photographing device and the vibration of the building are sufficiently different, it becomes possible to separate them in the spatio-temporal frequency domain as described in Patent Document 1. However, if the vibration characteristics of the imaging device and the vibration characteristics of the building overlap in the spatio-temporal frequency domain, they cannot be simply separated, making it impossible to correctly evaluate the characteristics such as the natural frequency of the vibration of the building that is originally intended to be measured. There are cases.

On the other hand, there are some physical techniques to reduce the vibration of photographic equipment, such as increasing the weight of the entire photographing system including the photographing equipment and incidental equipment such as tripods, and introducing equipment such as anti-vibration rubber and springs. Anti-vibration measures can also be considered. However, this measure cannot be expected to have a dramatic effect in the frequency band of several Hz or less, which is included in ground vibrations.

Therefore, there is a need for software-based processing that can detect (and even remove) the vibration components of the imaging device. Note that such software measures are not limited to the use of diagnosing the health of buildings using moving photography as described in Patent Document 1, but also include the use of diagnosing the health of buildings using fixed photography. The present invention is generally useful in vibration analysis methods that involve only moving image processing based on spatiotemporal filtering of objects, as typified by Documents 2 and 3 and Non-Patent Document 1.

As a technology related to software-based vibration component detection and removal processing, Patent Document 4 discloses a digital image stabilization technology that is not mechanical or optical. With this technology, it is possible to remove fluctuations in images caused by the movement of the imaging device itself, in visual applications such as viewing and recording, and in matching applications between images such as panoramic composition and three-dimensional reconstruction.

Japanese Patent Application Publication No. 2018-136191 US Patent Application Publication No. 2014/0072190 US Patent No. 9324005 JP2019-004451A

J.G. Chen, A. Davis, N. Wadhwa, F. Durand, W.T. Freeman, and O. Buyukozturk, “Video Camera-based Vibration Measurement for Condition Assessment of Civil Infrastructure”, International Symposium Non-Destructive Testing in Civil Engineering (2015)

However, in visual applications, the main targets that must be detected and removed are fluctuation components that extend over several pixels (pixels), and the technology disclosed in Patent Document 4 is capable of detecting and removing ultrafine fluctuations on the sub-pixel level. There is no mention of detection, etc. In addition, although sub-pixel-level accuracy may be required for matching purposes, it is often based on geometric transformation based on the correspondence between multiple pixels, and it is often If a vibration component from the photographing device is mixed into a subject that does not appear to be moving at first glance, it may not necessarily function correctly.

The present disclosure has been made in view of the above circumstances, and aims to provide an image processing device and an image processing program that can accurately detect minute vibration components of a photographing device from a moving image. .

The image processing device according to the present invention according to claim 1 includes: an acquisition unit that acquires a moving image that includes a plurality of objects as subjects and that is obtained by photographing with a photographing device; an extraction unit that extracts a plurality of object regions, each of which is one of the plurality of objects, in a moving image; and the plurality of object regions extracted by the extraction unit in the moving image acquired by the acquisition unit. an identification unit that performs vibration analysis on each of the plurality of object regions and identifies a vibration component that is common among each of the plurality of object regions as a vibration component of the photographing device.

According to the image processing device according to the present invention as set forth in claim 1, a plurality of object regions, each of which is one of the regions of a plurality of objects, are extracted from a moving image obtained by photographing with a photographing device; By performing vibration analysis on each of the plurality of object regions in the image and identifying a common vibration component between each of the plurality of object regions as a vibration component of the photographing device, it is possible to capture images from a moving image. It is possible to detect minute vibration components of the device with high precision.

An image processing apparatus according to the present invention according to claim 2 is the image processing apparatus according to claim 1, in which the extraction section extracts an area in which the S/N ratio included in the moving image is equal to or higher than a predetermined level. is detected, and each area of a spatially continuous partial pixel group in the detected area is extracted as the plurality of object areas.

According to the image processing device according to the present invention as set forth in claim 2, a region included in a moving image where the S/N ratio is equal to or higher than a predetermined level is detected, and a group of spatially continuous partial pixels in the detected region is detected. By extracting each region as the plurality of object regions, the plurality of object regions can be extracted more easily and with high reliability.

The image processing device according to the present invention according to claim 3 is the image processing device according to claim 1 or 2, which generates a phase image by performing complex spatial filtering processing on the moving image. and a phase variation signal that is a signal indicating a variation between each frame image in the moving image of the phase image generated by the generation unit, for the plurality of object regions extracted by the extraction unit. a derivation unit; a conversion unit that converts the phase fluctuation signal derived by the derivation unit into a time-frequency spectrum by frequency analysis; and a time-frequency spectrum in the same region using the time-frequency spectrum obtained by the conversion unit. further comprising a calculation unit that calculates, for each of the plurality of object regions, a phase fluctuation spectrum that is an average of A predetermined frequency range including the peak frequency is identified as a vibration component of the photographing device.

According to the image processing device according to the present invention as set forth in claim 3, a phase image is generated by performing complex spatial filtering processing on a moving image, and the moving image of the generated phase image is generated for the plurality of object regions. A phase fluctuation signal, which is a signal indicating the fluctuation between each frame image in the image, is derived, the derived phase fluctuation signal is converted into a time-frequency spectrum by frequency analysis, and the time-frequency spectrum is used to calculate the time within the same area. A phase fluctuation spectrum obtained by averaging the frequency spectra is calculated for each of the plurality of object regions, and in the calculated phase fluctuation spectrum, a predetermined frequency range including a peak frequency common to the plurality of object regions is determined by the vibration of the imaging device. By specifying it as a component, it is possible to specify the vibration component of the photographing device with higher accuracy.

The image processing device according to the present invention according to claim 4 is the image processing device according to claim 3, in which, when the phase image is a wrapped phase, the derivation unit The phase fluctuation signal is derived after unwrapping the phase of the pixel.

According to the image processing device according to the present invention as set forth in claim 4, when the phase image has a wrapped phase, the phase fluctuation signal is processed after unwrapping the phase of each pixel of the phase image. By deriving , the phase fluctuation signal can be derived with higher accuracy.

The image processing program according to the present invention according to claim 5 acquires a moving image in which a plurality of objects are included as subjects and is obtained by photographing with a photographing device, and each of the plurality of objects in the acquired moving image. A plurality of object regions, which are any regions of the object, are extracted, and a vibration analysis is performed on each of the plurality of extracted object regions in the acquired video image, and vibration analysis is performed on each of the plurality of object regions, which is common to each of the plurality of object regions. A computer is caused to execute a process of identifying the vibration component as a vibration component of the photographing device.

According to the image processing program according to the present invention as set forth in claim 5, a plurality of object regions, each of which is one of a plurality of object regions, in a moving image obtained by photographing with a photographing device are extracted, and By performing vibration analysis on each of the plurality of object regions in the image and identifying a common vibration component between each of the plurality of object regions as a vibration component of the photographing device, it is possible to capture images from a moving image. It is possible to detect minute vibration components of the device with high precision.

As described above, according to the present invention, minute vibration components of the photographing device can be detected with high accuracy from a moving image.

FIG. 1 is a block diagram illustrating an example of a hardware configuration of an image processing device according to an embodiment. FIG. 1 is a block diagram illustrating an example of a functional configuration of an image processing device according to an embodiment. FIG. 1 is a schematic diagram showing an example of the configuration of a moving image database according to an embodiment. It is a flow chart which shows an example of vibration component identification processing concerning an embodiment. FIG. 2 is a front view showing an example of a moving image (representative image) according to the embodiment. It is a front view showing an example of a phase image (horizontal component) concerning an embodiment. It is a front view showing an example of a phase image (vertical component) concerning an embodiment. FIG. 2 is a front view showing an example of a partial image of a moving image according to an embodiment. 7A is a front view showing an example of a result of labeling processing performed on the image shown in FIG. 7A. FIG. 3 is a graph showing an example of a phase fluctuation signal according to an embodiment. It is a front view showing an example of a sample image (sine wave image) concerning an embodiment. It is a graph showing an example of a time frequency spectrum concerning an embodiment. It is a graph which shows an example of the time frequency spectrum used for explanation of the demonstration experiment based on embodiment.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In this embodiment, a case will be described in which the present invention is applied to an image processing apparatus that processes a moving image of a building in a state of slight movement under the influence of wind excitation or ground vibration.

First, the configuration of an image processing apparatus 10 according to the present embodiment will be described with reference to FIGS. 1 and 2. Note that examples of the image processing device 10 include information processing devices such as a personal computer and a server computer.

As shown in FIG. 1, the image processing device 10 according to the present embodiment includes a CPU (Central Processing Unit) 11, a memory 12 as a temporary storage area, a nonvolatile storage section 13, an input section 14 such as a keyboard and a mouse, It includes a display section 15 such as a liquid crystal display, a medium read/write device (R/W) 16, and a communication interface (I/F) section 18. The CPU 11, memory 12, storage section 13, input section 14, display section 15, medium reading/writing device 16, and communication I/F section 18 are connected to each other via a bus B1. The medium read/write device 16 reads information written in the recording medium 17 and writes information to the recording medium 17 .

The storage unit 13 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. A vibration component identification program 13A is stored in the storage unit 13 as a storage medium. The vibration component identification program 13A is created by setting the recording medium 17 on which the vibration component identification program 13A has been written into the medium reading/writing device 16, and reading the vibration component identification program 13A from the recording medium 17. It is stored in the storage unit 13. The CPU 11 reads the vibration component identification program 13A from the storage unit 13, expands it into the memory 12, and sequentially executes the processes included in the vibration component identification program 13A. The storage unit 13 also stores various databases such as a moving image database 13B and a complex spatial filter database 13C.

In the image processing device 10 according to the present embodiment, a photographing device 20 that photographs moving images is connected to the communication I/F section 18 . The photographing device 20 is for photographing so as to include a plurality of buildings at the time of photographing. Note that the photographing method using the photographing device 20 may be any method such as aerial photographing, moving photographing by a person on the ground, or fixed photographing using a tripod or a fixed member. Further, in the present embodiment, a photographing device that takes a color image is used as the photographing device 20, but the invention is not limited to this. For example, a photographing device that takes a monochrome image can be applied as the photographing device 20. It may also be a form.

Next, with reference to FIG. 2, the functional configuration of the image processing device 10 according to this embodiment will be described. As shown in FIG. 2, the image processing device 10 includes an acquisition section 11A, an extraction section 11B, a generation section 11C, a derivation section 11D, a conversion section 11E, a calculation section 11F, and a specification section 11G. By executing the vibration component specifying program 13A, the CPU 11 of the image processing device 10 functions as an acquisition section 11A, an extraction section 11B, a generation section 11C, a derivation section 11D, a conversion section 11E, a calculation section 11F, and a specification section 11G.

The acquisition unit 11A according to the present embodiment acquires a moving image that includes a plurality of objects as subjects and that is obtained by photographing with the photographing device 20. Note that in this embodiment, a building is used as the object, but the object is not limited to this. For example, buildings other than buildings such as bridges and towers, natural objects such as mountains and trees, biological information such as pulsation, equipment such as air conditioning ducts and transformers, or combinations of multiple types of these can be applied as the above objects. It may also be in the form of

Further, the extraction unit 11B according to the present embodiment extracts a plurality of object regions, each of which is one of the plurality of objects, in the moving image acquired by the acquisition unit 11A. The identification unit 11G according to the present embodiment performs vibration analysis on each of the plurality of object regions extracted by the extraction unit 11B in the moving image acquired by the acquisition unit 11A, and A vibration component that is common between the images is identified as a vibration component of the photographing device 20. The extraction unit 11B according to the present embodiment detects a region in which the S/N ratio (Signal to Noise ratio) included in the video image is equal to or higher than a predetermined level, and extracts a group of spatially continuous partial pixels in the detected region. Each region is extracted as the plurality of object regions.

On the other hand, the generation unit 11C according to the present embodiment generates a phase image by performing complex spatial filtering processing on the moving image. Further, the derivation unit 11D according to the present embodiment generates a signal indicating a variation between each frame image in the moving image of the phase image generated by the generation unit 11C for the plurality of object regions extracted by the extraction unit 11B. Derive the phase variation signal. Further, the converting unit 11E according to the present embodiment converts the phase fluctuation signal derived by the deriving unit 11D into a time-frequency spectrum by frequency analysis. Further, the calculation unit 11F according to the present embodiment calculates a phase fluctuation spectrum obtained by averaging the time-frequency spectra in the same area for each of the plurality of object areas using the time-frequency spectrum obtained by the conversion unit 11E. do.

Here, in the phase variation spectrum calculated by the calculation unit 11F, the identification unit 11G according to the present embodiment identifies a predetermined frequency range including a peak frequency common to the plurality of object regions as a vibration component of the imaging device 20. Specify as. Furthermore, when the phase image has a wrapped phase, the derivation unit 11D according to the present embodiment performs an unwrapping process on the phase of each pixel of the phase image, and then derives a phase fluctuation signal.

Next, with reference to FIG. 3, the moving image database 13B according to this embodiment will be described. As shown in FIG. 3, the moving image database 13B according to the present embodiment stores moving image information obtained by photographing a moving image with the photographing device 20 for each moving image ID (Identification) assigned in advance. There is. In this manner, in this embodiment, moving image information is captured in advance from the photographing device 20 and registered in the moving image database 13B, but the present invention is not limited to this. For example, a configuration may be adopted in which the image capturing device 20 constantly performs image capturing, and the moving image information obtained from the image capturing device 20 when a vibration of a predetermined level or higher occurs is used online, in real time or non-real time.

On the other hand, in the complex space filter database 13C according to this embodiment, information indicating a predetermined complex space filter (in this embodiment, a complex Gabor filter) is registered. However, the complex spatial filter is not limited to the complex Gabor filter, but is a complex spatial filter that is a set of spatial filters (real part filter, imaginary part filter) whose spatial phase characteristics differ by 90 degrees and whose spatial amplitude characteristics are equal. If available, other filters may be applied as complex space filters.

Next, the operation of the image processing device 10 according to this embodiment will be explained with reference to FIGS. 4 to 10. When the user inputs an instruction to start executing the vibration component identification program 13A through the input unit 14, the CPU 11 of the image processing device 10 executes the vibration component identification program 13A, thereby executing the program shown in FIG. Vibration component identification processing is executed. Here, in order to avoid confusion, a case will be described in which the moving image database 13B and the complex spatial filter database 13C have been constructed, and the moving image information to be processed is specified by the user.

At step 200 in FIG. 4, the acquisition unit 11A acquires moving image information specified by the user (hereinafter referred to as "processing target moving image information") by reading it from the moving image database 13B.

In step 202, the generation unit 11C reads information indicating a complex space filter from the complex space filter database 13C, and performs complex space filtering using the complex space filter (in this embodiment, a complex Gabor filter) on the processing target video information. Processing is performed to generate a phase image.

That is, first, the generation unit 11C performs a complex space filtering process using the read complex space filter to generate the real part image _Ire and the imaginary part from each frame image of the moving image indicated by the processing target moving image information. Calculate the image I _im . Next, the generation unit 11C calculates the phase image I _θ by performing calculation according to the following equation (1) for each pixel.

I _θ = tan ⁻¹ (I _im /I _re ) (1)

The above processing is performed on all frame images of the moving image indicated by the processing target moving image information. For the complex spatial filtering process, either a method using convolution kernel convolution on the spatial domain or a method using filter product on the spatial frequency domain may be applied.

For example, if one of the moving images indicated by the processing target moving image information is the image shown in FIG. The phase image of the vertical component is as shown in FIG. 6B. Note that, for convenience, the image shown in FIG. 5 is a photograph of a structure created by the inventors of the present invention, rather than a building as a subject.

In step 204, the extraction unit 11B determines a region (hereinafter referred to as a "processing target region") in which the vibration component of the photographing device 20 is to be detected in the derived phase image _Iθ . In this embodiment, as a method for determining the processing target area, a method is applied in which a group of pixels having an S/N ratio of a predetermined level or higher is set as the processing target area. Here, as an example of a pixel group whose S/N ratio is a predetermined level or higher, in a moving image indicated by the processing target moving image information at the stage acquired by the acquisition unit 11A, a spatial first-order differential filter of a frame image (for example, This corresponds to an output image that has been subjected to an edge detection process that detects an edge image. If it is necessary to remove spike-like noise components and secure a cluster of regions, a median filter, dilation processing, and contraction processing are used in combination. The extraction unit 11B finally determines the processing target area by performing binarization using threshold processing.

Note that the method for determining the region to be processed is not limited to the above method, and for example, a region of interest designated in advance by the user may be determined as the region to be processed.

In step 206, the extraction unit 11B performs a labeling process on the binarized image corresponding to the processing target area obtained by the process in step 204, and separates a group of spatially continuous pixels into one block. , divides the processing target area into a plurality of partial pixel groups. The plurality of partial pixel groups obtained through this process correspond to the plurality of object regions described above, and hereinafter, the partial pixel groups will be referred to as object regions.

FIG. 7B shows an example of an object area obtained by the process in step 206 when the image to be processed is shown in FIG. 7A, which is a part of the image shown in FIG. There is. Note that in FIG. 7B, different densities are shown for each object region, and in the example shown in FIG. 7B, six object regions are extracted.

In step 208, the derivation unit 11D performs phase unwrapping processing on the phase image _Iθ obtained by the processing in step 202. That is, the phase information is generally wrapped in the range of -π to +π (ie, for example, π+π/4→-π/4). Therefore, in this embodiment, phase unwrap processing (phase connection processing) is performed. Examples of phase unwrapping processing include the Internet (URL: https://www.researchgate.net/publication/265151826), (URL: http://retrofocus28.blogspot.com/2013/12/phase-unwrapping_26.html) , (URL: https://jp.mathworks.com/help/dsp/ref/unwrap.html#f5-1119858) etc. can be applied. It goes without saying that if the derived phase image I _θ is not wrapped, there is no need to perform the process of step 208.

In step 210, the derivation unit _11D selects one of the object regions (hereinafter referred to as "processing target The above-mentioned phase fluctuation signal, which is time-series data obtained by cutting the phase image _Iθ as it is in the time direction at each pixel of the object region (referred to as "object region"), is calculated.

FIG. 8 shows an example of a phase fluctuation signal obtained by the process of step 210 before and after the phase connection by the phase unwrap process when the image to be processed is the one shown in FIG. 7A. In the example shown in FIG. 8, the phase fluctuation signal before phase connection is shown by a broken line, and the phase fluctuation signal after phase connection is shown by a solid line.

As shown in FIG. 8, due to the phase connection, a signal that has been folded over +180 degrees to -180 degrees is continuously expressed over +180 degrees. However, since the phase unwrapping process used in the example of Figure 8 does not assume that the phase rotates more than two times, the folding of the signal where the value of the horizontal axis in Figure 8 exceeds +360 degrees around the 120th frame remains. It's still there. This problem can be resolved by subjecting the unwrapped signal to the phase unwrapping process again. However, although the present invention targets weak vibrations at the sub-pixel level, if phase wrapping occurs, it can be interpreted as a large movement that exceeds the sub-pixel level. It is also possible to use phase unwrap processing for the purpose of detecting that a group or image region is to be excluded from the processing target region.

In step 212, the conversion unit 11E converts the phase fluctuation signal of the object region to be processed obtained by the processing in step 210 into a time-frequency spectrum, which is a signal in the time-frequency domain, by frequency analysis.

FIG. 10 shows a video in which the sine wave image of 100 pixels x 100 pixels with a spatial wavelength of 8 pixels shown in FIG. An example of a time-frequency spectrum obtained by the processing of step 212 in the image is shown. In the example shown in FIG. 10, a Gaussian function type bandpass filter having a peak at the spatial wavelength λ=8 pixels is used as the complex spatial filter.

In step 214, the calculation unit 11F calculates, for the time-frequency spectrum of the object region to be processed obtained by the processing in step 212, a time-frequency spectrum obtained by averaging the time-frequency spectra in the same region, as the above-mentioned phase fluctuation spectrum. Calculated as

In step 216, the calculation unit 11F determines whether or not the processing in steps 210 to 214 has been completed for all object regions, and if the determination is negative, the process returns to step 210, while if the determination is affirmative. At that point, the process moves to step 218. Note that when repeatedly executing the processing from step 210 to step 216, the CPU 11 sets an object region that has not been targeted for processing as an object region for processing.

In step 218, the identifying unit 11G estimates that the peak frequency common to each object region in the phase variation spectra of all object regions is a peak due to the vibration component of the imaging device 20, and specifies a predetermined frequency that includes the peak frequency. After specifying the range as a vibration component of the imaging device 20 and storing information indicating the specified vibration component in a predetermined area of the storage unit 13, the main vibration component identification process ends.

Next, a verification experiment regarding identification of the vibration component of the photographing device by the image processing device 10 according to the present embodiment will be described.

FIG. 11 shows an example of an analysis result by the image processing apparatus 10 when the moving image shown in FIG. 5 is applied as a processing target. Note that the example shown in FIG. 11 shows a case where the subject is vibrating at 13 Hz, the photographing device 20 is vibrating at 8 Hz, and four areas are arbitrarily extracted as object areas. .

As shown in FIG. 11, it was confirmed that 8 Hz, whose peak was detected across multiple object regions, could be identified as the vibration component of the imaging device 20.

As described above, according to the present embodiment, the acquisition unit 11A acquires a moving image in which a plurality of objects are included as subjects and is obtained by photographing with a photographing device, and the moving image acquired by the acquisition unit 11A. an extraction unit 11B that extracts a plurality of object regions, each of which is one of the plurality of objects, in the image; and the plurality of object regions extracted by the extraction unit 11B in the moving image acquired by the acquisition unit 11A. and a specifying unit 11G that performs vibration analysis on each of the plurality of object regions and specifies a vibration component that is common among each of the plurality of object regions as a vibration component of the photographing device. Therefore, minute vibration components of the photographing device can be detected with high accuracy from the moving image.

Further, according to the present embodiment, an area in which the S/N ratio included in a moving image is equal to or higher than a predetermined level is detected, and each area of a spatially continuous partial pixel group in the detected area is divided into the plurality of object areas. It is extracted as Therefore, the plurality of object regions can be extracted more easily and with high reliability.

Further, according to the present embodiment, a phase image is generated by performing complex space filtering processing on a moving image, and with respect to the plurality of object regions, variation between each frame image of the generated phase image in the moving image is A phase fluctuation signal that is a signal indicating A spectrum is calculated for each of the plurality of object regions, and in the calculated phase variation spectrum, a predetermined frequency range including a peak frequency common to the plurality of object regions is identified as a vibration component of the imaging device. . Therefore, the vibration component of the photographing device can be specified with higher accuracy.

Further, according to the present embodiment, when the phase image has a wrapped phase, the phase fluctuation signal is derived after unwrapping the phase of each pixel of the phase image. Therefore, the phase fluctuation signal can be derived with higher accuracy.

In the above embodiment, for example, the hardware of the processing unit that executes each process of the acquisition unit 11A, extraction unit 11B, generation unit 11C, derivation unit 11D, conversion unit 11E, calculation unit 11F, and identification unit 11G As the hardware structure, the following various processors can be used. As mentioned above, the various processors mentioned above include the CPU, which is a general-purpose processor that executes software (programs) and functions as a processing unit, as well as circuit configurations such as FPGA (Field-Programmable Gate Array) after manufacturing. A programmable logic device (PLD), which is a processor that can be changed, and a dedicated electric circuit, which is a processor that has a circuit configuration specifically designed to execute a specific process, such as an ASIC (Application Specific Integrated Circuit) etc. are included.

The processing unit may be configured with one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). It may be composed of. Further, the processing section may be configured with one processor.

As an example of configuring the processing unit with one processor, first, as typified by computers such as clients and servers, one processor is configured with a combination of one or more CPUs and software, and this processor is There is a form that functions as a processing section. Second, there is a form of using a processor, such as a system on chip (SoC), in which the functions of the entire system including a processing section are realized by one IC (Integrated Circuit) chip. In this way, the processing section is configured as a hardware structure using one or more of the various processors described above.

Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements can be used.

10 Image processing device 11 CPU
11A Acquisition unit 11B Extraction unit 11C Generation unit 11D Derivation unit 11E Conversion unit 11F Calculation unit 11G Specification unit 12 Memory 13 Storage unit 13A Vibration component identification program 13B Moving image database 13C Complex space filter database 14 Input unit 15 Display unit 16 Media reading/writing device 17 Recording medium 18 Communication I/F unit 20 Photographing device

Claims (5)

an acquisition unit that acquires a moving image that includes a plurality of objects as subjects and that is obtained by photographing with a photographing device;
an extraction unit that extracts a plurality of object regions, each of which is one of the plurality of objects, in the moving image acquired by the acquisition unit;
A vibration analysis is performed on each of the plurality of object regions extracted by the extraction section in the moving image obtained by the acquisition section, and a vibration component that is common between each of the plurality of object regions is detected by the imaging device. a specific part that is identified as being a vibration component of
An image processing device equipped with The extraction unit detects a region in the video image where the S/N ratio is equal to or higher than a predetermined level, and extracts each region of a spatially continuous partial pixel group in the detected region as the plurality of object regions. ,
The image processing device according to claim 1. a generation unit that generates a phase image by performing complex space filtering processing on the moving image;
a derivation unit that derives a phase fluctuation signal that is a signal indicating a variation between each frame image in the moving image of the phase image generated by the generation unit for the plurality of object regions extracted by the extraction unit;
a conversion unit that converts the phase fluctuation signal derived by the derivation unit into a time-frequency spectrum by frequency analysis;
a calculation unit that calculates, for each of the plurality of object regions, a phase variation spectrum obtained by averaging the time-frequency spectra in the same region using the time-frequency spectrum obtained by the conversion unit;
further comprising;
The identifying unit identifies a predetermined frequency range including a peak frequency common to the plurality of object regions as a vibration component of the imaging device in the phase fluctuation spectrum calculated by the calculating unit.
The image processing device according to claim 1 or claim 2. When the phase image is a wrapped phase, the derivation unit derives the phase fluctuation signal after performing an unwrapping process on the phase of each pixel of the phase image.
The image processing device according to claim 3. Obtaining a moving image containing multiple objects as subjects and obtained by shooting with a shooting device,
extracting a plurality of object regions, each of which is one of the plurality of objects, in the acquired video image;
performing a vibration analysis on each of the plurality of extracted object regions in the acquired moving image, and identifying a vibration component that is common among each of the plurality of object regions as a vibration component of the photographing device;
An image processing program that allows a computer to perform processing.