JP2022044393A - Method and device for analyzing image - Google Patents

Abstract

To determine motions of an imaging target.SOLUTION: A controller generates a first image including a first speckle pattern on the basis of a first imaging signal, generates a second image including a second speckle pattern on the basis of a second imaging signal generated in such a manner that an imaging device takes an image of the imaging target in a second time, approximates the displacement of a first feature vector in a first pixel aggregate in the first image, approximates the displacement of a second feature vector in a second pixel aggregate in a second image, and calculates the distance between pixels on a pixel-by-pixel basis between the first feature vector and the second feature vector, thereby determining the change between the first speckle pattern and the second speckle pattern.SELECTED DRAWING: Figure 5

Description

Application for application of Article 30, Paragraph 2 of the Patent Act Date: August 3, 2nd Reiwa Meeting name, venue: 23rd Image Recognition and Understanding Symposium MIRU2020 Online Holding https: // systems. Google. com / view / miru2020 /% E3% 83% 97% E3% 83% AD% E3% 82% B0% E3% 83% A9% E3% 83% A0 # h. p_vdNjmJ1fY_NX

The present disclosure relates to an image analysis method and an image analysis apparatus, and more particularly to an image analysis method and an image analysis apparatus for determining a slight movement of an imaging object based on speckles caused by interference of light reflected from the imaging object.

When the surface of an object is irradiated with light such as laser light, the reflected light is scattered and interferes with the uneven shape of the irradiation surface of the object, and a speckled pattern appears on the irradiation surface. This pattern is called speckle. When observing an image generated from an object irradiated with coherent light, speckles are observed in the image. Then, when the object moves, the speckle pattern also changes according to the movement.

Special Table 2019-591269 Gazette

Conventionally, a technique for determining the movement of an object irradiated with light by analyzing a speckle pattern in an image has been proposed. Patent Document 1 describes mechanical and / or acoustics, such as mechanical and / or acoustic vibrations associated with biological specimens, by collecting pieces of image data associated with secondary speckle patterns. It discloses a technology for monitoring target signals.

The technique disclosed in Patent Document 1 determines a correlation function indicating a spatial correlation between speckle patterns, and applies filtering to the determined correlation function in order to select a change in correlation that occurs at frequency. There is.

For example, if the distance between the camera and the object is short, the speckle pattern can change abruptly, even if the object moves slightly. In such a case, when the speckle pattern is continuously observed, the patterns become uncorrelated before and after the change, and it is not possible to determine the continuous slight movement of the object. Patent Document 1 does not present any solution for determining the slight movement of the above-mentioned object.

The computer device according to an embodiment is a computer device that determines the movement of an image-taking object, and is based on a first image-taking signal generated by the image pickup device taking an image of the image-taking object at a first time. A second speckle is generated based on a second imaging signal generated by generating a first image including the first speckle pattern and the imaging device photographing the imaged object at a second time. A second image containing a pattern is generated, a first feature vector is calculated from the brightness in the first pixel set in the first image, and the first feature vector is generated among the first pixel sets. The displacement of the second feature vector is approximated, the second feature vector is calculated from the brightness in the second pixel set in the second image, and the displacement of the second feature vector is approximated between the second pixel sets. The first speckle pattern and the second speckle pattern are obtained by calculating the distance between the pixels for each pixel between the approximated first feature vector and the approximated second feature vector. Includes a controller configured to determine changes to and from the speckle pattern.

Further, the method according to another embodiment is a method executed by a computer device that determines the movement of the imaged object, and is a first method generated by the image pickup device taking an image of the imaged object at the first time. A step of generating a first image including a first speckle pattern based on the imaging signal of the above, and a second imaging signal generated by the imaging apparatus photographing the imaged object at a second time. Based on, the first feature vector is calculated from the step of generating the second image including the second speckle pattern and the brightness in the first pixel set in the first image, and the first feature vector is calculated. The second feature vector is calculated from the step of approximating the displacement of the first feature vector between the pixel sets and the brightness in the second pixel set in the second image, and the second feature vector is calculated. The distance between pixels is calculated for each pixel between the step of approximating the displacement of the second feature vector and the approximated first feature vector and the approximated second feature vector. Thereby, the step of determining the change between the first speckle pattern and the second speckle pattern is included.

According to the image analysis system according to the embodiment, the correlation can be maintained before and after the change of the speckle pattern, and the slight movement of the object can be accurately determined.

It is a block diagram which shows the example of the structure of an image analysis system. It is a block diagram which shows the example of the configuration of a computer device. It is a figure which shows the example of taking a picture of the object to be taken. It is a figure which shows the example of the local plane. It is a flowchart which shows the example of the process which an image analysis system executes. It is a figure which shows the example of the local plane. It is a figure which shows the example of the state which pressed the cloth with a finger. It is a graph which shows the example of the change of vibration.

Hereinafter, the image analysis method and the image analysis apparatus according to the embodiment will be described in detail with reference to the attached drawings. In this embodiment, the image analysis method and the image analysis device are implemented in the image analysis system. Further, in the present embodiment, any object that is irradiated with light and whose movement or the like is determined is referred to as a “photographing target”.

First, an example of the configuration of the image analysis system 100 will be described with reference to FIG. In the present embodiment, the image analysis system 100 includes a computer device 1, an image pickup device 2a, 2b, and 2c (collectively, an image pickup device 2), a floodlight device 3a, 3b, and 3c (collectively, a floodlight device). 3), as well as the display device 4. The computer device 1 is coupled to at least the image pickup device 2 and the display device 4 via a bus, a network, or the like, and is optionally coupled to the floodlight device 3.

The computer device 1 is any information processing device including at least an arithmetic function. The computer device 1 generates an image based on the shooting signal received from the image pickup device 2, and determines the movement of the shooting target O based on the speckle pattern in the image.

The image pickup apparatus 2 includes one or a plurality of CCD image sensors, CMOS image sensors, and the like, captures an image capture target O, and transmits an image pickup signal to the computer device 1. In this embodiment, the image pickup device 2 is mounted by a camera. In the image pickup apparatus 2, it is desirable that more image sensors are arranged in order to generate a high pixel image.

In this embodiment, a plurality of image pickup devices 2 (imaging devices 2a, 2b, and 2c) for photographing a plurality of image pickup objects O at different positions are provided in order to determine the continuous movement of the image pickup object O. However, the number of image pickup devices 2 may be any number of 1 or more. For example, when one image pickup device 2 is provided, the image pickup device 2 may be a rotary and / or movable light source capable of photographing the image pickup target O in all directions.

The light projecting device 3 is a light source that irradiates coherent light toward the photographing target O. In coherent light, the phase relationship of light waves at any two points in the luminous flux is constant with no change in time, and even if the luminous flux is divided by an arbitrary method and then superposed again with a large optical path difference, complete interference occurs. Light that indicates sex. In the present embodiment, the floodlight device 3 emits laser light as coherent light.

Similar to the image pickup device 2, a plurality of light projecting devices 3 (light projecting devices 3a and 3b) that irradiate a plurality of shooting target O at different positions with laser light in order to determine the continuous movement of the shooting target O. , And 3c) are provided, but the number of the floodlight devices 3 may be any number of 1 or more. For example, when one floodlight device 3 is provided, the floodlight device 3 may be a rotary and / or movable light source capable of emitting laser light in all directions.

The display device 4 is a display that displays the movement of the object as a result of the computer device 1 determining the movement of the object. The display device 4 is not an essential component in the present embodiment, but when the present embodiment is applied, for example, for observing the vibration of a sound transmitted to an object, the vibration of the sound is visually expressed. can do.

In the present embodiment, the computer device 1, the image pickup device 2, the floodlight device 3, and the display device 4 are each mounted by independent devices, but the configuration is not limited to such. There may be a single device in which they are all integrated, or there may be a device in which any one of them is combined.

Next, with reference to FIG. 2, detailed components of the computer device 1 will be described. The computer device 1 includes a control device 11, a memory 12, a storage device 13, an input driver 14, and an output driver 15. The memory 12, the storage device 13, the input driver 14, and the output driver 15 are each coupled to the control device 11 through the internal bus and controlled by the control device 11.

The control device 11 is also referred to as a processor and includes a central processing unit (CPU), a graphic processing unit (GPU), and the like. The control device 11 generates an image based on the shooting signal received from the image pickup device 2, and executes an operation such as determining the movement of the shooting target O based on the speckle pattern in the image. The control device 11 may control the image pickup device 2 and the floodlight device 3 coupled to the computer device 1. In this case, for example, the control device 11 may control the timing at which the light projecting device 3 irradiates the image pickup target O with the laser beam, the timing at which the image pickup device 2 captures the image pickup target O, and the like (the image pickup device 2 and the image pickup device 2 and the image pickup device 2). By transmitting a control signal to the floodlight device 3).

The memory 12 is a volatile data storage device that stores computer-executable instructions processed by the control device 11, data after arithmetic processing by the instructions, and the like. The memory 12 may be implemented by a RAM (random access memory) (for example, a SRAM (static RAM) and a DRAM (dynamic RAM)).

The storage device 13 is a non-volatile data storage device that stores a program or the like including the computer-executable instructions described above. The storage device 13 may be mounted on a non-volatile semiconductor memory such as a ROM (read-only memory), a magnetic storage device (hard disk drive or the like), an optical disk, or the like. Data such as programs may be stored in NAS (Network Attached Storage) and / or SAN (Storage Area Network) in addition to or instead of the storage device 13.

The input driver 14 receives an input from an external device coupled to the computer device 1, such as receiving a shooting signal from the image pickup device 2, and transmits the input to the control device 11. The output driver 15 receives an output to an external device coupled to the computer device 1 such as transmitting an image signal to the display device 4 from the control device, and transmits the output to the external device.

Next, with reference to FIG. 3, the speckle pattern in the image generated by the imaging device 2 photographing the image target O will be described. As described above, when the object O to be photographed moves, the laser beam emitted by the floodlight device 3 scatters and interferes with the movement, and the speckle pattern also changes.

FIG. 3 shows three images generated by photographing the photographing object O moving in the direction indicated by the broken line arrow A at three photographing points. The imaging device 2a photographs the imaging target O at the imaging point S1, the imaging device 2b photographs the imaging target O at the imaging point S2, and the imaging device 2c photographs the imaging target O at the imaging point S3. In the present embodiment, the photographing target O is photographed at each of the three different shooting points S1 to S3, but the photographing target O may be photographed at the same position. That is, the photographing target O is photographed a plurality of times at different positions or at the same position at different time points.

The image generated (by the control device 11) by the image pickup device 2a photographing the image pickup target O corresponds to the image I1. Similarly, the image generated by the image pickup device 2b photographing the image pickup target O corresponds to the image I2, and the image generated by the image pickup device 2c photographing the image capture target O corresponds to the image I3. ..

As described above, when the continuous movement of the shooting target O is determined by analyzing the speckle pattern, the shooting target O is shot at a plurality of shooting points. At each shooting point, the laser light emitted from the light projecting device 3 scatters and interferes, causing a change in the speckle pattern. As shown in FIG. 3, the speckle patterns in the three images I1 to I3 are different from each other.

In this embodiment, a specific area in the image including the speckle pattern is regarded as a plane, and 3D plane parameters around a specific pixel (x, y) in the plane are estimated. In this embodiment, this specific plane region is referred to as a "local plane". With reference to FIG. 4, the local planes in the three images generated by photographing the moving object O (three times) will be described.

FIG. 4 shows three local planes LP1 to LP3. The local plane LP1 corresponds to the specific local plane in the image I1 shown in FIG. The local plane LP2 corresponds to the local plane corresponding to the local plane LP1 in the image I2 shown in FIG. The local plane LP3 corresponds to the local plane corresponding to the local plane LP1 and the local plane LP2 in the image I3 shown in FIG.

As described with reference to FIG. 3, since the images I1 to I3 are images generated by photographing the moving object O, the local plane LP2 is the coordinate position in the image with respect to the local plane LP1. It is displaced. Similarly, the local plane LP3 is also displaced at the coordinate position in the image with respect to the local plane LP1 and the local plane LP2.

FIG. 4 shows the displacements of the local plane LP2 and the local plane LP3 with respect to the local plane LP1. Displacement is represented by the distance between pixels in two local planes. In the example shown in FIG. 4, the specific pixel P1a in the local plane LP1 and the adjacent pixel P1b around the pixel P1a are referred to.

First, the local plane is regarded as the frame i, and the 3D plane parameters around the pixels P1a (x, y) in the frame i are estimated according to the equation (1).

は、３次元ベクトル、すなわち、（ｘ，ｙ）の各点で局所平面を定義する３次元パラメータを表す関数であり、

Is a three-dimensional vector, that is, a function representing a three-dimensional parameter that defines a local plane at each point (x, y).

は、（ｘ，ｙ）の近隣の画素Ｐ１ｂを表す。

Represents a pixel P1b in the vicinity of (x, y).

As shown in FIG. 4, the 3D plane parameter around the pixel P2a in the local plane LP2 is represented as ψ _i (x, y), and the 3D plane parameter around the pixel P3b in the local plane LP3 is.

Expressed as.

As described above, since the images I1 to I3 shown in FIG. 3 represent the movement of the image shooting target O, the offset between the pixel P1a in the local plane LP1 and the pixel P2a in the local plane LP2 is the displacement d1 (d). It is expressed as (x, y)). The displacement d1 corresponds to the movement of the imaging target O between the imaging point S1 and the imaging point S2 shown in FIG.

The displacement d1 is calculated according to the equation (2).

Similarly, the offset between the pixel P1b in the local plane LP1 and the pixel P2b in the local plane LP3 is the displacement d2 (

). The displacement d2 corresponds to the movement of the imaging target O between the imaging point S1 and the imaging point S3 shown in FIG.

By calculating the displacements d1 and d2 described above, changes in the speckle pattern in the image can be analyzed. The speckle pattern is easily affected by the relative position between the image pickup apparatus 2 and the image pickup target O. Therefore, the movement of the image target O can be determined by analyzing the change in the speckle pattern in each image generated by photographing the image target O a plurality of times.

When the speckle pattern changes abruptly, the speckle pattern becomes uncorrelated before and after the change. This leads to the inability to determine the continuous movement of the imaging target O and the inability to accurately determine a slight change. The image analysis system 100 according to the present embodiment determines the movement of the photographing target O based on the speckle pattern while maintaining the consistency of the speckle pattern regardless of the change in the speckle pattern.

The process executed by the image analysis system 100 will be described with reference to the flowchart of FIG. The process described with reference to FIG. 5 analyzes changes in the speckle pattern in the images I1 and I2 generated by photographing the moving image-taking target O at the image-taking points S1 and S2.

First, the light projecting device 3a and the light projecting device 3b each irradiate the imaging target O with laser light (step S501). Next, the image pickup device 2a shoots the shooting target O at the shooting point S1, transmits the shooting signal to the computer device 1, and the control device 11 of the computer device 1 generates the image I1 based on the shooting signal (step S502). .. The image pickup device 2b shoots the shooting target O at the shooting point S2, transmits the shooting signal to the computer device 1, and the control device 11 generates the image I2 based on the shooting signal (step S503).

Next, the control device 11 extracts the local plane LP1 from the image I1 (step S504). The local plane LP1 is extracted by identifying a particular region using techniques such as the Hough transform. In the following, the local plane LP1 is regarded as the frame i. In the present embodiment, the local plane LP1 corresponds to a rectangular region, but the shape of the region may have any shape.

Further, the control device 11 extracts the local plane LP2 corresponding to the local plane LP1 (frame i) based on the local plane LP1 (step S505). The local plane LP2 is extracted, for example, by identifying a region corresponding to the local plane LP1 using a projective transformation using a projective matrix. In the following, the local plane LP1 is regarded as the frame j. By the process of step S505, the pixels P2a and P2b in the frame j corresponding to the pixels P1a and P1b in the frame i are also specified.

Next, the control device 11 calculates the distance between the pixels for each pixel in each frame so as to obtain the displacement between the frame i and the frame j (step S506). Here, the luminance for the specific pixel P1a (x, y) in the frame i is I _i (x, y), and the luminance set for the pixel set (x _k , y _k ) is I _i (x _k , y). _k ). k represents the number of dimensions of the pixels in the frame. ^In this embodiment, k = 112.

Further, the luminance of the pixel P2a (x, y) corresponding to the pixel P1a in the frame j is I _j (x, y), and the luminance set of the pixel set (x _k , y _k ) is I _j (x _k ). , Y _k ). The distance between pixels in step S506 is calculated by the Euclidean distance between the feature vectors described above.

The luminance set I _i (x _k , y _k ) forms a feature vector F _i = I _i (x ₁ , y ₁ ), ..., I _i (x _k , y _k ) indicating the state of the frame i. Further, the luminance set I _j (x _k , y _k ) forms a feature vector F _j = I _j (x ₁ , y ₁ ), ..., I _j (x _k , y _k ) indicating the state of the frame j. ..

Next, the control device 11 reduces the number of dimensions k to the number of dimensions l (l <k) based on the 3D plane parameter ψ _i (x, y) of the frame i. The dimension vector Ψ _i is calculated (step S507).

The dimension reduction techniques described above include, for example, diffusion maps and Isomaps. In this embodiment, the diffusion map calculates the low-dimensional vector Ψ _i , which is a feature vector for the frame i with the reduced number of dimensions l, according to equation (4).

The input of dimensionality reduction is a set of feature vectors Fi (x, y) ( _i = 1, ..., N). N is the number of pixels in the frame i.

Further, the control device 11 is a low-dimensional vector for the frame j by a dimension reduction technique (diffusion map) that reduces the number of dimensions k to the number of dimensions l based on the 3D plane parameter ψ _j (x, y) of the frame j. Calculate Ψ _j (step S508).

The above-mentioned dimension reduction techniques also include diffusion maps, Isomaps, and the like. In the present embodiment, the diffusion map calculates the low-dimensional vector Ψ _j , which is a feature vector for the frame i with the reduced number of dimensions l, according to the equation (5). It is desirable that l, which is the number of dimensions reduced in steps S507 and S508, is, for example, any value of 3 to 20.

The input of dimensionality reduction is a set of feature vectors F _j (x, y) (j = 1, ..., N'). N'is the number of pixels in the frame j.

Calculating the distance according to equation (3) with the reduced number of dimensions l is calculated according to equation (6) instead of equation (3).

The processing in steps S507 and S508 is based on the displacement (Euclidean distance) between the feature vector for the frame i calculated in the dimension number k and the feature vector for the frame j, and the frame i is reduced in the number of dimensions l. The displacement between the feature vector of and the feature vector for frame j will be calculated.

By reducing the number of dimensions in steps S507 and S508, the distance is calculated in the reduced number of dimensions l as described later. For example, when the movement of the object does not move in the depth direction with respect to the image pickup device 2 and includes only the movement parallel to the image pickup device 2, if the displacement is calculated with a high number of dimensions, noise is generated in the calculation. This noise can be removed by calculating the distance (according to equation (6)) in the reduced number of dimensions l.

Further, by calculating the displacement in the reduced number of dimensions l, the displacement of the feature vector between each pixel in the frame i and the frame j becomes small, so that the linearity of the speckle pattern can be maintained. ..

Further, by calculating the distance with the reduced number of dimensions l, the amount of calculation when calculating the distance can be reduced. In particular, in the equation (3), the distance is calculated for each pixel, so calculating the distance without reducing the number of dimensions requires an enormous amount of calculation.

Next, the control device 11 applies a linear transformation to the low-dimensional vector Ψ _i for the frame i according to the equation (7) (step S509). Further, the control device 11 applies a linear transformation to the low-dimensional vector Ψ _i for the frame i according to the equation (8) (step S510).

The process of step S509 features a luminance set in frame i between each pixel of I _i (x _l , y _l ) (where l represents the number of dimensions to which the dimension reduction is applied in step S507). Approximate the displacement of the vector. Similarly, the process of step S510 approximates the displacement of the feature vector between the pixels of I _j (x _l , y _l ) for the luminance set in the frame j.

M (x, y) in the equations (7) and (8) is a transformation matrix of 3 × l, and Ψ _i (x, y) is an l × 1 column vector for the pixel P1a (x, y). Is.

In equations (7) and (8), equations (9) and (10) are used to approximate the displacement of the feature vector between a particular pixel and a pixel within a particular range of that pixel within the frame. ) Is imposed according to the constraint _ES . This constraint is, for example, as shown in FIG. 6, the distance (pixels) between the pixels P2a (x ₁ , y ₁ ) in the local plane LP2 (frame j) and the pixels P1a in the local plane LP1 (frame i). It is a constraint that approximates the distance (displacement) d1) between the pixels and the distance between the pixels P2b ₍ x2, y2) and the pixels P1b ₍ distance (displacement) d2 between the pixels). In this example, the distance d1 between the pixels and the distance d2 between the pixels correspond to the change (displacement) of the distance to the actual point corresponding to the same pixel when viewed from the image pickup apparatus 2, but represent the distance between the pixels. It may be another definition.

The constraint on the pair of the pixel P2a (x ₁ , y ₁ ) and the pixel P2b (x ₂ , y ₂ ) in the frame j is imposed according to the equation (11).

Is the product of element units.

In the equation (10), the column vector obtained by applying the linear matrix M (x, y) to the vectors of all the pixels is defined as m. Since the constraint is expressed by ^Am = 0 and A is a coefficient matrix calculated from equation (11), m is given as an eigenvector associated with the minimum eigenvalue of ATA. When the linear matrix M (x, y) is calculated, the low-dimensional vector ψ _i (x, y) and the low-dimensional vector ψ, which are calculated according to equations (7) and (8) and to which the linear transformation is applied, are applied. _j (x, y) is spatially consistent (ie, the displacement of the feature vector of the pixels in the frame, represented by the low dimensional vector, is within a certain range).

One of the problems in calculating the eigenvectors is, for example, that the temporal distributions of the low-dimensional vectors Ψ _i (x, y) for the pixels in the frame i are different. When Ψ _i (x, y) of the pixel (x, y) is almost zero, Ψ _i (x, y) affects the error even if the magnitude of M (x, y) is dominant in the eigenvector. Is not given, so the magnitude of M (x, y) is approximately equal to 1, and the parameters for the other pixels are approximately zero. Therefore, according to Eq. (12), the standard deviation of Ψ _i (x, y) is normalized before calculating the coefficient matrix A along the time axis.

Ψ _ik is the kth component of Ψ _i ,

Is the mean of Ψ _ik and ε is a small number to avoid division by zero.

For example, since the load of calculating the eigenvector from the image taken by the image pickup apparatus 2 having a high resolution is high, the transformation matrix M (x, y) is calculated after subsampling the pixels.

Returning to the description of FIG. 5, the control device 11 has a low-dimensional vector ψ _i (x, y) to which the linear transformation is applied in step S509 and a low-dimensional vector ψ _j (x, y) to which the linear transformation is applied in step S510. Based on y), the displacement between the frame i and the frame j is calculated for each pixel (step S511).

The calculation of the transformation matrix M (x, y) described above does not take into account the displacement calculated by Eq. (3). For the frame i, the transformation matrix M (x, y) is optimized so that the above-mentioned low-dimensional vector ψ _i (x, y) preserves the distance between the feature vectors _Fi (x, y).

By assuming that the distance (displacement) between the frame i and the frame j is very short, the distance indicated by the low-dimensional vector ψ _i (x, y) is the distance indicated by the feature vector _Fi (x, y). Conservative constraints are added to the error function that will be minimized according to equations (13) and (14).

E _s is a spatial constraint defined by Eq. (10), and λ is its weight. When the transformation matrix M (x, y) is optimized, the plane parameter ψ _i (x, y) is optimized, and the displacement per pixel is given as the third element of ψ _i (x, y). ..

The initial guess value ψ _i'for nonlinear minimization of equation (13 ⁾ is calculated above after normalizing the size of each pixel according to equations (15), (16), and (17). It is given by ψ _i (x, y).

Since the calculated transformation matrix M (x, y) is subsampled, the transformation matrix M (x, y) is interpolated before optimizing the parameters to obtain the displacement for the original resolution image. To. The interpolation is done by interpolating the displacement of the subsampled feature vector calculated by using the initial guess value ψ _i ^' . The pixels P1a (x, y) of the original resolution and the transformation matrix M (x, y) are the pixels P1b.

に対して式（１８）を満たす必要がある。

It is necessary to satisfy the equation (18).

Since the linear form of the transformation matrix M (x, y) is acquired for all the pixels in the local patch, the transformation matrix M (x, y) is acquired as a solution by the least squares method. Once the transformation matrix M (x, y) has been acquired for all pixels, the optimization is applied to the image at the original resolution.

By the process shown in FIG. 5, the displacement between the frame i and the frame j can be calculated. In the example described with reference to FIG. 5, only the displacement between the two frames is calculated, but when determining the continuous movement of the object O, the processes of steps S501 to S511 are repeated. That is, a moving object is photographed any number of times and the displacement between the corresponding number of images in the local plane is calculated.

As described above, even if the movement of the object O is slight, the speckle pattern may change significantly depending on the distance between the object O and the image pickup apparatus 2. According to this embodiment, since the displacement of the feature vector is approximated to each pixel in the local plane, the change of the feature vector in the local plane becomes small. Then, since the displacement of the feature vector is approximated to the subsequent pixels in the local plane, the change in the feature vector becomes small even between the local planes. This leads to maintaining the correlation before and after the change in the speckle pattern.

Also, since the number of dimensions is reduced when calculating the displacement (preferably any number from k to l, l is preferably any number from 3 to 20), the noise generated by calculating the displacement with a high number of dimensions should be reduced. And the amount of calculation can be reduced.

By the process according to the present embodiment, for example, the movement of an object can be visually expressed according to the change of the changing speckle pattern. An example thereof will be described with reference to FIGS. 7 and 8. FIG. 7 shows a state in which the surface of the canvas cloth is pressed with a finger. When you press the surface of the cloth with your finger, the vibration is transmitted from the pressed part by the force of your finger.

In such a state, the surface of the cloth is photographed multiple times at an arbitrary part (same or different part) of the cloth transmitting vibration, and the change in the speckle pattern is analyzed according to the process described with reference to FIG. For example, as shown in FIG. 8, a change in vibration can be expressed. The horizontal axis of the graph shown in FIG. 8 indicates the x-axis (that is, the distance traveled by the vibration wave in the lateral direction), and the vertical axis indicates the displacement (that is, the displacement between the local planes of the two images). show.

The graph shown in FIG. 8 is displayed on the display device 4 based on the calculated displacement under the control of the control device 11. The displacement corresponds to the amplitude of the vibration and the x-axis is derived by adding the displacements. In this way, by analyzing the speckle pattern in the image generated from the moving object, it is possible to visually express the slight change in the movement of the object.

As described above, the image analysis system 100 according to the present embodiment has been described. According to this embodiment, it is possible to accurately determine a slight movement of an object. Further, it is possible to express a moving object O (object) to be photographed according to a change in the speckle pattern. For example, the movement of the corresponding image shooting target O may be dynamically displayed on the display device 4 according to the change of the determined speckle pattern.

It should be noted that the hardware components described in the above embodiments are only exemplary and other configurations are possible. Further, the order of the processes described in the above embodiment does not necessarily have to be executed in the order described, and may be executed in any order. In addition, additional steps may be added without departing from the basic concepts of the invention.

Further, although the image determination according to the embodiment of the present invention is implemented by a computer program executed by the computer device 1, the computer program may be stored in a non-temporary storage medium. Examples of non-temporary storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disk devices, optomagnetic media, and Includes optical media such as CD-ROM discs and digital versatile discs (DVDs).

Claims (10)

A computer device that determines the movement of the object to be photographed.
A first image including the first speckle pattern is generated based on the first shooting signal generated by the image pickup device shooting the shooting target at the first time.
A second image including the second speckle pattern is generated based on the second shooting signal generated by the imaging device shooting the shooting target at the second time.
The first feature vector is calculated from the luminance in the first pixel set in the first image, and the displacement of the first feature vector is approximated between the first pixel sets.
The second feature vector is calculated from the luminance in the second pixel set in the second image, and the displacement of the second feature vector is approximated between the second pixel sets.
The first speckle pattern and the second spec by calculating the distance between pixels for each pixel between the approximated first feature vector and the approximated second feature vector. Judging the change between the vector and the pattern,
A computer device characterized by being equipped with a control device configured in such a manner. The control device is
The first feature vector and the second feature vector are calculated by the first dimension number, and the first feature vector and the second feature vector are calculated.
Based on the first feature vector, the first feature vector is calculated with a second dimension number smaller than the first dimension number.
Based on the second feature vector, the second feature vector is calculated with the second dimension number.
The computer device according to claim 1, wherein the computer device is further configured as described above. The computer device according to claim 2, wherein the control device is further configured to calculate the first feature vector and the second feature vector by a diffusion map. The computer device according to claim 2 or 3, wherein the second dimension number is any one of 3 to 20. The control device is
A first local plane is extracted from the first image by determining a specific region.
It is further configured to extract the second local plane from the second image by determining the region corresponding to the first local plane.
The first pixel set is a pixel set in the first local plane.
The second pixel set is a pixel set in the second local plane.
The computer device according to any one of claims 1 to 3. The control device is
By applying a linear transformation to the first feature vector, the displacement of the first feature vector is approximated between the first pixel sets.
By applying a linear transformation to the second feature vector, the displacement of the second feature vector is approximated between the second pixel sets.
The computer device according to claim 1, wherein the computer device is further configured as described above. The control device has the first feature vector and the second feature vector under the constraint that the distance between the pixels of each pixel between the first pixel set and the second pixel set is close to each other. 6. The computer device according to claim 6, further configured to apply the linear transformation to. Claim 1 is characterized in that the control device displays the movement of the photographing target on a display device based on the change between the first speckle pattern and the second speckle pattern. The computer device according to any one of 7 to 7. A method performed by a computer device that determines the movement of the subject to be photographed.
A step of generating a first image including a first speckle pattern based on a first shooting signal generated by the image pickup device shooting the shooting target at the first time.
A step of generating a second image including a second speckle pattern based on a second shooting signal generated by the imaging device taking a picture of the imaged object at a second time.
A step of calculating a first feature vector from the luminance in the first pixel set in the first image and approximating the displacement of the first feature vector between the first pixel sets.
A step of calculating a second feature vector from the luminance in the second pixel set in the second image and approximating the displacement of the second feature vector between the second pixel sets.
The first speckle pattern and the second spec by calculating the distance between pixels for each pixel between the approximated first feature vector and the approximated second feature vector. Steps to determine the change between the vector and
A method characterized by being equipped with. A computer program comprising computer-executable instructions, wherein the computer-executable instruction causes the computer device to perform the method according to claim 9, when executed by the computer device.

Images