JP5410232B2 - Image restoration device, program thereof, and multidimensional image restoration device - Google Patents

Description

  The present invention relates to a technique for restoring a restored image from non-uniformly sampled images sampled at unequal intervals, using a complexity indicating the complexity of each partial region obtained by dividing an input image into a plurality of parts.

  Conventionally, techniques based on interpolation processing such as nearest neighbor method (nearest neighbor method), bilinear method (bilinear method), and bicubic method (bicubic method) are known as techniques for converting the resolution of an image. .

  Further, as a technique for converting a low-resolution image with a low resolution into a high-resolution image with a high resolution, the movement amount of each pixel or block is estimated from the input image (low-resolution image) in the time direction, There is a super-resolution technique for generating a high-resolution image based on a point-in-time image group (see, for example, Patent Documents 1 and 2).

  In addition, there is a super-resolution technique that generates a high-resolution image with sharper image quality than linear interpolation by generating an interpolation value while adaptively determining spatial image features such as edges (for example, non-patented). Reference 1). Further, there is a seam carving method that performs spatially unequal image deformation so as not to impair the resolution of a visually important region (see, for example, Non-Patent Document 2).

Japanese Unexamined Patent Publication No. 2006-127241 (Patent No. 38337575) JP 2008-109375 A

Nobuyuki Matsumoto, Takashi Ida, “Examination of image quality improvement by region adaptive processing of intra-frame reconstruction super-resolution,” IEICE Technical Report, Vol. 108, no. 4, pp. 31-36 (2008). Shai Avidan, Ariel Shamir, “Seam Carving for Content-Aware Image Resizing,” ACM Trans.Graph. 26, 3, 10.

However, the above-described prior art has various problems as described below.
Using techniques based on interpolation processing such as nearest neighbor, bilinear method, bicubic method, etc., the input high resolution image is converted into a low resolution image, and this low resolution image is further reconverted into a high resolution image. Consider the case of output. In this case, the high-frequency component of the image information included in the input high-resolution image, that is, information on the fine pattern is lost, and the re-converted high-resolution image is blurred.

  In the super-resolution techniques described in Patent Documents 1 and 2 and Non-Patent Document 1, the amount of movement of each pixel or block is estimated from the input image in the time direction, and an interpolation value is obtained by a complicated nonlinear calculation according to the image feature. Need to be generated, and the amount of calculation becomes large.

  In addition, the seam carving method described in Non-Patent Document 2 performs a non-linear mapping while deforming the image size while maintaining the resolution and shape of a visually important part. A method for describing the degree or the like is not given, and restoration of the original image is not considered. That is, in the seam carving method described in Non-Patent Document 2, a reverse mapping is required to restore a deformed non-linear mapping. Information on this non-linear reverse mapping is obtained from all samples in the original image. Since this is point information, there is a problem that the amount of information increases.

  Therefore, an object of the present invention is to provide a technique for restoring a clear restored image from non-uniformly sampled images by using a deformation parameter with a small amount of information by a simple calculation.

In order to solve the above-described problem, an image restoration apparatus according to the first invention of the present application uses a complexity indicating a complexity for each partial region obtained by dividing an input image, and a space set in advance using the complexity. An unequal interval sampled image obtained by sampling the input image at unequal intervals on the sample axis in the direction or time direction is input from the unequal interval sampling device, and the unequal interval is obtained using the complexity. An image restoration apparatus for restoring a restored image from a sampled image, comprising: a correspondence calculation unit; and an image restoration unit, wherein the unequal interval sampled image to which the complexity is added as a digital watermark is input Rutotomoni, further comprising characterized Rukoto digital watermark detection means for detecting the complexity from the non-equidistant sampling images.

  According to such a configuration, the image restoration device uses the correspondence calculation unit to calculate a predetermined sample point from the complexity for each scanning line passing through all the partial regions continuous in the sample axis direction. The number of sample points in the restored image and the unequal interval sampling so that there are a large number of sample points in the partial area having a high complexity and a small number of sample points in the partial area having a low complexity. The correspondence with the coordinates of the sample points in the image is calculated. This complexity is a deformation parameter required when restoring non-uniformly sampled images, and is information calculated for each partial region. That is, the amount of information of the complexity is smaller than that of a conventional deformation parameter that is information of all sample points.

In addition, the image restoration device uses the image restoration unit to calculate pixel values of pixels corresponding to the coordinates of the sample points in the non-uniformly sampled image based on the correspondence calculated by the correspondence calculation unit. The restored image is restored as the pixel value of the pixel corresponding to the coordinates of the sample point at. Here, the non-uniformly sampled image is an input image in which a complicated part is finely sampled, and a simple part in the input image is roughly sampled. In other words, the image restoration apparatus can restore a restored image that is expressed finely with the same amount of information as compared to when images sampled at equal intervals are restored. Further, since the image restoration apparatus performs a one-dimensional restoration process, the calculation is simple. Furthermore, the image restoration apparatus can hide the presence of complexity.

Note that sampling with unequal intervals means sampling with different intervals between sample points, and the intervals between all sample points do not necessarily have to be different.
Also, sampling at equal intervals refers to sampling so that the intervals of all sample points are equal.

In order to solve the above-described problem, the image restoration device according to the second invention of the present application is an unequal interval sampled image obtained by sampling the input image from two or more sample axes from the unequal interval sampling device. And the complexity calculated for each sample axis, and the correspondence calculation means calculates the correspondence for each sample axis from two or more complexity calculated for each sample axis. A correspondence relationship calculation unit; and a correspondence relationship synthesis unit that synthesizes the correspondence relationships calculated by the two or more correspondence relationship calculation units. The image restoration unit is further configured based on the correspondence relationship synthesized by the correspondence relationship synthesis unit. The restored image is restored from the non-uniformly sampled images sampled at the two or more sample axes.

  According to such a configuration, the image restoration apparatus can restore a restored image by performing the restoration process only once even for non-uniformly sampled images sampled at two or more sample axes, and thus the calculation is simple. .

In order to solve the above-described problem, an image restoration program according to the third invention of the present application causes a computer to function as the image restoration device.

In order to solve the above-described problem, the multidimensional image restoration apparatus according to the fourth invention of the present application uses a complexity indicating the complexity of each partial region obtained by dividing the input image, and the complexity to be used in advance. An unequal interval sampled image obtained by sampling the input image at unequal intervals in the set spatial direction or time direction sample axis is input from the unequal interval sampling device, and using the complexity, An image restoration apparatus that restores a restored image from the non-uniformly sampled images, and is set in advance from the complexity for each scanning line that passes through all the partial regions that are continuous in the direction of the sample axis. The coordinates of the sample points in the restored image and the inequality so that there are many sample points in the partial area with the high complexity as many as the number of sample points and the sample points in the partial area with the low complexity. In interval sampled images A pixel value of a pixel corresponding to the coordinates of the sample points in the non-uniformly sampled image based on the correspondence relationship calculated by the correspondence relationship calculating unit; Is a multi-dimensional image restoration device in which k image restoration devices comprising image restoration means for restoring the restored image as pixel values of pixels corresponding to the coordinates of the sample points in the restored image are connected in series. The first image restoration device receives a k-dimensional image as the non-uniformly sampled image, and restores the k-dimensional image with a first sample axis in a preset spatial direction or time direction, The kth image restoration device is characterized in that the restored image restored by the k-1 unequal interval sampling device is inputted and the restored image is restored in the direction of the kth sample axis. that (however, k is 2 or more Integer).

  According to such a configuration, the multidimensional image restoration apparatus uses a complexity with a small amount of information compared to a conventional deformation parameter that is information of a sample point. In addition, the multidimensional image restoration apparatus can restore a restored image that is finely expressed with the same amount of information as compared with a case where an image sampled at equal intervals is restored. Furthermore, since the multidimensional image restoration apparatus only repeats the one-dimensional restoration process k times, the calculation is simple.

K described above indicates the number of dimensions of sampling, and is an integer of 2 or more.
Here, when k = 2, examples of the input image (two-dimensional image) include a still image. When k = 3, examples of the input image (three-dimensional image) include a moving image or a stereoscopic still image. Further, when k = 4, as an input image (four-dimensional image), for example, there is a stereoscopic moving image.

According to the present invention, the following excellent effects can be obtained.
According to the fourth invention of the present application, it is possible to restore a clear restored image from a non-uniformly sampled image by using a deformation parameter with a small amount of information with a simple calculation.

According to the first invention of the present application, the presence of complexity can be concealed, which is particularly preferable when the image restoration apparatus is used as a descrambling apparatus.
According to the second invention of the present application, the restored image can be synthesized in one time, and the amount of calculation can be reduced.

It is a figure explaining the outline of the image deformation restoration system in the present invention. It is a figure explaining sampling at equal intervals, (a) is a conceptual diagram of sampling at equal intervals, (b) is a figure showing an image sampled at equal intervals. It is a figure explaining the deformation | transformation image in this invention, (a) is a conceptual diagram of the sampling of an unequal interval, (b) is a figure which shows a deformation | transformation image. It is a block diagram which shows the structure of the image restoration apparatus of FIG. It is a figure explaining the complexity and sample point in this invention, (a) is a figure explaining complexity, (b) is a figure explaining calculation of the coordinate of a sample point, (c) is a figure. It is a figure explaining the interpolation of the coordinate of a sample point. 6A and 6B are diagrams for explaining the calculation of the coordinates of sample points in the uppermost partial area of FIG. 5A, FIG. 5A is a diagram showing the relationship between the complexity and the position of the partial area, and FIG. It is a figure which shows the relationship between a pixel area position. 6A and 6B are diagrams for explaining the calculation of the coordinates of sample points in the uppermost partial region of FIG. 5A, FIG. 5A is a diagram showing the relationship between the cumulative complexity and the pixel position, and FIG. It is a figure explaining the coordinate of the sample point calculated from the degree. 6A and 6B are diagrams illustrating calculation of the coordinates of sample points in the lowermost partial area of FIG. 5A, FIG. 5A is a diagram illustrating the relationship between the complexity and the position of the partial area, and FIG. It is a figure which shows the relationship between a pixel area position. 6A and 6B are diagrams illustrating calculation of the coordinates of sample points in the lowermost partial region of FIG. 5A, FIG. 5A is a diagram illustrating the relationship between cumulative complexity and pixel position, and FIG. It is a figure explaining the coordinate of the sample point calculated from the degree. It is a figure which shows the outline of the decompression | restoration process by the image restoration apparatus of FIG. It is a figure explaining the 1st example of the pixel value interpolation means of FIG. It is a figure explaining the 2nd example of the pixel value interpolation means of FIG. 6 is a flowchart for explaining the operation of the image restoration apparatus in FIG. 4. It is a block diagram which shows the modification 1 of the image restoration apparatus of FIG. It is a block diagram which shows the modification 2 of the image restoration apparatus of FIG. It is a block diagram which shows the modification 3 of the image restoration apparatus of FIG. It is a block diagram which shows the structure of the image restoration apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structure of the image restoration apparatus which concerns on 3rd Embodiment of this invention. In this invention, it is a figure which shows the partial area | region of a three-dimensional image. FIG. 20 is an enlarged view of a central partial region in the three-dimensional image of FIG. 19. FIG. 21 is a top view of FIG. 20. FIG. 21 is a right side view of FIG. 20. It is a figure explaining the interpolation process by the image deformation apparatus of FIG. It is a block diagram which shows the structure of the image restoration apparatus which concerns on 4th Embodiment of this invention.

[Outline of image deformation restoration system]
The outline of the image deformation restoration system 1 according to the present invention will be described below with reference to FIG. As shown in FIG. 1, the image transformation / restoration system 1 includes an image transformation device 2 and an image restoration device 3. In addition, the image deformation / restoration system 1 is connected to an image deformation device 2 and an image reconstruction device 3 via a network or a broadcast wave (not shown).

  The image deformation device 2 calculates a deformation parameter (complexity) of the input image, and samples the input image at unequal intervals in the direction of the sample axis by using the deformation parameter. A sampled image). Details of the image deformation device 2 are omitted.

  The image restoration device 3 receives the transformation image and the transformation parameters from the image transformation device 2 and restores the restoration image from the transformation image using the transformation parameters. Specifically, the image restoration device 3 calculates a correspondence relationship between the coordinates of the sample points of the input image and the deformation image from the deformation parameter, and based on this correspondence relationship, the image restoration device 3 generates a restoration image from the deformation image in the direction of the sample axis. Restore. Here, the sample axis of the image deformation device 2 and the sample axis of the image restoration device 3 are preferably in the same direction. Details of the image restoration device 3 will be described later.

The input image is an arbitrary image, for example, a two-dimensional image, a three-dimensional image, or a four-dimensional image. Here, for example, the input image will be described as a two-dimensional image.
The deformed image is an image generated from the input image by sampling at unequal intervals, and has the same number of dimensions as the input image. The deformed image corresponds to the non-uniformly sampled image described in the claims.

The deformation parameter is a parameter required when restoring the restored image from the deformed image, and has a complexity indicating the complexity of the pattern (shading change) for each partial area in the input image.
The restored image is an image restored from the deformed image using the deformation parameter.

<Specific examples of deformed images>
Hereinafter, a specific example of the deformed image will be described with reference to FIGS. 2 and 3 in comparison with a conventional image sampled at equal intervals. Further, in FIGS. 2A and 3A, the sampling interval (that is, the interval between sample points) is indicated by the size of the lattice. That is, the smaller the lattice, the smaller the sampling, and the larger the lattice, the coarser the sampling. Note that this lattice is illustrated for explanation, and this lattice is not added to the restored image.

  Also, as shown in FIGS. 2 and 3, the image of this specific example is that the upper left part is the background (for example, sky and clouds), and the other part is the subject (for example, the city scenery including the buildings). is there. Therefore, this image has a simple pattern in the upper left part and a complicated pattern in other parts.

  In the case of sampling at equal intervals, as shown in FIG. 2 (a), since each grid is the same size, regardless of the complexity of the image, even if the pattern is complex or the pattern is simple, it is the same. Sampling is performed at intervals of. In addition, in the image sampled at equal intervals, the subject and background included in the image are depicted without deformation as shown in FIG.

  On the other hand, in the non-uniform sampling according to the present invention, as shown in FIG. 3 (a), fine sampling is performed at a portion where the pattern is complex (small lattice), and rough sampling is performed at a portion where the pattern is simple. Do (large grid). The restored image obtained by restoring the deformed image can be expressed finely with the same amount of information as compared with the case where the image sampled at equal intervals is restored. Note that this deformed image is deformed so that the subject fills the background, for example, as shown in FIG.

(First embodiment)
[Configuration of image restoration device]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted. As shown in FIG. 4, the image restoration device 3 includes a mapping restoration unit (corresponding relationship calculation unit) 31 and an image restoration unit 33.

Hereinafter, the input image is I ₀ ([x ₀ , y ₀ ] ^T ) or simply I ₀ . Incidentally, x ₀ is the horizontal coordinate, y ₀ is the vertical coordinate, both the pixel unit. The width of the input image X ₀ pixels, a height of Y ₀ pixel. Here, the superscript T indicates transposition.
Also, the deformed image is J ([u, v] ^T ) or simply J. Note that u is a horizontal coordinate and v is a vertical coordinate, both of which are in pixel units. The width of the deformed image is U pixels, and the height is V pixels.
Also, the restored image is I ([x, y] ^T ) or simply I. Note that x is a horizontal coordinate, and y is a vertical coordinate, both in pixel units. The width of the restored image is X pixels and the height is Y pixels.

In the image restoration apparatus 3, for example, restore image I, the input image I _0, is set in advance to approximate except for scaling. Here, the preset scaling in the horizontal and vertical directions is defined as a positive constant α and a positive constant β. That is, α = X / X ₀ and β = Y / Y ₀ . At this time, there is a correspondence relationship between the restored image I and the input image I ₀ with horizontal coordinates x = horizontal coordinates αx ₀ and vertical coordinates y = vertical coordinates βy ₀ . Incidentally, for example, the same aspect ratio of each pixel of the restored image I and the input image I _0, and, when the aspect ratio of the whole of restored image I and the input image I ₀ is the same, positive constant alpha = positive constant β > 0. Further, for example, when the restored image I and the input image I ₀ have the same horizontal resolution and vertical resolution, the positive constant α = the positive constant β = 1. Further, when the restored image I is enlarged with respect to the input image I ₀ , the positive constant α> 1 and the positive constant β> 1 are satisfied, and when the restored image I is reduced with respect to the input image I _0, 0 <positive Constant α <1 and 0 <positive constant β <1.

Further, the input image I ₀ is divided (equally divided) into a total of M × N partial areas by M in the horizontal direction and N in the vertical direction by the image transformation device 2 in FIG. The description will be made assuming that the complexity is calculated. Here, the mth (m = 0, 1,..., M−1) partial region in the horizontal direction and the nth (n = 0, 1,..., N−1) partial region in the vertical direction are represented by B. _{Let m, n} .

  The mapping restoration unit 31 includes a sample point interpolation unit 31a, and a correspondence relationship between the coordinates of the sample point in the restored image I and the coordinates of the sample point in the transformed image J based on the input deformation parameter (complexity). Is calculated. The mapping restoration unit 31 corresponds to the correspondence calculation unit described in the claims.

Here, for example, equation (1) for associating the coordinate [u, v] ^T of the sample point in the deformed image J with the coordinate [x, y] ^T of the sample point in the restored image I can be defined.

Further, for example, equation (2) for associating the coordinate [x, y] ^T of the sample point in the restored image I with the coordinate [u, v] ^T of the sample point in the deformed image J can be defined.

Hereinafter, a first example in which the scanning line is in the horizontal direction and a second example in which the scanning line is in the vertical direction in the case where Expression (2) is used will be described. Here, as shown in FIG. 5 (a), the input image I _0, is divided into four in the horizontal direction (M = 4), described as being vertically divided into three (N = 3).

The scanning line S _n vertical positions in n is to bypass part region B _m, the representative point of _n in the direction of the sample axis. At this time, representative points are set in advance in each of the partial areas _{Bm, n} . The representative point may be, for example, as a central point of the partial regions B _{m, n,} the partial areas B _{m, n} vertices (e.g., upper right, upper left, lower right, lower left) may be arbitrary point, such as .

<Scanning line is in horizontal direction: first example>
First, as a premise, a method for calculating the coordinates of the sample point based on the complexity C _{m, n} will be described. In this case, the image transformation apparatus 2 of Figure 1, partial region B _m continuous in the horizontal _direction, the _n, to generate a scan line S _n passing through the representative points in the horizontal direction (direction of the sample axis). Here, the length of the scan line S _n is the same as the width X of the restored image I. The image transformation apparatus 2 in FIG. 1 calculates u th on the scanning line _{S n (u = 0, ···} , U-1) coordinates _x n sampling points of the (u) as follows ( However, 0 ≦ x _n (u) ≦ X−1). Incidentally, U is the number of a preset sampling points on one scan line S _n.

First, based on the complexity C _{m, n} , a function c _n (x) of the complexity is defined as the following equation (3).

Formula (3) is demonstrated with reference to FIG. FIG. 6A shows an example of the complexity C _{m, n} when m is changed while fixing _n, and corresponds to the uppermost scanning line Sn in FIG. 5B. Here, on the right side of the equation (3), the value obtained by rounding down the decimal point of the term “M · x / X” in the following equation (4) is the first index (m direction) of the complexity C _{m, n} , The function of FIG. 6B is obtained. That is, the graph of FIG. 6B is obtained by scaling the graph of FIG. 6A by X / M times in the horizontal axis direction and then applying the zero-order hold. Therefore, the “partial region position m” on the horizontal axis in FIG. 6A changes to the “pixel position x” on the horizontal axis in FIG. 6B.

Next, as shown in FIG. 7A, the complexity function c _n (x) is definitely integrated in the x direction from x = 0, and the cumulative complexity function a expressed by the following equation (5) is used. _{Let n} (x).

Further, as shown in FIG. 7B, for u = 0, 1,..., U−1, x satisfying a _n (x) = u is defined as ξ _n (u). Finally, as shown in the following equation (6), the decimal point of ξ _n (u) is rounded down to obtain the coordinate x _n (u) of the sample point.

Thus, an image transformation apparatus 2 in FIG. 1, by using the equation (6) from equation (3), for each scanning line S _n, the number U of sample points, complexity C _{m, n} higher partial areas B _m, many sample points in _n, and the complexity of _{C m,} so that _n is lower partial region _{B m,} the sample points in the _n decreases, and calculates the coordinates _x n sample point (u). In other words, the image transformation apparatus 2 in FIG. 1, for all the scan lines S _n, will be allocated the same number of sample points.

Here, an example of calculating the coordinates x _n (u) of the sample points for each of the uppermost partial areas B _{0, n in} FIG. At this time, the number U of sample points is set to 12. In this case, as shown in FIG. 6A, the value of the function c _n (x) of the complexity is 6, 2, 3, 1. Further, when a zero-order hold is applied to the complexity function c _n (x), the result is as shown in FIG.

Then, when the complexity function c _n (x) is definitely integrated by the equation (6), the cumulative complexity function a _n (x) of FIG. 7A is obtained. In FIG. 7A, the slope of the cumulative complexity function a _n (x) indicates the complexity C _{m, n} . Further, as shown in FIG. 7B, a perpendicular is extended from the intersection of the cumulative complexity function a _n (x) and the horizontal line of u (0,..., 11). Furthermore, x where the perpendicular intersects the horizontal axis is ξ _n (u), and after the decimal point of ξ _n (u) is rounded down, it is set as the coordinate x _n (u) of the sample point.

Similarly, a calculation example of the coordinate x _n (u) of the sample point for each of the lowermost partial regions B _{2, n in} FIG. As shown in FIG. 8A, the value of the complexity function c _n (x) is 1, 1, 3, 1. Further, when the zero-order hold is applied to the complexity function c _n (x), the result is as shown in FIG.

Then, when the complexity function c _n (x) is definitely integrated by the equation (6), the cumulative complexity function a _n (x) of FIG. 9A is obtained. Furthermore, as shown in FIG. 9B, a perpendicular is extended from the intersection of the cumulative complexity function a _n (x) and the horizontal line of u (0,..., 11). Furthermore, x where the perpendicular intersects the horizontal axis is ξ _n (u), and after the decimal point of ξ _n (u) is rounded down, it is set as the coordinate x _n (u) of the sample point.

For example, in each of the uppermost partial areas B _{0, n} in FIG. 5 (a), the complexity C _{0, n} is 6, 2, 3, 1 for a total of 12. On the other hand, in each of the partial regions B _{2, n} at the lowermost stage in FIG. 5A, the complexity C _{2, n} is _{1, 1} , 3 _{, 1} , and the total is 6. That is, in this example, it can be said that the upper side of the input image is a complex pattern and the lower side is a simple pattern. In this case, simply increasing the number of sample points at a complicated pattern results in a deformed image having a distorted shape such as an inverted trapezoid in which the number of upper pixels is larger than the number of lower pixels. Such a distorted deformed image often causes inconvenience during image processing. However, the image transformation apparatus 2 of Figure 1, from using Equation (5), complexity C _m, to whether the total value of _n, can equal the number of sample points of each scan line S _n (in FIG. 5 In the example, 12 images can be generated, and a deformed image having a well-shaped shape such as a square shape or a rectangular shape that is convenient for image processing can be generated.

In summary, the mapping restoring means 31, using the relationship expressed by the following equation (7), for each scanning line S _n, the number U of sample points, complexity C _{m, n} higher portion region B _m, many sample points in _n, and the complexity of C _{m, n} is the lower part region B _m, so that sample points in _n decreases, and calculates the coordinates of the sample points in the modified image J (FIG. 10 (See (a)).

Then, mapping restoring means 31, the sample point interpolator 31a, the coordinates of the sample points calculated in Equation (7), to interpolate between the scan line S _n by interpolation processing such as zero-order interpolation or linear interpolation ( (Refer FIG.10 (b)). Here, when the coordinates of the sample point include a decimal point, the sample point interpolation unit 31a may round the decimal point, round up, or round off to make the coordinate of the sample point an integer.

For example, in FIG. 5C, the coordinates of the calculated sample points are illustrated by black circles, and the coordinates of the interpolated sample points are illustrated by white circles. Here, the white circle top, from the top in the second and lowermost on the scanning line S _n, is obtained by interpolating the zero-order extrapolation. Also, the white circles located on the other scanning line S _n, is obtained by interpolation by linear interpolation. In this example, rounding processing is performed on the sample point at the decimal pixel position. Thus, since the coordinates of the sample point are interpolated, the image restoration device 3 can restore a clearer restored image I from the deformed image J. Thereafter, the function g of the correspondence relationship and the coordinates of the sample point in the deformed image J are output to the image restoration unit 33.

<Scanning line is vertical direction: second example>
In this case, the image transformation apparatus 2 of Figure 1, continuous partial area B _m in the vertical _direction, for _n, to generate a scan line T _m passing through the representative point in the vertical direction (direction of the sample axis). Here, the length of the scan line T _m is the same as the height Y of the input image I (x, y). 1 calculates the coordinates y _m (v) of the v-th (v = 0,..., V−1) sample point on the scanning line T _m as follows ( However, 0 ≦ y _m (v) ≦ Y−1). Incidentally, V is the number of a preset sampling points on one scan line T _m.

First, the complexity function d _m (y) is defined as in the following equation (8) based on the complexity C _{m, n} .

Here, on the right side of Expression (8), a value obtained by rounding down the decimal point of the term “N · y / Y” in Expression (9) below is the second index (in the n direction) of complexity C _{m, n} . .

Next, the complexity function d _m (y) is definitely integrated in the y direction from y = 0 to obtain a cumulative complexity function b _m (y) represented by the following equation (10).

Further, for v = 0, 1,..., V−1, y satisfying b _m (y) = v is defined as η _m (v). Finally, as shown in the following formula (11), the fractional part of η _m (v) is rounded down to obtain the coordinate y _m (v) of the sample point.

Therefore, the image deformation device 2 in FIG. 1 uses the equations (8) to (11), so that for each scanning line T _m , the partial region B having a high complexity C _{m, n} by the number V of sample points. _m, many sample points in _n, and the complexity of _{C m, n} is the lower part region _{B m,} so that sample points in _n decreases, and calculates the coordinates _y m of sample points (v). In other words, the image transformation apparatus 2 in FIG. 1, for all the scanning lines T _m, will be allocated the same number of sample points.

In summary, the mapping restoration means 31 uses a correspondence relationship represented by the following equation (12), and has a portion having a high complexity C _{m, n} by the number V of sample points for each scanning line T _m. region B _m, many sample points in _n, and calculates the complexity C _{m, n} is the lower part region B _m, so that sample points in _n decreases, the coordinates of the sample points in the modified image J.

Then, mapping restoring means 31, the sample point interpolator 31a, the coordinates of the sample points calculated in equation (12), interpolated between scanning lines T _m by interpolation processing such as zero-order interpolation or linear interpolation. Here, when the coordinates of the sample point include a decimal point, the mapping restoration unit 31 may round the decimal point, round it up, or round off to make the sample point coordinate an integer. Thereafter, the mapping restoration unit 31 outputs the function g of the correspondence relationship and the coordinates of the sample point in the deformed image J to the image restoration unit 33.

Returning to FIG. 4, the description of the configuration of the image restoration apparatus 3 will be continued.
The image restoration means 33 is provided with a pixel value interpolation means 33a, and a deformed image is input from the image deformation device 2 in FIG. 1, and a correspondence function g and the coordinates of the sample point in the deformed image J are input from the mapping restoration means 31. The

First, the image restoration unit 33 assigns a value (for example, a null value) indicating “undefined” to all pixels of the restored image I. Further, the image restoration means 33 converts the coordinates [u, v] of the sample points in the input deformed image J into the coordinates [x, y] of the sample points in the restored image I based on the correspondence function g. To do. Then, the image restoration means 33 uses the pixel value J ([u, v]) of the pixel corresponding to the coordinates of the sample point in the deformed image J as shown in the following equation (13), as the sample point in the restored image I. The pixel value I (g ([x, y])) of the pixel corresponding to the coordinates of.

  That is, the image restoration unit 33 generates a restored image I to which pixel values are assigned and outputs this (see FIG. 10C). Here, the pixel value interpolation means 33a performs an interpolation process on a pixel to which no pixel value is assigned in the restored image I, and assigns a pixel value to that pixel (see FIG. 10D). For example, the pixel value interpolation unit 33a interpolates the pixel value by interpolation processing using the pixel whose pixel value is calculated for the pixel whose pixel value is “undefined” in the restored image I. In this case, the pixel value interpolation means 33a performs interpolation processing such as primary interpolation, nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation.

<Interpolation Process Using Deformed Image as Reference: First Example>
Hereinafter, a first example and a second example of interpolation processing by the pixel value interpolation unit 33a will be described with reference to FIGS. 11 and 12 (see FIG. 4 as appropriate). In FIG. 11, pixels in the restored image I whose pixel values are to be interpolated are indicated by black circles, and pixels of the modified image J are indicated by white circles. As shown in FIG. 11, the pixel value interpolation means 33a obtains a pixel value by bilinear interpolation from a total of four pixels located above, below, left and right of the pixel (target point) for which the pixel value is to be interpolated. In this case, the pixel value interpolation means 33a may interpolate the pixel value using the following equation (14).

In Expression (14), I _{00 to} I ₁₁ indicate the pixel values of the upper, lower, left, and right pixels in the restored image, and I indicates the pixel value of the pixel in the modified image. In Expression (14), p is the length from the pixel as the attention point to the upper pixels I ₀₀ and I ₀₁ , and q is the length from the pixel as the attention point to the lower pixels I ₁₀ and I ₁₁ It is. Furthermore, in Expression (14), r is the length from the intersection point of the horizontal line passing through the attention point and the line segment of the pixels I ₀₀ and I ₁₀ on the left side of the attention point to the attention point, and s is the horizontal line passing through the attention point. This is the length from the intersection with the line segment of the pixels I ₀₁ and I ₁₁ on the right side of the point of interest to the point of interest.

<Interpolation process based on restored image: second example>
As shown in FIG. 12, the pixel value interpolating unit 33a obtains a pixel value by bilinear interpolation from a total of four pixels located on the top, bottom, left and right of the pixel (point of interest) whose pixel value is to be interpolated. In this case, the pixel value interpolation means 33a may interpolate the pixel value using the following equation (15). In FIG. 12, in the restored image I, the pixel whose pixel value is to be interpolated is illustrated by a white circle, and the pixel of the modified image J is illustrated by a black circle.

In equation (15), J _{00 to} J ₁₁ indicate the pixel values of the four pixels in the deformed image, and the pixel values of the pixels in the restored image. Further, in Expression (15), p is the length from the pixel as the attention point to the upper pixels J ₀₀ and J ₀₁ , and q is the length from the pixel as the attention point to the lower pixels J ₁₀ and J ₁₁ It is. Furthermore, in Expression (15), r is the length from the intersection point of the horizontal line passing through the attention point and the line segment of the pixels J ₀₀ and J ₁₀ on the left side of the attention point to the attention point, and s is the horizontal line passing through the attention point. This is the length from the intersection with the line segment of the pixels J ₀₁ and J ₁₁ on the right side of the point of interest to the point of interest.

[Operation of image restoration device]
Hereinafter, the operation of the image restoration apparatus in FIG. 4 will be described with reference to FIG. First, the image restoration device 3 calculates the correspondence such as the function g of the correspondence represented by the expression (7) or the expression (12) by the mapping restoration means 31 and calculates the coordinates of the sample point in the deformed image J. (Step S1).

  Following the processing of step S1, the image restoration device 3 interpolates the coordinates of the sample points in the deformed image J by the sample point interpolation means 31a (step S2). Further, the image restoration device 3 uses the image restoration means 33 to convert the coordinates of the sample points in the deformed image J to the coordinates of the sample points in the restored image I based on the correspondence relationship such as the correspondence function g. Is calculated as the pixel value of the pixel in the restored image I (step S3). Then, the image restoration device 3 interpolates the pixel values of the pixels in the restored image I by the pixel value interpolation means 33a and outputs the result as the restored image I (step S4).

As described above, according to the image restoration apparatus 3 according to the first embodiment of the present invention, a modified image in which fine sampling is performed on a portion having a complicated pattern and coarse sampling is performed on a portion having a simple pattern. To restore. Therefore, according to the image restoration device 3, it is possible to restore a restored image that is finely expressed with the same amount of information as compared with a conventional image sampled at equal intervals. Further, according to the image restoration device 3, since the deformation parameter is information for each partial region _{Bm, n} , the amount of information can be reduced as compared with the conventional information on all sample points. Furthermore, according to the image restoration device 3, the deformed image can be restored by a simple calculation in the sample axis direction (one-dimensional), and the calculation load can be reduced. As a result, the image restoration device 3 can restore the deformed image more efficiently than the conventional case of restoring the sampled images at regular intervals.

  Here, the image restoration device 3 cannot restore the deformed image without the deformation parameter. For this reason, the image restoration device 3 can also be used as a descrambling device using the deformation parameter as a descrambling key. Further, for example, the image restoration device 3 can be used as an image decoding device that decodes an image or a resolution conversion device that converts the resolution of an image.

  In the first embodiment, the sample axis is described as the horizontal direction or the vertical direction, but the present invention is not limited to this. For example, the sample axis may be an axis in a spatial direction such as an oblique direction, or may be a time axis between frame images or field images when an input image is a moving image.

  In the first embodiment, the mapping restoration unit 31 outputs the correspondence function g and the coordinates of the sample point in the deformed image J to the image restoration unit 33 as a mapping. However, the present invention is not limited to this. . For example, when a pointer to a function in C language or the like is used, or when a lookup table in which a function output value is calculated in advance for a typical argument value is used, it is arbitrarily changed according to the mounting method. it can. For example, the mapping restoration unit 31 may output the correspondence function g as a mapping to the image restoration unit 33, and output the coordinate value of the sample point obtained from the correspondence function g to the image restoration unit 33. Also good.

  In the first embodiment, the image restoration apparatus according to the present invention has been described as an independent apparatus. However, in the present invention, a program that causes a general computer arithmetic device, a storage device, and the like to operate cooperatively as the above-described units. Can also be realized. This program may be distributed via a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

(Modification 1)
Hereinafter, each modification of the image restoration device 3 in FIG. 4 will be described in detail.
As shown in FIG. 14, the image restoration device 3 according to the first modification of the present invention further includes a digital watermark detection unit 34. In this case, the digital watermark detection means 34 receives a modified image to which a deformation parameter is added as a digital watermark. Then, the digital watermark detection unit 34 detects the deformation parameter as a digital watermark from the deformed image, outputs the detected deformation parameter to the mapping restoration unit 31, and outputs the modified image from which the deformation parameter is extracted to the image restoration unit 33. To do. Thus, the image restoration device 3 can conceal the presence of the deformation parameter, and is particularly preferable when the image restoration device 3 is used as a descrambling device.

(Modification 2)
As shown in FIG. 15, the image restoration device 3 according to Modification 2 of the present invention further includes demultiplexing means 35, modified image decoding means 36, and modified parameter decoding means 37.

  The demultiplexing means 35 receives a multiplexed signal obtained by multiplexing the encoded deformation parameter and the encoded modified image. Further, the demultiplexing means 35 demultiplexes the multiplexed signal and separates the encoded deformation parameter and the encoded modified image. Then, the demultiplexing unit 35 outputs the encoded modified image to the modified image decoding unit 36 and outputs the encoded deformation parameter to the modified parameter decoding unit 37. Here, for example, additional information (for example, a bit string as header information or a bit string for synchronization) is added to the multiplexed signal to distinguish between the encoded modified image and the modified parameter.

  The modified image decoding unit 36 decodes the encoded modified image from the demultiplexing unit 35 and outputs the modified image to the image restoration unit 33. For example, the modified image decoding means 36 is a JPEG (Joint Photographic Experts Group) and PNG (Portable Network Graphics) for still images, a Moving Picture Experts Group (MPEG) -2 for moving images, MPEG-4, MPEG-4 AVC. / H. Decoders such as H.264, Motion JPEG, and Windows (registered trademark) Media can be used.

  The deformation parameter decoding unit 37 decodes the encoded deformation parameter from the demultiplexing unit 35 and outputs the deformation parameter to the mapping restoration unit 31. Here, the method, algorithm, and encoding parameters used by the deformation parameter decoding unit 37 are arbitrary. For example, the deformation parameter decoding unit 37 can use a decoder such as thinning out coordinate information and Huffman coding.

(Modification 3)
As shown in FIG. 16, in the image restoration device 3 according to the third modification of the present invention, the mapping restoration unit 31 includes a modification parameter demultiplexing unit 311 and L partial mapping restoration units (correspondence calculation units) 313A. ˜313L and a mapping composition unit (corresponding relationship composition unit) 315 (see FIG. 4 as appropriate).

  The deformation parameter demultiplexing unit 311 receives a deformation parameter obtained by multiplexing the deformation parameters from the first sample axis to the Lth sample axis from the image deformation apparatus 2 in FIG. Further, the deformation parameter demultiplexing unit 311 separates the deformation parameter from the first sample axis to the deformation parameter of the Lth sample axis from the multiplexed deformation parameter. Then, the deformation parameter demultiplexing unit 311 outputs the deformation parameter of the first sample axis to the first partial mapping restoration unit 313A, and outputs the deformation parameter of the second sample axis to the second partial mapping restoration unit 313B. The deformation parameter of the Lth sample axis is output to the Lth partial mapping restoration unit 313L. Note that L indicates the number of sampling dimensions, and is an integer of 2 or more.

  The first partial mapping restoration unit 313A calculates the correspondence in the direction of the first sample axis using the deformation parameter of the first sample axis from the deformation parameter demultiplexing unit 311. Then, the first partial mapping restoration unit 313A outputs the correspondence relationship in the direction of the first sample axis to the mapping composition unit 315. Note that the first partial mapping restoration unit 313A calculates the correspondence as in the mapping restoration unit of FIG.

  In addition, the second partial mapping restoration unit 313B calculates the correspondence in the direction of the second sample axis in the same manner as the first partial mapping restoration unit 313A, and outputs it to the mapping synthesis unit 315. Further, the L-th partial mapping restoration unit 313L calculates the correspondence in the direction of the L-th sample axis in the same manner as the first partial mapping restoration unit 313A, and outputs it to the mapping synthesis unit 315. Note that the L partial mapping restoration units 313A to 313L correspond to the correspondence calculation unit described in the claims.

The mapping composition unit 315 synthesizes the correspondences input from each of the L partial mapping restoration units 313A to 313L and outputs the combined relationship to the image restoration unit 33 in FIG. Here, the mapping composition unit 315 synthesizes the correspondence using, for example, the following equation (16). In Expression (16), g represents the combined correspondence (mapping), and g _L represents the correspondence of each dimension (each sample axis).

  The image restoration unit 33 (see FIG. 4) restores a restored image from the non-uniformly sampled images sampled with L sample axes based on the correspondence relationship input from the mapping synthesis unit 315. Note that the restoration process of the image restoration unit 33 is the same as that of the image restoration unit 33 of FIG. As a result, the image restoration apparatus 3 can restore the restored image by performing the restoration process only once for the non-uniformly sampled images sampled by the L sample axes, and the calculation is simple.

(Second Embodiment)
[Configuration of image restoration device]
Hereinafter, the configuration of the image restoration apparatus according to the second embodiment of the present invention will be described with reference to FIG. As shown in FIG. 17, an image restoration device (multidimensional image restoration device) 200 is obtained by connecting two image restoration devices 3 in FIG. 4 in series. An image restoration device 3B and deformation parameter demultiplexing means 38 are provided. The image restoration apparatus 200 corresponds to the case of k = 2 in claim 6.

  Here, the first sample axis is assumed to be the horizontal direction, and the second sample axis is assumed to be the vertical direction. The input image is a two-dimensional image (two-dimensional still image). In FIG. 17, the respective units of the first image restoration device 3A and the second image restoration device 3B are not shown.

  The deformation parameter demultiplexing means 38 is a deformation parameter obtained by multiplexing the deformation parameter of the first sample axis (horizontal direction) and the deformation parameter of the second sample axis (vertical direction) from the image deformation apparatus 2 of FIG. Is entered. Further, the deformation parameter demultiplexing means 38 separates the deformation parameter of the first sample axis and the deformation parameter of the second sample axis from the multiplexed deformation parameter. Then, the deformation parameter demultiplexing means 38 outputs the deformation parameter of the first sample axis to the first image restoration device 3A, and outputs the deformation parameter of the second sample axis to the second image restoration device 3B.

  The first image restoration device 3A receives the deformation parameter of the first sample axis from the deformation parameter demultiplexing means 38 and the deformation image from the image deformation device 2 in FIG. Then, the first image restoration device 3A restores the deformed image in the direction of the first sample axis using the deformation parameter of the first sample axis. Further, the first image restoration device 3A outputs the restored image to the second image restoration device 3B.

  The second image restoration device 3B receives the deformation parameter of the second sample axis from the deformation parameter demultiplexing unit 38 and the restored image from the first image restoration device 3A. Then, the second image restoration device 3B restores the restored image in the direction of the second sample axis using the deformation parameter of the second sample axis. Further, the second image restoration device 3B outputs the restored image.

  As described above, according to the image restoration apparatus 200 according to the second embodiment of the present invention, the same effects as those of the image restoration apparatus 3 in FIG. 4 are obtained. Furthermore, according to the image restoration apparatus 200, a two-dimensional image can be restored only by repeating a simple one-dimensional restoration process twice, thereby reducing a calculation load.

  In the second embodiment, the first sample axis is the horizontal direction and the second sample axis is the vertical direction. However, the present invention is not limited to this. For example, when the first sample axis and the second sample axis are spatial axes, the two axes need only intersect. Further, for example, one of the first sample axis and the second sample axis may be a spatial axis, and the other may be a temporal axis.

In the case of one dimension (in the case of k = 1), the division of the partial area is a division of a one-dimensional number line, and instead of the function g in the formula (7) or the formula (12), the formula (6) Alternatively, x _n (u) in Equation (11) (n = 1 because it is one-dimensional) may be used as it is as the position of the sample point.

(Third embodiment)
[Configuration of image restoration device]
Hereinafter, the configuration of the image restoration apparatus according to the third embodiment of the present invention will be described with reference to FIG. As shown in FIG. 18, an image restoration device (multidimensional image restoration device) 300 is obtained by connecting three image restoration devices 3 in FIG. 4 in series, and includes a first image restoration device 3A and a second image restoration device 3A. An image restoration device 3B, a third image restoration device 3C, and deformation parameter demultiplexing means 38B are provided. The image restoration apparatus 300 corresponds to the case of k = 3 in claim 6.

  Here, the first sample axis is the horizontal direction (x direction in FIGS. 19 to 22), the second sample axis is the vertical direction (y direction in FIGS. 19 to 22), and the third sample axis is the depth direction (see FIG. 19). 19 to z in FIG. 22). The input image is a three-dimensional image (stereoscopic still image). In FIG. 18, the respective units of the first image deformation device 2A, the second image deformation device 2B, and the third image deformation device 2C are not shown. The first image restoration device 3A and the second image restoration device 3B are the same as the respective devices in FIG.

  The deformation parameter demultiplexing means 38B receives the deformation parameter of the first sample axis (horizontal direction), the deformation parameter of the second sample axis (vertical direction), and the third sample axis (depth) from the image deformation apparatus 2 in FIG. A deformation parameter obtained by multiplexing the deformation parameter of (direction) is input. Further, the deformation parameter demultiplexing means 38B separates the deformation parameter for the first sample axis, the deformation parameter for the second sample axis, and the deformation parameter for the third sample axis from the multiplexed deformation parameters. Then, the deformation parameter demultiplexing means 38B outputs the deformation parameter of the first sample axis to the first image restoration device 3A, outputs the deformation parameter of the second sample axis to the second image restoration device 3B, and The deformation parameters of the three sample axes are output to the third image restoration device 3C.

  The third image restoration device 3C receives the deformation parameter of the third sample axis from the deformation parameter demultiplexing unit 38B and the restored image from the second image restoration device 3B. Then, the third image restoration device 3C restores the restored image in the direction of the third sample axis using the deformation parameter of the third sample axis. Further, the third image restoration device 3C outputs the restored image.

<Interpolation of partial area, scanning line and sample point of 3D image>
Hereinafter, with reference to FIGS. 19 to 23, the interpolation of the partial region, the scanning line, and the sample point of the three-dimensional image will be described in detail. Since the input image is a three-dimensional image, the first image restoration device 3A divides the input image (three-dimensional image) into cubic or cuboid partial regions as shown in FIG. FIG. 20 is an enlarged view of a partial region located in the center of FIG. Further, in FIG. 19 and FIG. 20, only a part of the partial area is shown for the sake of simplicity.

Further, as shown in FIG. 20, the first image restoration device 3A calculates the coordinates of the sample points from the complexity of the first sample axis (horizontal direction) for each partial region. In the example of FIG. 20, the complexity of the partial area BL1 on the upper left front side is 5, the complexity of the partial area BL2 on the lower left front side is 2, and the complexity of the partial area BL3 on the upper right front side is 2. The complexity of the partial area BL4 on the lower right front side is 3. In the example of FIG. 20, the number U of sample points per scanning line S _{1 to} S ₄ is 5. Note that the complexity of the rear partial regions BL5 to BL8 is omitted for the sake of simplicity.

First, the first image restoration apparatus 3A, the representative point of the partial region BL1 to BL8 (e.g., center point) of each scan line _S 1 to S ₄ which passes in the horizontal direction to calculate the coordinates of the sampled points. Here, the scan lines _{S 1} is 21, as shown in FIG. 22, it passes through the center point of the partial region BL1, BL3 horizontally. Further, the scanning lines _{S 2} passes through the center point of the partial region BL2, BL4 in the horizontal direction. Then, the scan lines _{S 3} passes through the center point of the partial region BL5, BL7 horizontally. Further, the scanning lines _{S 4} passes through the center point of the partial region BL6, BL8 horizontally.

21 is a top view of FIG. 20 (view from the direction of the black arrow in FIG. 20), and FIG. 22 is a right side view of FIG. 20 (view from the direction of the white arrow in FIG. 20). ). Here, in FIG. 22, the scanning lines S _{1 to} S ₄ look like dots when viewed from the right side.

Further, the first image restoration device 3A calculates the coordinates of the sample points as shown in FIG. In FIG. 23, the calculated sample points are indicated by black circles, and the interpolated sample points are indicated by white circles. More specifically, as shown in FIG. 23 (b), the first image restoration apparatus 3A, on the scanning line _{S 5} between the scan lines _S 1, _{S 2} of the two, by interpolation, 5 Interpolate the coordinates of the sample points. Further, as shown in FIG. 23 (c), the first image restoration apparatus 3A, on the scanning line _{S 6} between the scan lines _S 3, _{S 2} of the two, by interpolation, the five sample points Interpolate the coordinates of. Furthermore, as shown in FIG. 23 (d), the first image restoration apparatus 3A, on the scanning line _{S 7} between the _S 5, _{S 6} 2 scanning lines, by interpolation, the five sample points Interpolate the coordinates of.

  In FIG. 19 to FIG. 23, the example in which the sample point is horizontally interpolated by the first image restoration device 3 </ b> A has been described. Further, in the same procedure, the sample point is interpolated in the vertical direction by the second image restoration device 3B and the sample point is interpolated in the depth direction by the third image restoration device 3C.

  As described above, according to the image restoration apparatus 300 according to the third embodiment of the present invention, the same effects as those of the image restoration apparatus 3 in FIG. 4 can be obtained. Furthermore, according to the image restoration apparatus 300, a three-dimensional image can be restored only by repeating a simple one-dimensional restoration process three times, thereby reducing a calculation load.

(Fourth embodiment)
[Configuration of image restoration device]
The configuration of the image restoration apparatus according to the fourth embodiment of the present invention will be described below with reference to FIG. As shown in FIG. 24, an image restoration device (multidimensional image restoration device) 400 is obtained by connecting four image restoration devices 3 of FIG. 4 in series, and includes the first image restoration device 3A and the second image restoration device 3A. An image restoration device 3B, a third image restoration device 3C, a fourth image restoration device 3D, and a deformation parameter demultiplexing means 38C are provided. The image restoration apparatus 400 corresponds to the case of k = 4 in claim 6.

  Here, the first sample axis is the horizontal direction, the second sample axis is the vertical direction, the third sample axis is the depth direction, and the fourth sample axis is the time direction. The input image is a four-dimensional image (stereoscopic moving image). In FIG. 24, the respective units of the first image restoration device 3A, the second image restoration device 3B, the third image restoration device 3C, and the fourth image restoration device 3D are not shown. The first image restoration device 3A, the second image restoration device 3B, and the third image restoration device 3C are the same as the respective devices in FIG.

  The deformation parameter demultiplexing means 38C receives the deformation parameter of the first sample axis (horizontal direction), the deformation parameter of the second sample axis (vertical direction), and the third sample axis (depth) from the image deformation apparatus 2 of FIG. (Direction) deformation parameter and the fourth sample axis (time direction) deformation parameter are multiplexed. Further, the deformation parameter demultiplexing means 38C determines the deformation parameter of the first sample axis, the deformation parameter of the second sample axis, the deformation parameter of the third sample axis, and the deformation parameter of the fourth sample axis from the multiplexed deformation parameters. And are separated. Then, the deformation parameter demultiplexing means 38C outputs the deformation parameter of the first sample axis to the first image restoration device 3A, outputs the deformation parameter of the second sample axis to the second image restoration device 3B, and The deformation parameters of the three sample axes are output to the third image restoration device 3C, and the deformation parameters of the fourth sample axis are output to the fourth image restoration device 3D.

  The fourth image restoration device 3D receives the deformation parameter of the fourth sample axis from the deformation parameter demultiplexing unit 38C and the restored image from the third image restoration device 3C. Then, the fourth image restoration device 3D restores the restored image in the direction of the fourth sample axis using the deformation parameter of the fourth sample axis. Furthermore, the fourth image restoration device 3D outputs the restored image.

  As described above, according to the image restoration apparatus 400 according to the fourth embodiment of the present invention, the same effects as those of the image restoration apparatus 3 in FIG. 4 can be obtained. Furthermore, according to the image restoration apparatus 400, a four-dimensional image can be restored only by repeating a simple one-dimensional restoration process four times, thereby reducing a calculation load.

DESCRIPTION OF SYMBOLS 1 Image deformation | transformation restoration system 2 Image deformation | transformation apparatus 3 Image restoration apparatus 3A 1st image restoration apparatus 3B 2nd image restoration apparatus 3C 3rd image restoration apparatus 3D 4th image restoration apparatus 31 Mapping restoration means (correspondence calculation means) )
31a Sampling point interpolation means 311 Deformation parameter demultiplexing units 313A to 313L Partial mapping restoration unit (correspondence calculation unit)
315 Mapping composition unit (corresponding relationship composition unit)
33 Image restoration means 33a Pixel value interpolation means 34 Digital watermark detection means 35 Demultiplexing means 36 Modified image decoding means 37 Modified parameter decoding means 38, 38B, 38C Modified parameter demultiplexing means 200 Image restoration apparatus (multidimensional image restoration apparatus) )
300 Image restoration device (multidimensional image restoration device)
400 image restoration device (multi-dimensional image restoration device)

Claims (4)

The input image is sampled at unequal intervals using a complexity indicating the complexity of each partial area obtained by dividing the input image and the complexity, and a predetermined sample axis in the spatial direction or time direction. A non-uniformly sampled image is input from the nonuniformly sampled device, and using the complexity, an image restoring device that restores a restored image from the nonuniformly sampled image,
For each scanning line that passes through all the partial areas that are continuous in the direction of the sample axis, there are many sample points in the high-complexity partial area from the complexity by the number of preset sample points, and Correspondence calculating means for calculating the correspondence between the coordinates of the sample points in the restored image and the coordinates of the sample points in the non-uniformly sampled image so that the number of sample points is reduced in the partial area with low complexity When,
Based on the correspondence calculated by the correspondence calculation means, the pixel value of the pixel corresponding to the coordinates of the sample points in the non-uniformly sampled image is changed to the pixel value of the pixel corresponding to the coordinates of the sample points in the restored image. Image restoration means for restoring the restored image;
Equipped with a,
Wherein with complexity is added to said non-equidistant sampled image as a digital watermark is input, further comprising characterized Rukoto digital watermark detection means for detecting the complexity from the non-equidistant sampling image Image restoration device. The input image is sampled at unequal intervals using a complexity indicating the complexity of each partial area obtained by dividing the input image and the complexity, and a predetermined sample axis in the spatial direction or time direction. A non-uniformly sampled image is input from the nonuniformly sampled device, and using the complexity, an image restoring device that restores a restored image from the nonuniformly sampled image,
For each scanning line that passes through all the partial areas that are continuous in the direction of the sample axis, there are many sample points in the high-complexity partial area from the complexity by the number of preset sample points, and Correspondence calculating means for calculating the correspondence between the coordinates of the sample points in the restored image and the coordinates of the sample points in the non-uniformly sampled image so that the number of sample points is reduced in the partial area with low complexity When,
Based on the correspondence calculated by the correspondence calculation means, the pixel value of the pixel corresponding to the coordinates of the sample points in the non-uniformly sampled image is changed to the pixel value of the pixel corresponding to the coordinates of the sample points in the restored image. Image restoration means for restoring the restored image;
With
From the unequal interval sampling apparatus, the unequal interval sampled image obtained by sampling the input image with two or more sample axes, and the complexity calculated for each sample axis are input,
The correspondence calculation means
Two or more correspondence calculation units for calculating the correspondence for each sample axis from the complexity calculated for each sample axis;
Additionally and a correspondence synthesizing unit for synthesizing the correspondence between said two or more correspondence calculation unit has calculated,
The image restoration means includes
The correspondence relationship combining unit based on the synthesized correspondence, the two or more from the non-equidistant sampling images that have been sampled at a sampling axis, images restoration device characterized by restoring the restored image. An image restoration program for causing a computer to function as the image restoration device according to claim 1. The input image is sampled at unequal intervals using a complexity indicating the complexity of each partial area obtained by dividing the input image and the complexity, and a predetermined sample axis in the spatial direction or time direction. A non-uniformly sampled image is input from the nonuniformly sampled device, and using the complexity, an image restoring device that restores a restored image from the nonuniformly sampled image,
For each scanning line that passes through all the partial areas that are continuous in the direction of the sample axis, there are many sample points in the high-complexity partial area from the complexity by the number of preset sample points, and Correspondence calculating means for calculating the correspondence between the coordinates of the sample points in the restored image and the coordinates of the sample points in the non-uniformly sampled image so that the number of sample points is reduced in the partial area with low complexity When,
Based on the correspondence calculated by the correspondence calculation means, the pixel value of the pixel corresponding to the coordinates of the sample points in the non-uniformly sampled image is changed to the pixel value of the pixel corresponding to the coordinates of the sample points in the restored image. Image restoration means for restoring the restored image;
A multi-dimensional image restoration apparatus connected to the image restoration apparatus in k stand series with a,
The first image restoration device receives a k-dimensional image as the non-uniformly sampled image, and restores the k-dimensional image with a preset first sample axis in a spatial direction or a time direction,
The kth image restoration device is characterized in that the restored image restored by the k-1 unequal interval sampling device is inputted and the restored image is restored in the direction of the kth sample axis. Multidimensional image restoration apparatus (where k is an integer of 2 or more).

Images