JP2010193154A - Image processor and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor capable of automatically detecting an important part inside an image without specifying the important part by an input device and enlarging or reducing an image while preferentially securing the visibility of the important part. <P>SOLUTION: For the image input to an image input part 100, the image is divided into two or more parts on the basis of the total sum of the edge intensity values of pixels in the horizontal direction and the vertical direction and a magnification coefficient is calculated by part in a coefficient calculation part 200. The image input to the image input part 100 is enlarged or reduced by part on the basis of the calculated magnification coefficients in an enlargement/reduction processing part 300, and is output from an image output part 400. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method for partially enlarging or reducing an image in an apparatus for displaying and editing an image such as a personal computer, a portable terminal, and a car navigation system.

  Images taken by a digital camera or the like are stored as digital data in a memory or hard disk. Confirmation of a photographed image is generally performed by displaying the image on a monitor attached to a photographing device or a monitor connected to a personal computer. The number of pixels and the aspect ratio of the image vary depending on the specifications of the photographing device and the editing content of the image data, and often do not match the number of pixels on the monitor display screen. Therefore, a process is performed in which the entire image is uniformly enlarged or reduced so that the number of pixels and the aspect ratio match the monitor display screen as much as possible.

  For example, when displaying an image with a small image size, such as a picture taken with a mobile terminal, on a monitor connected to a personal computer, the monitor display screen has more pixels than the image to be displayed, and there is room in the monitor display screen. The visibility of the image can be enhanced by displaying the entire image at a uniform magnification. However, if the enlargement magnification is too large, a portion that is not displayed on the monitor display screen becomes large, and it becomes difficult to confirm the entire image.

  As a solution to this problem, there is a technique in which an enlarged region is selected by an input device, and an image is displayed by partially enlarging only the enlarged region while maintaining continuity at the boundary between the enlarged region and the non-enlarged region. (For example, refer to Patent Document 1). By selectively enlarging only the part important for the user, it is possible to improve the visibility of the part important for the user while keeping the part not displayed on the monitor as small as possible.

  In recent digital cameras, the number of pixels of still images that can be photographed exceeds megapixels (1 million pixels), whereas the number of pixels on the display screen of a portable terminal or a car navigation system is several tens. Many are limited to about 10,000 pixels. That is, since the number of pixels of the image to be displayed is larger than the number of pixels of the display screen, it is necessary to reduce the image in order to display the entire image. In this case, the entire image can be displayed by reducing the image at a uniform magnification.

JP-A-10-105153 (page 4-7, FIG. 1)

  By the way, in the technique described in Patent Document 1, it is necessary for a user to designate a portion to be enlarged using an input device. Therefore, it is possible to easily specify an important part to be enlarged if the device can use an input device that can specify an intuitive area such as a mouse or a cursor. However, in a portable terminal or a car navigation system, it is generally intuitive. There is no input device or mechanism that can specify a specific area, and there is a problem that a portion to be enlarged cannot be easily specified.

  In addition, if the number of pixels of the image to be displayed is larger than the number of pixels on the display screen, the entire image can be displayed by reducing the image at a uniform magnification. Then, the visibility of an image will deteriorate. Therefore, there has been a demand for preferentially ensuring the visibility of important portions while reducing the entire image to fit the display screen.

  The present invention has been made to solve the above-described problems. Even if an important part is not designated by an input device, the important part in the image is automatically detected and priority is given to the visibility of the important part. An object of the present invention is to provide an image processing apparatus capable of enlarging or reducing an image while ensuring it.

  In the image processing apparatus according to the present invention, paying attention to the fact that the important part that the user wants to ensure visibility is a part having a large edge intensity value, the image is divided into a plurality of parts based on the edge intensity value of each pixel of the image, A magnification coefficient indicating a magnification for enlarging or reducing each of these parts was calculated, and each part was enlarged or reduced according to the magnification coefficient.

  In the image processing apparatus according to the present invention, there is an effect that it is possible to obtain an image in which the visibility of the important part is preferentially secured without specifying the important part.

It is a block diagram which shows the structure of the image processing apparatus in Embodiment 1 of this invention. It is an example of the image in each step in Embodiment 1 of this invention. It is explanatory drawing of the calculation process of the magnification factor in Embodiment 1 of this invention. It is explanatory drawing of the example of calculation of the magnification coefficient determination threshold value THx in Embodiment 1 of this invention. FIG. 3 is a magnification coefficient distribution diagram showing magnification coefficients in horizontal and vertical directions for each pixel position according to the first embodiment of the present invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 2 of this invention. It is an example of the image in each step in Embodiment 2 of this invention. It is explanatory drawing of the calculation process of the magnification factor in Embodiment 1 of this invention. It is explanatory drawing of the calculation process of the magnification factor in Embodiment 2 of this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 3 of this invention. It is an example of the image which shows the processing result in each step in Embodiment 3 of this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 4 of this invention. It is explanatory drawing of the magnification coefficient change operation | movement in Embodiment 4 of this invention. It is explanatory drawing of the input image and enlarged image in Embodiment 4 of this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 5 of this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 6 of this invention. It is an example of the image in each step in Embodiment 6 of this invention. It is a figure explaining the positional relationship of the to-be-photographed object at the time of imaging | photography of the input image used in Embodiment 6 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to Embodiment 1 of the present invention. The image processing apparatus according to the first embodiment includes an image input unit 100, a coefficient calculation unit 200, an enlargement / reduction processing unit 300, and an image output unit 400.

  Image data constituting a still image or video data constituting a moving image is input to the image input unit 100. For example, image data captured by a camera, image data or video data stored in a memory, and image data or video data in a file format held in a hard disk are input. These image data or video data are input in various formats such as signal data such as RGB (red green blue) and YCbCr (luminance color difference), image file format such as bitmap format and JPEG format, and device-specific data format. The If the input image data is, for example, in YCbCr format, the image input unit 100 extracts the luminance signal data from the luminance data and outputs it to the coefficient calculation unit 200. Further, the image input unit 100 outputs the input image data itself in the YCbCr format to the enlargement / reduction processing unit 300. When temporally continuous video data is input, the same processing as that of the image data can be performed by once converting the image data into frame-based image data.

  In the following description, a system for inputting image data captured by a camera in the YCbCr format, outputting RGB data, and displaying the image data on a monitor screen attached to a personal computer will be described as an example. It is not limited.

  The coefficient calculation unit 200 includes an edge extraction unit 201, a feature amount calculation unit 202, and a magnification coefficient determination unit 203. The edge extraction unit 201 calculates the edge intensity value of each pixel of the image data input to the image input unit 100 from the luminance signal data that is the output of the image input unit 100 and outputs it. The feature amount calculation unit 202 calculates a feature amount such as a sum or average value of edge strength values for each pixel row in the horizontal direction and for each pixel column in the vertical direction with respect to the edge strength value output from the edge extraction unit 201. And output. The magnification coefficient determination unit 203 determines a magnification coefficient representing a magnification for enlarging or reducing each pixel in the horizontal and vertical directions in the image formed from the output of the feature amount calculation unit 202.

  The enlargement / reduction processing unit 300 enlarges or reduces the image according to the horizontal / vertical magnification coefficient for each pixel, which is the output of the coefficient calculation unit 200, with respect to the image data from the image input unit 100.

  The image output unit 400 rearranges the image data output from the enlargement / reduction processing unit 300 for display on a monitor or the like and outputs it together with a synchronization signal, converts it into a file format, stores it, or stores it in a memory. Processing according to the format is performed as necessary.

  FIG. 2 is an example of an image at each stage of the image processing apparatus of FIG. 1, and (a) is an original image input to the image input unit 100. As an example, here, the original image (a) is a natural image with the number of pixels in the horizontal direction being W and the number of pixels in the vertical direction being H, taken with a person in focus with mountains and clouds in the background. (B) is a result image obtained by calculating the edge intensity value by the edge extraction unit 201 from the luminance signal data of the original image (a). Here, a portion where the edge strength value is large is indicated by a solid line, and a portion where the edge strength value is small is indicated by a dotted line. (C) is an output image from the image output unit 400. In FIG. 2A, the coordinates represent the position of each pixel in the horizontal direction and the vertical direction as (horizontal direction x, vertical direction y), and the position of the upper left pixel of the image is (0, 0). The position of the lower right pixel is (W-1, H-1). The same applies to other drawings unless otherwise noted.

  In the original image (a), an important part that the user of the image processing apparatus according to the present embodiment wants to ensure visibility is a person part. Since the person portion in this image is photographed while focusing on the person, the edge intensity value is relatively larger than the background mountain and cloud portions as shown in FIG. As described above, in many cases, an important portion where the user wants to ensure visibility is generally a portion having a large edge strength value. The object of the present embodiment is to detect this human part having a relatively large edge strength value as an important part and obtain an image in which the visibility of this part is secured with priority.

  Next, the operation at each stage in the first embodiment of the present invention will be described.

  Assume that image data as shown in FIG. 2A is input to the image input unit 100 in the YCbCr format. In response to this, the image input unit 100 extracts only the luminance signal data Y from the image data and outputs it to the coefficient calculation unit 200. Further, the image input unit 100 outputs to the enlargement / reduction processing unit 300 the image data corresponding to FIG. 2A converted into RGB data by inverse matrix conversion. In the present embodiment, the image input unit 100 outputs 8 bits of luminance signal data per pixel to the coefficient calculation unit 200, and a total of 24 RGB signals of 8 bits per pixel to the enlargement / reduction processing unit 300. Bit data is sequentially output by scanning the image horizontally and vertically.

  The luminance signal data Y from the image input unit 100 is input to the edge extraction unit 201. For the edge intensity value extraction process, a general filter such as a Sobel filter is used, and a value obtained by taking the absolute value of the output of the filter is used as the edge intensity value. The edge intensity value increases as the luminance gradient difference between adjacent pixels increases. The edge intensity value can be calculated for each horizontal and vertical direction, but in order to limit the edge intensity value per pixel to one, the larger one of the calculated values in each horizontal and vertical direction is selected. To do. As a result of the edge extraction, the image of FIG. 2A becomes as shown in FIG.

  The feature amount calculation unit 202 calculates the sum of the edge intensity values obtained by the edge extraction unit 201 for each pixel row and each pixel column in each horizontal and vertical direction of the image. First, with respect to the vertical direction of the image, the pixels in the vertical direction position y = 0 to y = (H−1) for each pixel column in which the horizontal position is between x = 0 and x = (W−1). Vertical sum total Sx (x) (x is an integer from 0 to (W−1)), which is the sum of the edge intensity values of. Similarly, in the horizontal direction, for each pixel row whose vertical position is between H = 0 and y = (H−1), the horizontal position x = 0 to x = (W−1). Horizontal sum Sy (y) (y is an integer from 0 to (H−1)), which is the sum of the edge intensity values of the pixels.

  The magnification coefficient determination unit 203 compares each value from x = 0 to x = (W−1) of the vertical direction total sum Sx (x) obtained by the feature amount calculation unit 202 with a predetermined magnification coefficient determination threshold THx. . When the magnification coefficient determination threshold value THx is equal to or larger than the magnification coefficient determination threshold value THx, the horizontal magnification coefficient of each pixel included in the pixel column is set to m (m is a number greater than 1), for example. If the magnification coefficient is smaller than the threshold THx, the horizontal magnification coefficient of each pixel included in the pixel column is set to 1, for example.

  Similarly, each value from y = 0 to y = (H−1) of the horizontal total sum Sy (y) is also compared with a predetermined magnification coefficient determination threshold value THy. If the magnification coefficient determination threshold THy is equal to or greater than the magnification coefficient determination threshold THy, the vertical magnification coefficient of the pixels included in the pixel row is set to, for example, n (n is a number greater than 1). If the magnification coefficient determination threshold value THy is smaller, the vertical magnification coefficient of each pixel included in the pixel row is set to 1, for example.

  The magnification coefficient determines how many times the number of pixels in the horizontal direction or the vertical direction of the pixel is enlarged or reduced. If the magnification coefficient is 1, it represents the same magnification. Enlargement or reduction is performed by the enlargement / reduction processing unit 300. For example, when 10 pixels with a horizontal magnification factor of 1.5 are arranged in the horizontal direction, the enlargement is performed so that 15 pixels are arranged in the horizontal direction. Is done.

  Hereinafter, a method for determining a magnification factor in the magnification factor determination unit 203 will be described with reference to FIG. FIG. 3 is an explanatory diagram of the calculation process of the magnification coefficient used in the magnification coefficient determination unit 203, and shows an example of calculating the sum of the edge strength values and the magnification coefficient for each of the horizontal and vertical directions with respect to the image of FIG. ing.

  FIG. 3A is a graph in which the vertical direction total sum Sx (x) is obtained for each of the pixels whose horizontal direction position x is 0 to (W−1), and the vertical axis indicates the vertical direction total sum Sx (x) and the horizontal direction. A horizontal position was taken on the axis. The horizontal position x, which is the horizontal axis of the graph, is displayed approximately in accordance with the horizontal position of the image. In the image of FIG. 3, since the edge intensity value of the person portion is large, the vertical direction sum Sx (x) is a large value at the pixel position where the person is photographed. On the other hand, FIG. 3C is a graph of the magnification factor in the horizontal direction for the pixels included in each pixel column where the horizontal position x is between 0 and (W−1). In FIG. 3C, the horizontal axis represents the horizontal position, and the vertical axis represents the horizontal magnification coefficient of the pixel at each horizontal position. Further, FIG. 3B is a graph obtained by rotating the graph in which the horizontal axis represents the vertical position and the vertical axis represents the horizontal sum Sy (y) 90 degrees to the left as in FIG. 3A. is there. 3D, as in FIG. 3C, the graph in which the horizontal axis represents the vertical position and the vertical axis represents the vertical magnification coefficient of the pixel at each vertical position is rotated 90 degrees to the left. It has been made.

  As shown in FIG. 3A, a predetermined value is set as the magnification coefficient determination threshold value THx for the vertical direction sum Sx (x). In the graph of the sum total Sx (x) in the vertical direction, the horizontal magnification coefficient m is a pixel corresponding to the horizontal direction intersecting the magnification coefficient determination threshold THx and a pixel at a horizontal position above the magnification coefficient determination threshold THx. The horizontal magnification coefficient is set to 1 for the pixel in the horizontal position that is below the magnification coefficient determination threshold THx. The horizontal magnification coefficient thus obtained is as shown in FIG.

By performing processing similar to that in the horizontal direction for the vertical direction of the image, a graph of the total sum Sy (y) in the horizontal direction as shown in FIG. 3B is obtained. The graph of the horizontal sum Sy (y) is a pixel row intersecting the magnification factor determination threshold THy and pixels included in a vertical position that is above the magnification factor determination threshold THy. For a pixel included in a vertical position below the determination threshold THy, the vertical magnification factor is set to 1. Accordingly, as shown in FIG. 3D, a graph of the magnification factor in the vertical direction is obtained for the pixels whose vertical position y is between 0 and (H−1).

  Next, a calculation example of the magnification coefficient determination threshold in the present embodiment will be described with reference to FIG. FIG. 4 is a graph for explaining the graph of FIG. 3A in more detail, and is an explanatory diagram of an example of calculation of the magnification coefficient determination threshold THx in the feature amount calculation unit 202.

  First, for the vertical direction sum Sx (x), the minimum value MINx and the average value AVEx when the horizontal position x is between 0 and (W−1) are obtained. Next, half of the value obtained by subtracting the minimum value MINx from the average value AVEx is set as OFFSETx, and the value obtained by subtracting OFFSETx from the average value AVEx is set as the magnification coefficient determination threshold THx. Since the edge strength value uses an absolute value, all values are 0 or more, and the magnification coefficient determination threshold THx is always set to a value between the average value AVEx and the minimum value MINx.

  The magnification coefficient determination threshold THy is also set for the horizontal sum Sy (y) in the same manner as the magnification coefficient determination threshold THx. That is, the minimum value MINy and the average value AVEy when the horizontal position y is between 0 and (H−1) are obtained. Next, half of the value obtained by subtracting the minimum value MINy from the average value AVEy is set as OFFSETy, and the value obtained by subtracting OFFSETy from the average value AVEy is set as the magnification coefficient determination threshold THy.

  In the present embodiment, the magnification coefficient determination thresholds THx and THy are calculated independently, but the same value may be set. In addition, although the calculation is performed from the image data, a different calculation formula may be employed, or a result obtained by a plurality of calculation formulas may be selected and selected from values prepared in advance. Furthermore, it may be configured such that it can be set to a fixed value regardless of the image, or can be directly designated from the outside of the image processing apparatus.

  FIG. 5 is a magnification coefficient distribution diagram showing the horizontal and vertical magnification coefficients determined by the magnification coefficient determining unit 203 for each pixel position. In this figure, the value on the left side in parentheses indicates the horizontal magnification factor, and the right side indicates the vertical magnification factor. For example, an area indicated by (m, 1) indicates that the horizontal magnification factor of the pixels included in the area is m and the vertical magnification factor is 1. FIGS. 5C and 5D are the same graphs as FIGS. 3C and 3D. In this example, as shown in FIG. 5, the image is divided into nine regions corresponding to the magnification factor, and in the central region, the horizontal direction is enlarged by m times and the vertical direction is enlarged by n times. The upper right and lower right four areas remain the same in both the horizontal and vertical directions, while the upper middle and lower center areas expand to m times in the horizontal direction. In the middle region, the horizontal direction remains the same magnification, and the vertical direction is enlarged n times.

In this way, the coefficient calculation unit 200 calculates one horizontal magnification coefficient for each column of the image and one vertical magnification coefficient for each row, and outputs them to the enlargement / reduction processing unit 300.

  In the above example, the horizontal magnification coefficient is m and the vertical magnification coefficient is n (m and n are both greater than 1) and different values in each direction. However, the same value may be used.

  The magnification factor is selected from two types of 1 and a number greater than 1 (m in the horizontal direction and n in the vertical direction). By using a plurality of magnification factor determination thresholds, three or more types of magnification factors are used. It is good also as such a structure.

  The magnification coefficient is set to 1 and a number larger than 1. However, if the magnification coefficient of a pixel whose sum of edge intensity values is larger than the magnification coefficient determination threshold is a value relatively larger than the magnification coefficient of a small pixel, the number is 1. It may be a small number. In this case, the enlargement / reduction processing unit 300 described below performs the reduction process.

  The same applies to the second and subsequent embodiments.

  Next, in FIG. 1, an enlargement / reduction processing unit 300 performs an enlargement process and a reduction process on an image input from the image input unit 100 based on the horizontal and vertical magnification coefficients calculated by the coefficient calculation unit 200. . As an interpolation method, a general method such as a bilinear method or a bicubic method is used. In the portion where the magnification factor is 1, when one pixel is input, one pixel is output. In the portion where the magnification coefficient is not 1, when one pixel is input, the same number of pixels as the magnification coefficient are output. The output pixel value depends on the interpolation method.

  FIG. 2C illustrates an output image that is enlarged according to the magnification coefficient illustrated in FIG. 5 and output from the image output unit 400. In this output image, the person portion in the image is enlarged in the horizontal direction and the vertical direction, and the number of pixels of the output image is larger than the input image. The cloud on the upper right side of the image is the same size in both the horizontal and vertical directions. On the other hand, the background mountain is distorted by straddling a plurality of regions having different magnification factors in the vertical direction or the horizontal direction. Since the magnification factor is determined depending on the contents of the input image, the aspect ratio is not stored.

  The image output unit 400 outputs image data so that the image subjected to the enlargement process by the enlargement / reduction processing unit 300 can be displayed on the screen of a monitor attached to the personal computer.

In addition, when the data input to the image input unit 100 is a plurality of temporally continuous image data or video data, a plurality of frames are represented so that the same portion can be enlarged between the plurality of frames. Alternatively, the luminance data of one frame may be sent to the coefficient calculation unit 200 to calculate the magnification factor for this frame, and the enlargement / reduction processing unit 300 may enlarge the remaining frames with the same magnification factor.

  As described above, in the image processing apparatus according to the present embodiment, the image is divided into a plurality of parts based on the edge intensity value of each pixel of the image, and the magnification coefficient of the part with the large edge intensity value is greater than the magnification coefficient of the part with the small edge intensity value. The magnification factor is calculated for each part so that it becomes, and each part is enlarged or reduced according to the magnification coefficient, so that the visibility of the important part is secured with priority without specifying the important part. Images can be obtained.

Embodiment 2. FIG.
FIG. 6 is a block diagram showing the configuration of the image processing apparatus according to Embodiment 2 of the present invention. The same parts as those in the image processing apparatus shown in FIG. A difference is that a threshold processing unit 204 is added between the edge extraction unit 201 and the feature amount calculation unit 202 of the unit 220.

  Hereinafter, the case of enlarging an image will be described as an example, but the present embodiment can be applied even when the image is reduced.

  FIG. 7 is an example of an image at each stage of FIG. 6, (a) is an original image input to the image input unit 100, (b) is a result image obtained by the edge extraction unit 201, (c ) Is a result image obtained by the threshold processing unit 204. There are trees and grass in the background of the input original image (a), which is more complicated than the background of FIG. 2 (a) in the first embodiment. Therefore, although the edge intensity value at each pixel of the tree or grass is small, the vertical direction sum Sx (x) and the horizontal direction sum Sy (y) exceed the magnification coefficient determination thresholds THx and THy, and the portion of the tree or grass is enlarged. It will be done.

  Therefore, in the present embodiment, the threshold processing unit 204 compares the edge intensity value of each pixel output from the edge extraction unit 201 with a predetermined threshold, and the edge intensity value of a pixel having a smaller edge intensity value. Is replaced with zero before summing the edge intensity values.

  FIG. 8 is an explanatory diagram of a magnification coefficient calculation process in the coefficient calculation unit 200 when the image of FIG. 7A is processed in the first embodiment having the configuration of FIG. In FIG. 8, the image is a result image obtained by the same edge extraction unit 201 as in FIG. The description of the graph is the same as in FIG. 3, FIG. 8A is the vertical sum Sx (x) of the pixel columns at each horizontal position, and FIG. 8B is the horizontal direction of the pixel row at each vertical position. The sum Sy (y), FIG. 8C shows the horizontal magnification coefficient of the pixel at each horizontal position, and FIG. 8D shows the vertical magnification coefficient of the pixel at each vertical position.

  FIG. 9 is an explanatory diagram of a magnification coefficient calculation process in the coefficient calculation unit 220 when the image of FIG. 7A is processed with the configuration of FIG. 6 of the present embodiment to which the threshold processing unit 204 is added. In FIG. 9, the image is a result image obtained by the same threshold processing unit 204 as in FIG. The description of the graph is the same as in FIGS. 3 and 9. FIG. 9A shows the vertical sum Sx (x) of the pixel columns at each horizontal position, and FIG. 9B shows the pixel rows at each vertical position. The total sum Sy (y) in the horizontal direction, FIG. 9C shows the horizontal magnification coefficient of the pixel at each horizontal position, and FIG. 9D shows the vertical magnification coefficient of the pixel in each vertical direction. .

  First, the case where the magnification coefficient is calculated in the first embodiment when the original image is shown in FIG. 7A will be described. In FIG. 7A, the image is focused on the person and the edge intensity value of the background portion is almost small. Therefore, when the edge extraction processing is performed by the edge extraction unit 201, the result as shown in FIG. 7B is obtained. Is obtained.

  Next, when the feature amount calculation unit 202 calculates the sum of the edge strength values in the pixel row and pixel column directions, the edge strength value in each pixel of the background portion is small, but the edge strength value in each pixel is added. The sum is increased as shown in FIGS. 8 (a) and 8 (b). Then, the vertical direction total sum Sx (x) and the horizontal direction total sum Sy (y) exceed the magnification coefficient determination thresholds THx and THy even at a position unrelated to the person. Therefore, as shown in FIGS. 8C and 8D, the number of pixels for which the magnification coefficients are determined to be m and n increases, so that portions other than the human portion that do not need to be enlarged are also enlarged.

  Next, the operation of the threshold processing unit 204 when the image of FIG. 7A is input in this embodiment will be described. The threshold processing unit 204 compares the edge intensity value of each pixel sequentially input with a predetermined threshold value, and holds the edge intensity value of the pixel if the edge intensity value of the corresponding pixel is equal to or greater than the threshold value. If the value is smaller than the threshold value, the value is replaced with zero. As an example, the threshold value is four times the average value of the entire image. In the image of FIG. 7B, since the edge of the background tree or grass portion is weak and the value is small, the edge intensity value of the corresponding pixel is replaced with zero, and around the person as shown in FIG. Only the edge strength value remains.

  Therefore, the sum Sx (x) and Sy (y) of the edge intensity values calculated by the feature amount calculation unit 202 are values in the pixel rows and pixel columns corresponding to the background portion as shown in FIGS. 9A and 9B. As a result of the magnification coefficient determination unit 203, it is determined that only a portion corresponding to a person is enlarged as shown in FIGS. 9C and 9D, and determined to be a magnification coefficient of m or n.

  As described above, in the image processing apparatus according to the present embodiment, by adding the threshold processing unit 204 to eliminate the influence of a small edge intensity value in advance, it is possible to perform more appropriate determination of an important part.

  Note that the threshold value used by the threshold value processing unit 204 of the present embodiment is a value that is four times the average value of the entire image. However, even if an adaptive value is used depending on the image, it does not depend on the image. It may be fixed. Or it is good also as a structure which can be designated directly from the outside.

Embodiment 3 FIG.
FIG. 10 is a block diagram showing the configuration of the image processing apparatus according to Embodiment 3 of the present invention. Components identical with those of the pixel processing apparatus of Embodiment 1 shown in FIG. The difference is that an image size adjustment rate determination unit 205 and an image size adjustment processing unit 500 are added.

  In the image size adjustment rate determination unit 205, the image is adjusted so as to satisfy the specifications of the image size (number of pixels) output from the image output unit 400 from the horizontal and vertical magnification coefficient values calculated by the magnification coefficient determination unit 203. Similarly, a reduction rate (or enlargement rate) for reduction (or enlargement) is calculated.

  The image size adjustment processing unit 500 reduces or enlarges the image data output from the enlargement / reduction processing unit 300 at the magnification determined by the image size adjustment rate determination unit 205.

  Hereinafter, an example of determining the magnification in the image size adjustment rate determination unit 205 will be described with reference to FIG. 11A is an input image to the image input unit, FIG. 11B is an enlarged image that is an output of the enlargement / reduction processing unit 300, and FIGS. 11C and 11D are image size adjustment processing units. 5 is an example of an image obtained as a result of performing a reduction process at 500; Hereinafter, as an example, a case will be described in which the image size adjustment processing unit 500 uniformly reduces the image so that the image size (number of pixels) output from the image output unit 400 is satisfied. Even in this case, the present embodiment can be applied.

  The image in FIG. 11A, which is an input image to the image input unit 100, is the same horizontal W pixel and vertical H pixel image as in FIG. As shown in FIG. 11B, the size of the enlarged image output from the enlargement / reduction processing unit 300 is ZW pixels in the horizontal direction (ZW is larger than W) and ZH pixels in the vertical direction (ZH is larger than H). ). Also, here, as an example, a case where image data output from the image output unit 400 is displayed on a monitor is considered, and the image size output from the image output unit 400 is set so that the entire image is not lost on the monitor display screen. It is assumed that W pixels in the horizontal direction and H pixels in the vertical direction are equal to the input image.

  There are two methods for preventing the entire image of the enlarged image of FIG. 11B from being lost on the monitor display screen. In the first method, the larger image size ZW or ZH of the enlarged image is reduced at the same magnification in both the horizontal and vertical directions of the enlarged image in accordance with the output image size W or H. In the second method, the vertical and horizontal directions are reduced at different magnifications so as to have the same aspect ratio as the input image and the output image. The image size adjustment rate determination unit 205 and the image size adjustment processing unit 500 may take either method.

  First, the first method will be described. In this case, the image size adjustment rate determination unit 205 calculates the same (H / ZH) magnification in each vertical and horizontal direction, and the image size adjustment processing unit 500 reduces the magnification in each vertical and horizontal direction at this magnification. In the enlarged image of FIG. 11B, the horizontal enlargement ratio is ZW / W and the vertical enlargement ratio is ZH / H. For example, if (ZW / W) <(ZH / H). The enlargement ratio is larger in the vertical direction than in the horizontal direction. At this time, if the vertical and horizontal directions of the enlarged image are reduced at the same magnification (H / ZH), the reduced image size is H in the vertical direction and (ZW × H / ZH) in the horizontal direction. From the precondition that (ZW / W) <(ZH / H), W> (ZW × H / ZH), and the image size in the horizontal direction of the reduced image reduced in the image size adjustment processing unit 500 is that of the input image. It is smaller than the image size W, as shown in FIG.

  In FIG. 11 (c), a blank portion without image information (shaded portion in the figure) is generated on the right side of the image. This blank portion may be on the left side or divided on both the left and right sides, and is blank. The number of pixel columns is constant (W− (ZW × H / ZH)).

  Next, the second method will be described. In this case, the image size adjustment rate determination unit 205 calculates different magnifications such as W / ZW in the horizontal direction and H / ZH in the vertical direction, and the image size adjustment processing unit 500 calculates the respective magnifications in the vertical and horizontal directions. Reduce with. As a result, the image size of the reduced image output from the image size adjustment processing unit 500 is W pixels in the horizontal direction and H pixels in the vertical direction, and the image after reduction is as shown in FIG.

  In this way, the image size adjustment rate determination unit 205 according to the present embodiment uses the determination result of the magnification coefficient determination unit 203 and the image size (number of pixels) output from the image output unit 400 to make the image horizontal and vertical. A uniform reduction ratio (enlargement ratio) is determined in each direction. In the determination result of the magnification coefficient determination unit 203, the magnification coefficient is determined for each pixel column and each pixel row of the image, and thus is not uniform for the entire image, but is determined by the image size adjustment rate determination unit 205. The reduction ratio (enlargement ratio) is uniform throughout the image in each of the horizontal and vertical directions of the image.

  As described above, in the image processing apparatus according to the present embodiment, the image size of the output image can be appropriately controlled because it is uniformly reduced or enlarged in accordance with the size of the image output in the image output process.

  Further, by adopting a configuration in which the magnification in each of the horizontal and vertical directions can be changed, the output image can be reduced so as to have the same aspect ratio as the input image. As a result, a partially enlarged image can be displayed with the same image size and aspect ratio as the input image.

  Note that the image processing apparatus according to the present embodiment can be combined with the threshold processing unit 204 of FIG. 6 described in the second embodiment.

Embodiment 4 FIG.
FIG. 12 is a block diagram showing the configuration of the image processing apparatus according to Embodiment 4 of the present invention. The same parts as those of the image processing apparatus shown in FIG. The difference is that a magnification factor adjustment unit 206 is added after the magnification factor determination unit 203 of the unit 240.

  In the example of the first embodiment, the image output from the image output unit 400 is distorted like the mountain in the background in FIG. Therefore, in the present embodiment, when the magnification coefficient of each pixel determined by the magnification coefficient determination unit 203 is suddenly changed in the magnification coefficient adjustment unit 206, the value of the magnification coefficient is changed so that the change becomes gentle. Thereby, distortion of the image output from the image output unit 400 can be reduced.

  FIG. 13 is a diagram for explaining the operation of the magnification coefficient adjustment unit 206. FIG. 13A is a graph of the total sum Sx (x) in the vertical direction with respect to the image obtained by the feature amount calculation unit 202, and FIG. 13B is a horizontal magnification output from the magnification coefficient determination unit 203 in FIG. The coefficient graph, FIG. 13C, is a graph showing the horizontal magnification coefficient changed by the magnification coefficient adjustment unit 206, and in each graph, the horizontal axis indicates the horizontal position.

  FIG. 14 is an explanatory diagram of an input image input to the image input unit 100 and an enlarged image that is an output of the enlargement / reduction processing unit 300 in the present embodiment. FIG. 14A shows an example of an input image input to the image input unit 100 in the present embodiment. 14B, 14C, and 14D are enlarged images that are outputs of the enlargement / reduction processing unit 300 under different conditions, as will be described later. FIG. 14 (e) is a partially enlarged view in which a portion surrounded by a circle in FIG. 14 (d) is enlarged.

  As described in the first embodiment, the magnification coefficient determination unit 203 uses the magnification coefficient determination threshold THx and the vertical total Sx (x) at the horizontal position x as shown in FIG. It is assumed that either 1 or m (a number greater than 1) is selected as the horizontal magnification coefficient at the position x, and a horizontal magnification coefficient as shown in FIG. 13B is obtained.

  As the horizontal magnification coefficient m representing the magnification of the portion that is enlarged in the horizontal direction is set to a larger value, the relative magnification of the enlarged portion with respect to the entire image is increased, and the visibility is improved. However, if the value of m is too large, the change in the image content in the enlarged image becomes steep and the image is distorted.

  The result of the magnification coefficient determination unit 203 varies depending on the values of the magnification coefficient determination thresholds THx and THy. For example, the determination result for the star-shaped graphic image in FIG. 14A is an area determined to be expanded m times in the horizontal direction and n times in the vertical direction when the values of the magnification coefficient determination thresholds THx and THy are different. Hereinafter, in this embodiment, the region is referred to as an enlargement target region), which is different like a region 701 surrounded by a dotted line and a region 702 surrounded by a dotted line. Hereinafter, this example will be described in detail.

  In any case, as described with reference to FIG. 5 in the first embodiment, the image is divided into nine regions corresponding to the magnification coefficient, and is surrounded by the region 701 or the dotted line that is the central region. In the area 702, the horizontal direction is enlarged by m times and the vertical direction is enlarged by n times, and the corresponding four areas of the upper left side, the lower left side, the upper right side, and the lower right side remain the same in both the horizontal direction and the vertical direction. In the upper center and lower center areas, the horizontal magnification is m times, the vertical direction is the same magnification, the left middle and right middle areas are the horizontal magnification, and the vertical direction is n times magnification. Become.

  When the image of FIG. 14A is input, the magnification coefficient determination unit 203 assigns a value larger than 1 only to the periphery of the central star-shaped figure, but the region is determined by the values of the magnification coefficient determination thresholds THx and THy. The shape of can change and may not be a desirable decision. In the image of FIG. 14A, the region 701 is determined as the enlargement target region if the magnification factor determination threshold is appropriate, and the region 702 is determined as the magnification factor determination threshold is too large. In either case, the magnification coefficient is a rectangular graph as shown in FIG. 13B, but the width of the portion where the magnification coefficient is determined to be m is different, and the area 701 is the enlargement target area. Is wider.

  When the enlargement target area of the magnification coefficient determination unit 203 is the area 701, the enlargement / reduction processing unit 300 enlarges the entire star figure and obtains an enlarged image as shown in FIG. 14B.

  When the enlargement target area of the magnification coefficient determination unit 203 is the area 702, the end of the star figure is no longer the enlargement target, and the enlargement / reduction processing unit 300 enlarges only the vicinity of the center of the star figure. ) Is a star-shaped figure indicated by a solid line. In FIG. 14C, at the end of the star-shaped figure indicated by the solid line, the figure is distorted due to the change of the magnification factor. For comparison, a star-shaped figure without distortion is shown by a dotted line. A region 703 surrounded by a square dotted line indicates a region corresponding to the region 702 of the image before enlargement shown in FIG.

  On the other hand, in the present embodiment, a magnification coefficient adjustment unit 206 is added, and the magnification coefficient is changed around a pixel where the change in the magnification coefficient is abrupt as shown in FIG. In FIG. 13C showing the changed magnification factor, the magnification factor is changed in a linear function. As a result, the star figure output from the enlargement / reduction processing unit 300 becomes a star figure as shown by the solid line in FIG. In this figure, the magnification coefficient is similarly changed not only in the horizontal direction but also in the vertical direction. For comparison, a star-shaped figure indicated by a dotted line is a star-shaped figure without distortion. An area 704 surrounded by a square dotted line indicates an area corresponding to the area 702 of the image before enlargement shown in FIG. A region 705 surrounded by a square solid line is a region in which the magnification factor in the horizontal direction or the vertical direction is changed in the magnification factor adjustment unit 206.

  FIG. 14E is a partially enlarged view in which a portion surrounded by a circle in FIG. 14D is enlarged. In FIG. 14 (e), a part of the star figure when the magnification coefficient is adjusted by the magnification coefficient adjustment unit 206 is indicated by a solid line 707, and a part of the star figure when not adjusted is indicated by a broken line 706. As shown in FIG. 14E, when the magnification coefficient is changed by the magnification coefficient adjustment unit 206, the distortion of the star figure can be reduced as compared with the case where the magnification coefficient is not changed.

  As described above, in the present embodiment, when the change of the magnification coefficient is abrupt by the magnification coefficient adjustment unit 206, the magnification coefficient is changed so that the change becomes gentle. Even in this case, the distortion to be enlarged can be reduced.

  In the above example, as shown in FIG. 13 (b), an extreme example in which the magnification coefficient has only two values of 1 and m is handled. When the magnification coefficient takes three or more values, an appropriate predetermined value is set in advance, and the rate of change of the magnification coefficient with respect to the change in the position of the pixel is equal to or lower than the original change rate. What is necessary is just to change a magnification factor so that it may become.

  Further, in FIG. 13C, the slope is simply changed in a linear function, but it may be a curved change.

  Further, the image processing apparatus according to the present embodiment can be combined with the threshold processing unit 204 of FIG. 6 described in the second embodiment. Further, it can be combined with the image size adjustment rate determination unit 205 and the image size adjustment processing unit 500 of FIG. 10 described in the third embodiment.

Embodiment 5 FIG.
FIG. 15 is a block diagram showing the configuration of the image processing apparatus according to the fifth embodiment of the present invention. The same parts as those of the image processing apparatus shown in FIG. The difference is that a block dividing unit 207 is added after the unit 201.

  The block dividing unit 207 regards the input edge strength value of each pixel as one block with some pixels in each horizontal and vertical direction, and represents the block from the edge strength value of each pixel included in the block. Determine the edge strength value. This value is called a block edge strength value. As a method of determining the edge intensity value of a block, characteristic values such as an average value, maximum value, minimum value, and intermediate value of edge intensity values of pixels included in the block are used. Alternatively, there is a method of setting an edge intensity value of a pixel arranged at a specific position in a block.

  In the calculation and processing after the block dividing unit 207, the edge strength value of the block is used instead of the edge strength value of each pixel, and the calculation and processing performed for each pixel are performed for each block. Thereby, the number of data used for calculation and processing after the feature amount calculation unit 202 is reduced from the total number of pixels of the input image to the number of blocks divided by the block dividing unit 207.

  Specifically, the calculation of the sum of edge strength values performed by the feature amount calculation unit 202 described in the first embodiment, the threshold processing performed by the threshold processing unit 204 described in the second embodiment, and the first embodiment described above. Determination of the magnification coefficient of the magnification coefficient determination unit 203, enlargement / reduction processing performed by the enlargement / reduction processing unit 300 described in the first embodiment, change of the magnification coefficient by the magnification coefficient adjustment unit 206 described in the fourth embodiment, The determination of the reduction or enlargement magnification performed by the image size adjustment rate determination unit 205 described in the third embodiment and the uniform reduction or enlargement processing performed by the image size adjustment processing unit 500 described in the third embodiment are performed in pixel units. Rather than in blocks. When it is necessary to use a value (for example, each RGB data) other than the edge intensity value set for the pixel in the above calculation and processing, the block representative value is used in the same manner as the edge intensity value. As a method for determining the representative value of a block, characteristic values such as an average value of pixel values included in the block, a maximum value, a minimum value, and an intermediate value are used. Alternatively, there is a method of setting the value of a pixel arranged at a specific position in the block.

  As described above, in the present embodiment, by performing the calculation and processing performed in units of pixels in units of blocks, it is possible to reduce the processing time required for the coefficient calculation unit or the circuit scale required for calculation.

Embodiment 6 FIG.
FIG. 16 is a block diagram showing the configuration of the image processing apparatus according to Embodiment 6 of the present invention. The same parts as those of the image processing apparatus shown in FIG. The difference is that the detection unit 600 and the edge changing unit 208 are added, and the point that the image input unit 110 outputs the input image data to the moving body detection unit 600.

In the present embodiment, when a plurality of temporally continuous images are input, a moving body part in the image is detected, and only the area of the moving body is preferentially secured to ensure visibility. Objective.

  The moving object detection unit 600 acquires a plurality of temporally continuous images from the image input unit 110, detects an object that moves between image frames, and detects the area, shape, and position of a region corresponding to the moving object. Etc. There is no limitation on the number of frames of an image used for detection, and a general method such as pattern matching or motion vector detection is used as the detection method.

  The edge changing unit 208 changes the edge intensity value output by the edge extracting unit 201 by comparing with the information on the area detected by the moving body detecting unit 600 as a moving body part.

  Next, an example of the operation of this embodiment will be described with reference to FIGS. 17A and 17B are examples of two consecutive images input to the image input unit 110. FIG. Further, (c) is a diagram showing a region where the edge intensity value is large in (b), and (d) is a diagram showing a detection result of the moving body portion in the moving body detection unit 600 of (b). In (c), areas with large edge intensity values are indicated as 801 and 802, and in (d), an area 803 detected as a part corresponding to a moving object in the moving object detection unit 600 is surrounded by a dotted line. (E) is an enlarged image output from the enlargement / reduction processing unit 300.

  FIG. 18 is an explanatory diagram of a positional relationship when the person, the car, and the camera 900 at the time of photographing shown in FIGS. 17A and 17B are viewed from above. As shown in this figure, the car is moving so as to approach the camera, and from the viewpoint of the camera 900, the car is moving from the back toward the front. The image of FIG. 17A is an image taken by the camera 900 when the car is at position C in FIG. 18 and the image of FIG. 17B is the image when the car is at position D in FIG.

  First, it is assumed that a plurality of images (a) and (b) that are continuous in time are input to the image input unit 110. Following FIG. 17A, FIG. 17B is input to the image input unit 110. The image input unit 110 outputs luminance signal data to the edge extraction unit 201 and image data to the enlargement / reduction processing unit 300, and also outputs image data to the moving object detection unit 600.

  According to the operation described in the first embodiment, in the image of FIG. 17B, both the car portion 801 and the person portion 802 are portions where the edge intensity value is larger than the background. And 802 are determined to be important parts.

  On the other hand, comparing FIGS. 17A and 17B, since the person is not moving but the car is moving toward the front, the moving object detection unit 600 moves the portion 803 in FIG. 17D. Detected to be a body.

  The edge changing unit 208 compares the output data of the edge extracting unit 201 with the area, shape, position, etc. of the moving body portion that is the detection result of the moving body detecting unit 600, and the edge strength is compared with FIG. Only the edge strength value of the region 801 where the moving object detection region 803 matches the portion whose value is equal to or greater than the predetermined value is left as it is, and the edge strength value of the other portion is changed to zero.

  As a result, the important part is only the car part, and the enlargement / reduction processing unit 300 performs the enlargement / reduction process only on the part corresponding to the car, and only the car is enlarged as shown in FIG. Get an image that remains doubled.

  As described above, in this embodiment, when a plurality of temporally continuous images are input, a moving body in the image can be detected, and visibility can be preferentially secured with only the area of the moving body as an important part.

  In the present embodiment, the operation of preferentially securing the visibility with only the moving object being described has been described, but conversely, only a stationary object may be enlarged.

DESCRIPTION OF SYMBOLS 100 Image input part 200 Coefficient calculation part 201 Edge extraction part 202 Feature-value calculation part 203 Magnification coefficient determination part 204 Threshold processing part 205 Image size adjustment rate determination part 206 Magnification coefficient adjustment part 207 Block division part 208 Magnification coefficient change part 300 Enlargement / Reduction processing unit 400 Image output unit 500 Image size adjustment processing unit 600 Moving object detection unit 900 Camera

Claims (14)

An image input unit for inputting an image;
A coefficient calculation unit that divides the image into a plurality of parts based on the edge intensity value of each pixel of the image input to the image input unit, and calculates and outputs a magnification coefficient indicating a magnification to be enlarged or reduced for each part. When,
An enlargement / reduction processing unit for enlarging or reducing the image input to the image input unit for each part based on a magnification coefficient calculated by the coefficient calculation unit;
An image processing apparatus comprising: an image output unit that outputs an image enlarged or reduced by the enlargement / reduction processing unit. The coefficient calculator
An edge extraction unit that extracts and outputs an edge intensity value of each pixel of the image input to the image input unit;
A feature amount calculating unit that calculates the sum of the edge intensity values of the pixels in each row and the sum of the edge intensity values of each column from the edge intensity values of each pixel of the image;
A magnification coefficient determination that divides the image into a plurality of parts based on the sum of the edge intensity values of the pixels in each row and the sum of the edge intensity values of each column calculated by the feature amount calculation unit, and determines a magnification coefficient for each of these parts The image processing apparatus according to claim 1, further comprising: an image processing unit.   The edge intensity value of each pixel output by the edge extraction unit is compared with a predetermined threshold value, and for the pixel whose edge intensity value is smaller than the predetermined threshold value, the edge intensity value is replaced with zero and output to the feature amount calculation unit The image processing apparatus according to claim 2, further comprising a threshold processing unit that performs the processing.   The image size adjustment processing unit further reduces or enlarges the image enlarged or reduced by the enlargement / reduction processing unit according to the size of the image output by the image output unit. The image processing apparatus according to 2 or 3.   For the magnification factor determined by the magnification factor determination unit, when the rate of change of the magnification factor with respect to the change in pixel position is greater than or equal to a predetermined value, the magnification factor is changed so that the rate of change is less than or equal to the rate of change when the rate is greater than or equal to the predetermined value. The image processing apparatus according to claim 2, further comprising a magnification coefficient adjustment unit. A moving body detection unit that detects a moving body in an image when a plurality of temporally continuous images are input;
The image processing apparatus according to claim 2, wherein a part to be enlarged or reduced by the enlargement / reduction processing unit is changed based on a detection result of the moving body detection unit. An image input unit for inputting an image;
A block dividing unit that divides an image input to the image input unit into blocks each including a plurality of pixels;
A coefficient calculation unit that divides the image into a plurality of parts based on the edge strength value of each block of the image divided by the block division unit, and calculates and outputs a magnification coefficient indicating a magnification to be enlarged or reduced for each part. When,
An enlargement / reduction processing unit for enlarging or reducing the image input to the image input unit for each part based on a magnification coefficient calculated by the coefficient calculation unit;
An image processing apparatus comprising: an image output unit that outputs an image enlarged or reduced by the enlargement / reduction processing unit. An image input process in which an image is input;
A coefficient calculation step of dividing the image into a plurality of parts based on the edge intensity value of each pixel of the image input in the image input step, and calculating and outputting a magnification coefficient indicating a magnification to be enlarged or reduced for each of the parts. When,
An enlargement / reduction processing step for enlarging or reducing the image input in the image input step for each portion based on the magnification coefficient calculated in the coefficient calculation step;
An image output step of outputting the image enlarged or reduced in the enlargement / reduction processing step. The coefficient calculation process
An edge extraction step of extracting and outputting the edge intensity value of each pixel of the image input in the image input step;
A feature amount calculating step of calculating the sum of the edge strength values of the pixels in each row and the sum of the edge strength values of each column from the edge strength values of each pixel of the image;
A magnification coefficient determination that divides the image into a plurality of parts based on the sum of the edge intensity values of the pixels in each row and the sum of the edge intensity values of each column calculated in the feature amount calculation step, and determines a magnification coefficient for each of these parts The image processing method according to claim 8, further comprising a step.   The edge intensity value of each pixel output in the edge extraction process is compared with a predetermined threshold value, and for the pixel whose edge intensity value is smaller than the predetermined threshold value, the edge intensity value is replaced with zero and output to the feature amount calculation unit The image processing method according to claim 9, further comprising: a threshold processing step.   10. The image size adjustment processing step of further reducing or enlarging the image enlarged or reduced in the enlargement / reduction processing step in accordance with the size of the image output in the image output step. Or the image processing method of 10.   For the magnification factor determined in the magnification factor determination step, when the rate of change of the magnification factor relative to the pixel position change is equal to or greater than a predetermined value, the magnification factor is changed to be equal to or less than the rate of change when the rate of change is equal to or greater than the predetermined value. The image processing method according to claim 9, further comprising a magnification coefficient adjustment step. A moving body detecting step of detecting a moving body in the image when a plurality of temporally continuous images are input;
The image processing method according to claim 9, wherein a portion to be enlarged or reduced by the enlargement / reduction processing unit is changed based on a detection result in the moving object detection step. An image input process in which an image is input;
A block dividing step of dividing the image input in the image input step into blocks composed of a plurality of pixels;
A coefficient calculation step of dividing the image into a plurality of parts based on the edge strength value of each block of the image divided in the block division step, and calculating and outputting a magnification coefficient indicating a magnification to be enlarged or reduced for each of the parts. When,
An enlargement / reduction processing step for enlarging or reducing the image input in the image input step for each portion based on the magnification coefficient calculated in the coefficient calculation step;
An image output step of outputting the image enlarged or reduced in the enlargement / reduction processing step.

Images