JP6118623B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and program for cutting out and outputting a low resolution image having a lower resolution than that of the high resolution image from the high resolution image.

  Conventionally, various image processing devices that cut out a part of an image from a high-resolution image (image or video with a large number of pixels) and output it as a low-resolution image (an image or video with a smaller number of pixels than a high-resolution image) have been proposed. Has been.

  For example, an apparatus that cuts out a low-resolution image with a focus position of a photographed high-resolution image as a center is known (see, for example, Patent Document 1). There is also known an apparatus that captures a high-resolution image and encodes and transmits a low-resolution screen cut out from the video by a photographer (for example, see Patent Document 2). In addition, an apparatus is known that cuts out a part of a Super Hi-Vision screen and then reduces or enlarges it to convert it to an HD image quality (for example, see Non-Patent Document 1).

Japanese Patent No. 5003803 JP 2004-336805 A

“High-performance“ small SHV down converter ”developed for the first time! -Dramatically improved SHV "One Resource Multi-use" program production- ", [online], April 17, 2009, NHK, [March 28, 2013 search], Internet <URL: http : //www.nhk.or.jp/pr/marukaji/pdf_ver/231.pdf>

  However, the conventional image processing apparatus has not cut out a low-resolution image at a position suitable for encoding in consideration of encoding. Therefore, there is room for reducing deterioration due to encoding of the cut out low resolution image.

  Accordingly, an object of the present invention made in view of such a point is to provide an image processing apparatus and a program capable of cutting out a low resolution image with little coding deterioration from a high resolution image.

  In order to solve the above problems, an image processing apparatus according to the present invention is an image processing apparatus that cuts out a low-resolution image having a lower resolution than a high-resolution image from a high-resolution image. A cut-out area search range including a cut-out area is set in advance, and an edge detection unit that obtains an edge strength within the cut-out area search range, and a boundary position of a coding block when coding the temporary cut-out area And moving a temporary cutout area having the same size as the temporary cutout area and having the same encoding block boundary position within the cutout area search range, and a boundary of the encoded block of the temporary cutout area Based on the position and the edge strength, the optimal output region search unit for obtaining the optimal output region, and the low resolution image output that cuts out the optimal output region from the high resolution image and outputs it as a low resolution image. Characterized in that it comprises a part, a.

  In the image processing apparatus according to the present invention, the most appropriate extracted area search unit calculates an integrated edge intensity obtained by adding edge intensity at a boundary position of the encoded block in the temporary cut area, and the integrated edge intensity The region where the maximum value is the largest is set as the most appropriate output region.

Further, in the image processing apparatus according to the present invention, the most appropriate outgoing region search unit adds a weight corresponding to the size of the coding block for each coding block when calculating the integrated edge strength or It is characterized by multiplication .

Further, in the image processing apparatus according to the present invention, the most appropriate extracted area search unit calculates an integrated non-edge intensity obtained by adding edge intensity at a position other than a boundary position of the encoded block of the temporary cut area, A region where the integrated non-edge intensity is minimum is set as the most appropriate output region.

In the image processing apparatus according to the present invention, the most appropriate extracted area search unit includes an integrated edge strength obtained by adding edge strengths of boundary positions of the encoded blocks in the temporary cutout area, and the temporary cutout area. An integrated edge / non-edge intensity that is a linear sum of the integrated non-edge intensities obtained by adding the edge intensities at positions other than the boundary position of the coding block is calculated, and an area where the integrated edge / non-edge intensity is maximum is calculated as the maximum. It is characterized by an appropriate exit area .

Further, in the image processing apparatus according to the present invention, the most appropriate outgoing area search unit calculates the integrated edge / non-edge intensity by an integrated edge intensity−k × an integrated non-edge intensity, where k is an integrated edge intensity. The number of pixels used in the calculation of ÷ the number of pixels used in the calculation of the integrated non-edge intensity .

  Further, in the image processing apparatus according to the present invention, the high-resolution image or a part thereof is presented to the user to obtain information on the provisional cut-out area designated by the user, and from the most appropriate outgoing area search unit A cutout that obtains information on the most suitable cutout area, presents the optimum cutout area to the user, and causes the user to designate either the provisional cutout area or the optimum cutout area as the final cutout area. An area setting interface unit is further provided, and the low-resolution image output unit cuts out the final cut-out area from the high-resolution image and outputs it as a low-resolution image.

  Further, in the image processing apparatus according to the present invention, when the user moves the provisional cutout area in order to designate a final cutout area, the cutout area setting interface unit positions the optimum cutout area. It is characterized by giving a feeling of adsorption.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the image processing apparatus.

  According to the present invention, it is possible to output a low-resolution image with little coding deterioration by adjusting the position of the boundary of the encoded block of the low-resolution image to be cut out to match the edge position of the image. In particular, it is possible to reduce coding deterioration (mosquito noise) of visually conspicuous edge portions.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a figure which shows each area | region used by the image processing apparatus which concerns on this invention. It is a figure explaining the case where the image processing apparatus which concerns on this invention is applied to a moving image. It is a block diagram which shows the structure of the image processing apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the boundary position of the encoding block used by the image processing apparatus which concerns on this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to the first embodiment of the present invention. In the example illustrated in FIG. 1, the image processing apparatus 1 includes a high-resolution image input unit 11, an edge detection unit 12, a most appropriate output area search unit 13, and a low-resolution image output unit 14.

  The image processing apparatus 1 detects an edge position in the high-resolution image, and controls the cut-out position of the low-resolution image so that the ratio that the edge position matches the encoded block boundary of the low-resolution image becomes high.

  The high resolution image input unit 11 outputs a high resolution image input from the outside to the edge detection unit 12 and the low resolution image output unit 14.

  Information indicating the temporary cut-out area is input to the most appropriate extraction area search unit 13. Here, the provisional cutout area is a provisional cutout area of a low resolution image set by a user who uses the image processing apparatus 1.

  In addition, information indicating the cut-out area search range is input to the edge detection unit 12 and the most appropriate extraction area search unit 13. Here, the cut-out area search range is a range that includes the temporary cut-out area, and is an area in which an optimum cut-out area is searched in consideration of encoding. The cut area search range may be set in advance by the user, or may be automatically set by the image processing apparatus 1 from the temporary cut area.

  FIG. 2 is a diagram illustrating each area used by the image processing apparatus 1. A temporary cutout region 21 is set in the high resolution image 20 input to the image processing apparatus 1 by the user. The cut-out area search range 22 is set as an area obtained by enlarging the provisional cut-out area 21 vertically and horizontally by the enlargement widths 23 and 24 in order to detect the edge of the image. The temporary cut-out area 26 is an area having the same size as the temporary cut-out area 21 and the boundary position of the same encoded block. As will be described later, the optimal extraction area search unit 13 moves the temporary extraction area 26 within the extraction area search range 22 to determine the optimal extraction area 25 having the same size as the temporary extraction area 21. .

  When the cutout area search range 22 is set wide, a cutout area with improved encoding image quality can be obtained, but the change from the composition of the provisional cutout area initially set by the user becomes large. On the other hand, if the cut-out area search range 22 is set narrow, a low-resolution image closer to the composition intended by the user can be obtained, but there is a high possibility that the image quality after encoding and decoding will not be optimal. In order to obtain an optimal low-resolution image that is close to the user's intention and has little coding deterioration, it is preferable that the expansion widths 23 and 24 be half the size of the coding block size. For example, when the encoding block size is 16 × 16 pixels, the expansion widths 23 and 24 are each 8 pixels.

  The edge detection unit 12 obtains the edge strength for each pixel within the cutout region search range 22 of the high resolution image 20 and outputs the edge strength to the most appropriate output region search unit 13. Any method may be used for edge detection as long as the edge strength for each pixel is obtained, such as a Sobel filter or a Canny filter. The edge strength may be binary (the presence or absence of an edge) or multi-value (a value indicating a stronger edge as the value increases or decreases). In this specification, for the sake of explanation, it is assumed that a multi-value edge strength that indicates a stronger edge as the value increases is used. Such multi-value edge strength can be obtained by a Sobel filter or the like. Note that edge detection need not be performed for all pixels of the high-resolution image.

  The most appropriate extraction area search unit 13 determines the boundary position of the encoding block when encoding the provisional extraction area 21. The boundary position of the coding block is determined according to the coding method. Then, within the cutout area search range 22, the temporary cutout area 26 having the same size as the temporary cutout area 21 and having the same coding block boundary position is moved vertically and horizontally, Based on the boundary position of the coded block and the edge strength obtained by the edge detection unit 12, the most appropriate output region 25 that is predicted to have little coding deterioration is determined, and the information of the optimal output region 25 is output as a low-resolution image. To the unit 14. Three specific methods will be described below as a specific method for determining the most appropriate out area 25.

  In the first optimum extraction area search method, the accumulated edge is obtained by sequentially shifting the temporary extraction area 26 within the extraction area search range 22 and adding the edge strength of the boundary position of the encoded block of the temporary extraction area 26. Calculate the intensity. Then, the provisional cutout area 26 having the maximum accumulated edge strength is set as the most appropriate extraction area 25.

  In the second most appropriate extraction area search method, the temporary cutout area 26 is sequentially shifted within the cutout area search range 22, and the edge strengths at positions other than the boundary position of the encoded block of the temporary cutout area 26 are added. Calculate the integrated non-edge strength. Then, the provisional cutout area 26 in which the integrated non-edge intensity is minimum is set as the most appropriate extraction area 25.

  In the third most suitable extraction area search method, the integrated edge strength obtained by sequentially shifting the temporary cutout area 26 within the cutout area search range 22 and adding the edge strengths of the boundary positions of the encoded blocks of the temporary cutout area 26 is added. And the integrated non-edge strength obtained by adding the edge strengths at positions other than the boundary position of the encoded block in the provisional cutout region 26 is calculated. Then, the integrated edge / non-edge intensity is calculated as a linear sum of the integrated edge intensity and the integrated non-edge intensity, and the provisional cutout region 26 having the maximum integrated edge / non-edge intensity is set as the most appropriate extraction area 25. For example, integrated edge / non-edge intensity = integrated edge intensity−k × integrated non-edge intensity. Here, the value of the coefficient k can be set freely, but it is preferable to set the number of pixels used for calculation of the integrated edge strength divided by the number of pixels used for calculation of the integrated non-edge strength, or a value close thereto.

  MPEG-4 AVC / H. H.264, MPEG-H HEVC / H. In an encoding method such as the H.265 method, there are a plurality of sizes of encoded blocks. In such an encoding method, the amount of information for dividing the encoded block changes according to the size of the encoded block. When the boundary of the largest coding block (macroblock) and the edge position of the image overlap, it is not necessary to divide the coding block. For this reason, by adjusting the boundary of the large-sized coding block and the edge position of the image as much as possible, the amount of information indicating division can be reduced, and the coding efficiency is improved.

  Therefore, when encoding using an encoding method in which a plurality of encoding block sizes exist, the most appropriate extraction area search unit 13 uses the boundary position of the encoding block when encoding the provisional cutout area 21. The boundary position that can be divided is included. Note that whether or not to actually divide the encoded block is determined by an encoding device (not shown) connected to the subsequent stage of the image processing device 1. When obtaining the integrated edge, the most suitable outgoing area search unit 13 preferably multiplies or adds the weight corresponding to the coding block size to the edge strength that overlaps the boundary of the coding block. In particular, it is preferable to set a large weight at the boundary of a large-sized coding block and a small weight at the boundary of a small-sized coding block. For example, when the size of a macroblock is 16 × 16 pixels and this can be divided into an encoded block of 8 × 8 pixels or an encoded block of 4 × 4 pixels, as shown in FIG. The weights are decreased in the order of the boundary a of the macroblock indicated by the solid line, the boundary b of the 8 × 8 pixel size indicated by the one-dot chain line, and the boundary c of the 4 × 4 pixel size indicated by the dotted line. .

  The low resolution image output unit 14 cuts out the most appropriate output area determined by the most appropriate output area search unit 13 from the high resolution image, and outputs it as a low resolution image.

  Next, a case where the present invention is applied to a moving image will be described. In this case, the user may designate the provisional cutout area 21 for each frame, designate the provisional cutout area 21 only for the first frame and the last frame, and search for the most suitable cutout area for each of the other frames. The unit 13 may determine the provisional cutout region 21 by interpolation.

  FIG. 3 is a diagram illustrating a case where the image processing apparatus according to the present invention is applied to a moving image. In the case of a moving image, as shown in FIG. 3A, the provisional cut-out area 21 designated by the user may change with time (frame). In such a case, if the most appropriate output area is changed for each frame, there arises a problem that the cut out low resolution image appears to vibrate.

  In order to prevent this, with respect to the temporary cutout area 21 that moves across a plurality of frames, as shown in FIG. 3B, the temporary cutout area 26 that is displaced from the temporary cutout area 21 by the same amount. The above-described integrated edge strength is calculated, the integrated edge strength is added for a plurality of frames, and the region where the displacement having the maximum value is added to the provisional cutout region 21 of each frame is set as the most appropriate extraction region 25. The range of displacement is within the cutout area search range 22. Further, instead of the integrated edge strength, the above-described integrated non-edge strength or the integrated edge / non-edge strength may be used.

  As described above, the image processing apparatus 1 uses the edge detection unit 12 to obtain the edge strength within the cut-out area search range 22. Next, when the optimum extraction area search unit 13 determines the boundary position of the encoding block when encoding the provisional extraction area 21, and moves the temporary extraction area 26 within the extraction area search range 22. The most appropriate output area 25 is obtained based on the boundary position and edge strength of the encoded block. The low-resolution image output unit 14 cuts out the most appropriate output area 25 from the high-resolution image 20 and outputs it as a low-resolution image. For this reason, according to the image processing apparatus 1 of the first embodiment, the position of the boundary of the coding block of the low-resolution image can be adjusted to match the edge position of the image. Can be output. In particular, it is possible to reduce coding deterioration (mosquito noise) of visually conspicuous edge portions.

  In addition, when calculating the integrated edge strength, it is preferable that the most appropriate outgoing region search unit 13 adds or multiplies a weight corresponding to the size of the encoded block for each encoded block. In particular, it is preferable to set a large weight for a boundary of a large block size and a small weight for a boundary of a small block size. Thereby, in an encoding method in which a plurality of encoded block sizes exist, the amount of information indicating the division of the encoded block can be reduced, and the encoding efficiency can be improved.

(Second Embodiment)
Next, an image processing apparatus according to the second embodiment of the present invention will be described. FIG. 4 is a block diagram illustrating a configuration of an image processing apparatus according to the second embodiment. In the example illustrated in FIG. 4, the image processing apparatus 2 includes a high-resolution image input unit 11, an edge detection unit 12, a most appropriate extraction region search unit 13, a low-resolution image output unit 14, and a cut-out region setting interface unit. 15. The image processing apparatus 2 of the second embodiment is different from the image processing apparatus 1 of the first embodiment shown in FIG. 1 in that it further includes a cutout area setting interface unit 15. Note that the same components as those in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  The image processing apparatus 2 sets a cutout region by interactive operation with the user.

  The cut-out area setting interface unit 15 inputs a high resolution image from the high resolution image input unit 11, displays the high resolution image or a part thereof on a display unit (not shown), and presents it to the user. The user designates the provisional cutout area 21 from the presented high resolution image. The cut-out area setting interface unit 15 acquires information on the temporary cut-out area 21 designated by the user.

  When the most appropriate extraction area search unit 13 determines the most appropriate extraction area 25 as in the first embodiment described above, it outputs the determined optimal extraction area to the cut-out area setting interface unit 15. The cut-out area setting interface unit 15 presents both the provisional cut-out area specified by the user first and the most suitable cut-out area determined by the most suitable cut-out area search unit 13 on the display unit, and Let the user specify as the final cutout area. Then, the cutout region setting interface unit 15 outputs information on the final cutout region designated by the user to the low resolution image output unit 14.

  The low-resolution image output unit 14 cuts out a final cut-out area designated by the cut-out area setting interface unit 15 from the high-resolution image and outputs it as a low-resolution image.

  When the user moves the provisional cutout area 21 in order to designate a final cutout area using an input device such as a touch panel, a mouse, or a joystick, the cutout area setting interface unit 15 uses the optimum cutout area. It is preferable to give a feeling of adsorption at the position. When a feeling of adsorption is given to the most appropriate exit area 25, it is possible to easily designate the most appropriate exit area 25 without disturbing the user's free area setting. The feeling of adsorption means that the movement of the area is temporarily stopped at the position of the optimum extraction area 25 while the user moves the position of the temporary cutout area 21 using the input device, This means that when the vicinity of the area 25 is specified, the area 25 is forcibly moved to the most appropriate outgoing area 25.

  As described above, the image processing apparatus 2 presents a high-resolution image or a part thereof to the user, acquires information on the provisional cut-out area 21 designated by the user, Cutout that obtains information on the most suitable cutout area 25, presents the optimum cutout area 25 to the user, and causes the user to designate either the provisional cutout area 21 or the optimum cutout area 25 as the final cutout area An area setting interface unit 15 is provided. For this reason, according to the image processing apparatus 2 of the second embodiment, in addition to the effects of the image processing apparatus 1 of the first embodiment, the user confirms the most appropriate extraction area 25 and then the provisional extraction area. 21 or the most appropriate output area 25 can be selected.

  Further, the cut-out area setting interface unit 15 can give the user a feeling of adsorption in the most appropriate extraction area 25, thereby allowing the user to easily specify the optimal extraction area 25.

  In addition, a computer can be suitably used to cause the image processing apparatuses 1 and 2 to function as described above, and such a computer can store a program describing processing contents for realizing the functions of the image processing apparatuses 1 and 2. It can be realized by storing the program in a storage unit of a computer and reading and executing the program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments can be combined into one, or one constituent block can be divided.

  The present invention is useful for applications that cut out a low-resolution image (for example, high-definition) from a high-resolution image (for example, super high-definition or 4K).

DESCRIPTION OF SYMBOLS 1, 2 Image processing apparatus 11 High resolution image input part 12 Edge detection part 13 Optimal extraction area search part 14 Low resolution image output part 15 Cutout area setting interface part 20 High resolution image 21 Temporary cutout area 22 Cutout area search Range 23, 24 Enlarged width 25 Optimum extraction area 26 Temporary extraction area

Claims (9)

An image processing apparatus that cuts out a low-resolution image having a lower resolution than a high-resolution image from a high-resolution image,
A provisional cutout area and a cutout area search range including the provisional cutout area are set in advance,
An edge detection unit for obtaining edge strength within the cut-out area search range;
A boundary position of a coding block for encoding the provisional cutout area is determined, and a provisional cut having the same size as the provisional cutout area and the same coding block boundary position within the cutout area search range. An optimal region extraction unit for determining the optimal region based on the boundary position of the encoded block of the provisional region and the edge strength;
A low-resolution image output unit that cuts out the most appropriate output region from the high-resolution image and outputs it as a low-resolution image;
An image processing apparatus comprising:   The most appropriate extraction region search unit calculates an integrated edge strength obtained by adding edge strengths at the boundary positions of the coding blocks of the provisional cutout region, and determines a region where the integrated edge strength is the largest as the optimal output region The image processing apparatus according to claim 1, wherein: The most appropriate exit area searching unit, when calculating the cumulative edge strength, for each of the coding blocks, and wherein the adding or multiplying the weight corresponding to the size of the encoded block, according to claim 2 the image processing apparatus according to.   The most appropriate extraction area search unit calculates an integrated non-edge intensity obtained by adding edge intensity at positions other than the boundary position of the encoded block in the provisional cut-out area, and determines an area where the integrated non-edge intensity is minimum. The image processing apparatus according to claim 1, wherein the optimal output area is used.   The most appropriate extraction area search unit is configured to add an integrated edge strength obtained by adding edge strengths of boundary positions of the encoded blocks in the temporary cut-out area, and positions other than the boundary positions of the encoded blocks in the temporary cut-out area. Calculate the integrated edge / non-edge intensity that is a linear sum of the integrated non-edge intensity obtained by adding the edge intensity, and make the area where the integrated edge / non-edge intensity is the maximum suitable output area, The image processing apparatus according to claim 1. The optimum output area search unit calculates the integrated edge / non-edge intensity by the integrated edge intensity−k × the integrated non-edge intensity, where k is the number of pixels used for calculating the integrated edge intensity / the integrated non-edge. The image processing apparatus according to claim 5 , wherein the number of pixels is used for intensity calculation. Presenting the high-resolution image or a part thereof to the user to obtain information on the provisional cut-out area designated by the user, and obtaining information on the most suitable out-area from the most suitable out-area search unit, A cutting area setting interface unit that presents the most appropriate extraction area to the user and allows the user to specify either the provisional cutting area or the most appropriate extraction area as a final cutting area;
The image processing according to any one of claims 1 to 6, wherein the low-resolution image output unit cuts out the final cut-out region from the high-resolution image and outputs the cut-out region as a low-resolution image. apparatus.   The cut-out area setting interface unit is characterized in that when the user moves the temporary cut-out area in order to specify a final cut-out area, it gives a feeling of adsorption at the position of the most appropriate cut-out area. The image processing apparatus according to claim 7.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 8.

Images