JP2013196009A - Image processing apparatus, image forming process, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image forming apparatus in which while keeping visual information related to a feature of an original image which is visually recognized by portions other than an attention area of the original image, a thumbnail image in which the attention area is highlighted is generated.SOLUTION: According to the embodiment, an image forming apparatus includes a first area processing part, a second area processing part, a deformation information calculating part, and a deformation processing part. The first area processing part obtains, from a target image, a first area which is subjected to enlarging deformation. The second area processing part obtains a second area which is subjected to reducing deformation in accordance with the enlarging deformation of the first area or an area including the first area. The deformation information calculation part obtains deformation information indicative of a method of deforming the first area or the area including the first ara. The deformation processing part performs deformation processing in which the first area or the area including the first area is subjected to enlarging deformation, and in accordance with this, the second area is subjected to reducing deformation.

Description

  Embodiments described herein relate generally to an image processing apparatus, an image processing method, and a program.

  For example, when a large number of images are displayed on a display, there is a method of using thumbnail images as a method of effectively using a limited display space. Several methods are known for generating a thumbnail image from an original image. For example, as a general method, a method of generating a thumbnail image by uniformly reducing the entire image is well known. However, thumbnails created by this method have a problem that it is difficult to visually recognize an object (for example, a building or a person) representing the characteristics of the image, and it is difficult to understand the content of the image at a glance. As another method, an area that is characteristic of the image or an area that is likely to be noticed by the user (attention area) is selected or extracted from the original image on some basis, and the attention is selected. By removing the image other than the region (or the region of interest and its vicinity) (that is, by leaving only the region of interest or its vicinity (hereinafter referred to as the region of interest)), the region of interest is relatively emphasized. There is also a technique called clipping that generates a thumbnail image. According to this method, a problem such as forward tilt is solved. However, since information other than the attention area is lost, for example, the same person as an image obtained by photographing a certain person in a certain situation is used. There is still a problem that when comparing with other images taken in different situations, it becomes difficult to distinguish or distinguish or understand the contents of those images.

  As a technique for solving the above problems, there is a method of performing a deformation process that distorts an area other than the attention area while enlarging the attention area in the image. In this method, after executing the deformation process, a thumbnail image is created by uniformly reducing the entire image. For example, the target region is emphasized by enlarging the target region, and the region other than the target region is compressed by being distorted. As a result, the attention area can be emphasized without deleting the information of the entire image. However, according to this technique, since the area other than the attention area is distorted, for example, an object (for example, a building, a person, a moving object, an animal or a plant, a background object, etc.) constituting the situation of the image in the area other than the attention area As a result, there is a problem that the situation or atmosphere of the original image is greatly collapsed.

JP 2006-313511 A

  Visuals relating to features of the original image that are visually recognized by a portion of the original image other than the region that is likely to be noticed by the user or the region of interest that is likely to be characteristic (for example, a building or a person) in the original image It has not been possible to generate a thumbnail image in which the attention area is emphasized (enlarged) while maintaining information (for example, the situation where the original image was captured or the atmosphere of the original image) to some extent.

  The present embodiment is an image processing device capable of generating a thumbnail image that highlights a region of interest while visually recognizing a part other than the region of interest in the original image and maintaining visual information about features of the original image to some extent, It is an object to provide a processing method and a program.

  According to the embodiment, a first area processing unit, a second area processing unit, a deformation information calculation unit, and a deformation processing unit are provided. The first area processing unit obtains a first area to be deformed so as to be enlarged from the target image. The second area processing unit obtains a second area to be deformed so as to be reduced as the first area or the area including the first area is deformed to be enlarged. The deformation information calculation unit obtains deformation information indicating a deformation method of the first area or the area including the first area. The deformation processing unit performs a deformation process for deforming the first area or the area including the first area to be enlarged based on the deformation information and for deforming the second area in accordance with the deformation. Do.

1 is a block diagram illustrating a configuration example of an image processing apparatus according to a first embodiment. It is a figure for demonstrating the image processing by the image processing apparatus of this embodiment. It is a figure for demonstrating an example of the original image expressed with a mesh. It is a figure for demonstrating an example of connection information. It is a figure for demonstrating an example of the expansion process of an attention area. 4 is a flowchart illustrating an example of a processing procedure of the image processing apparatus according to the embodiment. It is a figure for demonstrating a non-mesh image production | generation part. It is a figure for demonstrating a mesh image generation part. It is a figure for demonstrating an example of the production | generation method of a deformation | transformation influence area | region. It is a figure for demonstrating an example of the production | generation method of a deformation | transformation influence area | region. It is a figure for demonstrating an example of the determination method of magnification. FIG. 5 is a diagram illustrating a configuration example of an image processing apparatus according to a second embodiment. 4 is a flowchart illustrating an example of a processing procedure of the image processing apparatus according to the embodiment.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that, in the following embodiments, the same numbered portions are assumed to perform the same operation, and repeated description is omitted.

(First embodiment)
The first embodiment will be described below.

  The image processing apparatus according to this embodiment generates a thumbnail image corresponding to an original image by processing an image to be processed (hereinafter also referred to as an original image).

  In this embodiment, a case where the original image is expressed by a mesh (a mesh including a triangle patch) will be described as an example. However, the present embodiment is not limited to this. It can be modified as appropriate.

  FIG. 1 shows a configuration example of an image processing apparatus according to the present embodiment.

  As shown in FIG. 1, the image processing apparatus according to the present embodiment includes a mesh acquisition unit 11, a region of interest calculation unit (first region processing unit) 12, a deformation influence degree calculation unit 13, a deformation influence region group calculation unit (first 2 region processing unit) 14, deformation influence region selection unit (second region selection unit) 15, deformation information calculation unit 16, mesh deformation unit (deformation processing unit) 17, and mesh presentation unit 18.

  The image processing apparatus according to the present embodiment may be, for example, an information terminal capable of displaying an image, hardware or software incorporated in the information terminal, or a combination thereof. It may be a device connected to the outside of the information terminal, hardware or software incorporated in a device connected to the outside of the information terminal, or a combination thereof. The information terminal may be, for example, a PC (personal computer), a PDA, a mobile phone, a smartphone, a digital camera, or the like. However, these are merely examples, and the mounting method of the image processing apparatus of the present embodiment is not limited.

  Here, first, a process in which the image processing apparatus of the present embodiment generates a thumbnail image from an original image will be described with reference to FIG. Here, an outline of the processing will be described regardless of whether the original image is represented by a mesh.

  FIG. 2 shows, as an example of the original image, an image of a landscape where a plurality of sheep are in a pasture area in a certain garden. Here, the sheep in the center of the image is an example of an object that seems to be characteristic of the image of FIG. 2 or an object that is likely to be noticed by the user. Moreover, if FIG. 2 is seen, the situation where the original image was image | photographed, for example can be recognized visually easily. However, when the thumbnail image is generated from the original image of FIG. 2 by the conventional method, for example, a portion other than the sheep in the center of the image is greatly distorted or completely deleted, and thus created. Looking at the thumbnail images, only sheep are recognized, and it is difficult or impossible to recognize or guess the situation or atmosphere of the original image of FIG.

  Therefore, in the present embodiment, a thumbnail image is generated so that the situation or atmosphere of the original image can be recognized by looking at the thumbnail image.

  First, the image processing apparatus calculates an “attention area” in the original image to be processed. The attention area is, for example, an area that is likely to be noticed by the user or a characteristic area in the original image. The attention area may be calculated by performing processing using a well-known attention area extraction technique. In the original image of FIG. 2, for example, a portion indicated by 200 (a portion including the sheep at the center of the image) is extracted as a region of interest. This attention area 200 will be enlarged later.

  Next, the image processing apparatus calculates one or a plurality of “deformation affected areas” prior to performing the process for enlarging the attention area 200. The deformation-affected area is an area that is set to be compressed (distorted) as the attention area 200 is enlarged. A deformation-affected region (a closed loop of 201, 202, 203 or a closed band-like or donut-shaped region in the figure) surrounds the region of interest 200, or is in addition to the region of interest 200 or one inside of itself. It is configured to surround a plurality of deformation affected areas. Although it is possible to generate thumbnail images by defining only one deformation-affected region, it is desirable to define a plurality of deformation-affected regions. In the following description, a case where a plurality of deformation-affected areas are provided and a repetition process related to a plurality of deformation-affected areas is described as an example. However, when only one deformation-affected area is defined, the repetition is repeated. Processing will be omitted.

  As shown in the example of FIG. 2, the deformation influence area 201 closest to the attention area 200 (innermost) is configured to surround the attention area 200. Of the two boundaries of the deformation affected area 201, the boundary 211 in contact with the area including the attention area 200 is defined as “inscribed boundary”, and the other boundary 212 is defined as “external boundary”. In the example of FIG. 2, the inscribed boundary 211 of the deformation affected area 201 matches the boundary of the attention area 200, but all or a part of the inscribed boundary 211 may not match the boundary of the attention area 200. May be.

  The deformation influence area 202 that is next closest to the attention area 200 (next inside) is configured to surround the deformation influence area 201 and the attention area 200 that exist inside of the attention area 200. In the figure, 221 is an inscribed boundary of the deformation influence area 202, and 222 is a circumscribed boundary.

  The next deformation influence area 203 is configured to surround the two deformation influence areas 201 and 202 and the attention area 200 existing inside thereof. In the figure, reference numeral 231 denotes an inscribed boundary of the deformation effect area 203, and 232 denotes a circumscribed boundary.

  In the example of FIG. 2, the inscribed boundary 231 of the deformation affected area 203 does not coincide with the circumscribed boundary 222 of the deformation affected area 202, and the inscribed boundary 221 of the deformation affected area 202 is the circumscribed boundary of the deformation affected area 201. 212, but part or all of the inscribed boundary of the outer deformation influence area may coincide with the circumscribed boundary of the adjacent inner deformation influence area.

  Further, when there is a deformation influence area, each deformation influence area is configured to surround all the deformation influence areas and the attention area 200 inside thereof in the same manner.

  Note that it is desirable that each deformation-affected region is provided so as to avoid a region portion that is greatly affected when deformed. For example, as illustrated in FIG. 2, it is desirable that the deformation influence region is set so as not to include sheep existing outside the attention region 200 (see, for example, 100 and 101 in FIG. 2). Details of the processing for this will be described later.

  Hereinafter, a set composed of one or more calculated deformation-affected areas is referred to as a “deformation-affected area group”.

  In the deformation influence area group, each deformation influence area may be identified by a deformation influence area identifier. Here, as an example, a description will be given assuming that an area number is assigned to each deformation affected area as a deformation affected area identifier. Here, the area number is a serial number that gradually increases from the deformation influence area closest to the attention area (innermost) to the outside. For example, in the example of FIG. 2, if the area number of the deformation affected area 201 closest to the attention area 200 is “0”, the area numbers “0” to “2” exist, and then the outer deformation affected area 202 The area number is “1”, and the area number of the deformation affected area 203 located on the outermost periphery is “2”. In this case, the order from the innermost side or the outermost side of the deformation influence area or the deformation influence area adjacent to the deformation influence area can be specified directly from the area number of the deformation influence area.

  Note that the deformation affected area identifier or the area number is not limited to the above. Note that it is desirable to be able to specify the order from the innermost side or the outermost side or the adjacent deformation-affected region from the deformation-affected region identifier or region number.

  The image processing apparatus calculates the deformation influence degree for each edge constituting the mesh of the original image, and further corrects the deformation influence area based on the deformation influence degree, which will be described in detail later. . The deformation influence degree is an index indicating, for example, “how much the appearance (appearance) is affected when the area is deformed”.

  Subsequently, the image processing apparatus first selects one deformation influence area (for example, 201) from the deformation influence area group. Then, the area included by the inscribed boundary (for example, 211) of the selected deformation-affected area is enlarged according to a predetermined method.

  For example, based on one or both of a circumscribed boundary (for example, 212) and an inscribed boundary (for example, 211) of the selected deformation-affected region (for example, 201), a region ( That is, the image inside the inscribed boundary may be enlarged. The predetermined closed loop is located outside the inscribed boundary. Further, the predetermined closed loop may preferably be the circumscribed boundary or may be located inside the circumscribed boundary.

  Specifically, for example, various methods including the following methods are conceivable.

  (1) The region enclosed by the inscribed boundary is expanded until the inscribed boundary (for example, 211) comes into contact with the circumscribed boundary (for example, 212).

  (2) A region included by the inscribed boundary until the inscribed boundary comes into contact with a closed loop passing between the circumscribed boundary (for example, 212) and the inscribed boundary (for example, 211) To enlarge.

  (3) The region enclosed by the inscribed boundary is expanded until the inscribed boundary (for example, 211) increases outward by a predetermined amount (for example, a predetermined pixel (for example, one pixel)).

  In addition to these, it is possible to enlarge the region enclosed by the inscribed boundary of the selected deformation-affected region by various methods.

  The image processing apparatus repeats the enlargement process as described above for all or part of the deformation affected areas included in the deformation affected area group sequentially (for example, in ascending order of area numbers), thereby recreating the original image. Generate a processed thumbnail image.

  As a result, the thumbnail image is obtained by enlarging the image in the attention area 200 in the original image and reducing the image in one or more deformation affected areas.

  Next, an image (original image) to be processed will be described with reference to FIGS. 3 and 4.

  FIG. 3 shows an example of the expression format of the original image.

  As shown in FIG. 3, the original image may be represented by a mesh including a plurality of patches 501 in the coordinate space defined by the position (x, y) and the luminance I. Hereinafter, an image expressed by a mesh is referred to as a “mesh image”.

  Here, the patch 501 is a single surface surrounded by an edge 503 connecting the vertices 502. In FIG. 3, a triangular patch having a triangular shape is shown as an example of the patch. However, the patch is not limited to the triangular patch, and may be a patch having another shape. Hereinafter, a case where a triangular patch is used will be described as an example.

  The mesh may be represented by “coordinate information (x, y, I)” of each vertex 502 and “connection information” indicating to which vertex 502 each vertex 502 is connected by the edge 503.

  FIG. 4A shows an example in which an original image is displayed on xy coordinates, and FIG. 4B shows an example of a connection relationship between vertices corresponding to a part (240) of the image in FIG. For example, the connection information is information indicating that “vertex P1 is connected to vertices P2 to P8” for the connection as shown in FIG. 4B. The connection information may be expressed in any format.

  Each patch may have a patch identifier for identifying each patch. Each edge may be given an edge identifier for identifying each edge. Each vertex may be given a vertex identifier for identifying each vertex.

  In the above description, the case where each vertex has only the luminance I has been described as an example. For example, when the original image is a color image, each vertex is added to the luminance I or instead of the luminance I. Color information (eg, RGB values) can be included.

  Next, an example of attention area expansion processing according to the present embodiment when the original image is expressed by a mesh will be described with reference to FIG.

  The mesh in FIG. 5 represents a part of the original image. In FIG. 5, a central area 300 represents the attention area. The outer periphery of the attention area 300 is configured by any edge of any patch.

  In FIG. 5, a region surrounding the attention region 300 is a deformation influence region 301, and a region further outside is a deformation influence region 302. Further, in FIG. 5, inscribed boundaries 311 and 321 and circumscribed boundaries 312 and 322 are shown. The inscribed boundary is constituted by any edge of any patch, and the circumscribed boundary is constituted by any edge of any patch (in FIG. 5, the inscribed boundary and the circumscribed boundary are For the sake of distinction, for the sake of convenience, a portion is shown in the vicinity of the edge, not on the edge).

  In FIG. 5, when performing the enlargement process, for example, the image inside the inscribed boundary 311 of the deformation affected area 301 is enlarged within the range up to the circumscribed boundary 312 by, for example, the predetermined method as described above. Similarly, the image inside the inscribed boundary 321 of the deformation affected area 302 is enlarged within the range up to the circumscribed boundary 312.

  In the following, the image processing apparatus according to the present embodiment and the thumbnail image generation process performed by the image processing apparatus according to the present embodiment will be described in more detail when the original image to be processed is represented by a mesh.

  The outline of each block of the image processing apparatus in FIG. 1 will be described below.

  The mesh acquisition unit 11 is for acquiring a mesh constituting the original image.

  The attention area calculation unit (first area processing unit) 12 is for calculating the attention area in the original image acquired by the mesh acquisition unit 11.

  The deformation influence degree calculation unit 13 calculates the deformation influence degree, which is an index indicating “how much the appearance (appearance) is affected when the area is deformed” for each edge constituting the mesh of the original image. It is for calculating.

  A deformation influence area group calculation unit (second area processing unit) 14 calculates one or a plurality of deformation influence areas (preferably, a plurality of deformation influence areas) for the original image acquired by the mesh acquisition unit 11. Is.

  For example, the deformation influence area group calculation unit 14 further determines the deformation influence degree and the outermost deformation influence area at the present time (however, if the deformation influence area has not been calculated yet, the attention area) and further outside the deformation influence area. The process of calculating a new deformation-affected region that surrounds is repeated. The outermost deformation influence area at the present time is, for example, the deformation influence area corresponding to the area number having the largest value at the present time when the area number is used.

  For example, in the example of FIG. 5, first, a deformation influence area 301 surrounding the outside of the attention area 300 is generated, and then a deformation influence area 302 surrounding the outside of the deformation influence area 301 is generated.

  The deformation influence area group calculation unit 14 may repeat generation of the deformation influence area, for example, until generation of the deformation influence area becomes impossible, or repeat generation of the deformation influence area a predetermined number of times. It may be a combination thereof. There may be various other methods for determining the number of times the deformation-affected region group calculation unit 14 repeats the generation of the deformation-affected region.

  The deformation influence area selection unit (second area selection unit) 15 is for sequentially selecting deformation influence areas one by one in a predetermined order from the deformation influence area group calculated by the deformation influence area group calculation unit 14. belongs to.

  For example, the deformation influence area selection unit 15 may select the deformation influence area from the deformation influence area group in the order from the innermost side to the outer side. The order from the innermost side toward the outer side is, for example, the order having the smaller area numbers when using area numbers.

  The deformation information calculation unit 16 determines the inscribed boundary of the selected deformation influence region based on information about the deformation influence region selected by the deformation influence region selection unit 15 (hereinafter referred to as “selected deformation influence region”). This is for calculating the enlargement center and the enlargement magnification when enlarging the image inside.

  The mesh deformation unit 17 (deformation processing unit) deforms the inscribed boundary of the selected deformation-affected area and the vertex group constituting the area included by the inscribed boundary (hereinafter referred to as “deformed vertex group”). This is for enlarging a part of the mesh by moving based on the enlargement center and enlargement magnification calculated by the information calculation unit 16.

  The mesh presentation unit 18 converts an image (mesh image) obtained by deforming the original image into a general image representation (non-mesh image) that is not represented by a mesh. This non-mesh image becomes a thumbnail image.

  Next, FIG. 6 shows an example of a processing procedure for generating a thumbnail image of the image processing apparatus of this embodiment.

  Hereinafter, the processing by each block will be described in detail along the processing flow.

<Mesh acquisition unit 11>
The mesh acquisition unit 11 acquires an original image (mesh image) to be processed (step S101).

  The mesh acquisition unit 11 may acquire a mesh image generated in advance as an original image from the outside. In this case, the mesh acquisition unit 11 may acquire a mesh image recorded in advance on a general storage medium such as a ROM, RAM, or HDD. The mesh acquisition unit 11 may acquire a mesh image via a wired or wireless network such as the Internet or an intranet. The mesh acquisition unit 11 may acquire a mesh image from a nearby fixed or portable device directly connected by wire or wireless. Alternatively, the mesh image may be created by the user on this image processing apparatus. In addition to this, various acquisition methods are possible.

<Attention Area Calculation Unit 12>
The attention area calculation unit 12 calculates an attention area that is an area that is likely to be noticed by the user or an area that is likely to be characteristic of the original image (step S102).

  For example, first, an original image that is a mesh image is converted into an image (non-mesh image) composed of general pixels that are not represented by a mesh. This conversion can be realized, for example, by calculating the luminance I from the XY coordinates of each pixel and a patch including the coordinates on the XY plane. When the original image is a color image, the luminance I may be calculated for each channel.

  For this conversion, for example, as shown in the configuration example of FIG. 7A, the attention area calculation unit 12 may include a non-mesh image generation unit 21 that generates a non-mesh image from a mesh image. . Alternatively, as shown in the configuration example of FIG. 7B, the mesh acquisition unit 11 includes a non-mesh image generation unit 21, generates a non-mesh image from the acquired mesh image, and A mesh image and a non-mesh image may be supplied from the acquisition unit 11 to the attention area calculation unit 12.

For example, from a non-mesh image, an arbitrary region-of-interest detection method generally performed in the field of image processing is used. Called). For example,
(1) After generating an attractiveness map from a non-mesh image, an area with a high attractiveness is taken out from the non-mesh image, and the area is set as a basic attention area, or
(2) After generating the attractiveness map from the non-mesh image, further image clustering is performed, the attractiveness in the cluster obtained by this is calculated, and the cluster with the highest attractiveness is set as the basic attention area, or ,
(3) A person is recognized from a non-mesh image using face detection, and a cluster including the person is set as a basic attention area.
Various methods are possible.

  Thereafter, for example, among a plurality of patches in the original image, a patch group including patches having a predetermined relationship with the extracted basic attention area may be set as the attention area in the original image to be obtained. Here, the patch having the predetermined relationship is, for example, (1) an area where at least a part of the patch overlaps the basic attention area, or (2) an area of a portion where the patch overlaps the basic attention area. Various criteria can be used, for example, the ratio obtained by dividing the total area of the patch exceeds a predetermined threshold, or (3) the entire area of the patch is included in the basic attention area.

  By the way, the mesh acquisition part 11 may acquire a non-mesh image instead of a mesh image by the acquisition method as mentioned above, for example. In this case, for example, as shown in the configuration example of FIG. 8, the mesh acquisition unit 11 includes a mesh image generation unit 22 that generates a mesh image from a non-mesh image, for example, “Nakamura Norihiro, Abe Hidehiko, Nishio Koji , Kenichi Kobori, “A method for resolution conversion using Delaunay triangulation”, Image Electronics Society Journal, Vol.35, No.5, 2005. An original image that is a mesh image may be newly generated, and the mesh image and the non-mesh image may be supplied from the mesh acquisition unit 11 to the attention area calculation unit 12.

  In addition, when the original image which is a mesh image is produced | generated from the non-mesh image acquired by the mesh acquisition part 11, the attention area calculation part 12 uses the non-mesh image supplied from the mesh acquisition part 11 as it is. May be.

<Deformation influence calculation unit 13>
The deformation influence calculation unit 13 calculates the deformation influence degree for each edge constituting the mesh (step S103).

As the deformation influence degree, for example, a geometric feature amount can be used. Various geometric features such as an angle DAangle formed by normal vectors of two surfaces having an edge as a component, a minimum value of the area Area, and an average value of the area Area can be used. As a suitable example, for example,
Deformation influence = DA Angle (1)
Deformation influence level = Min (Area) * DANGLE (2)
Deformation influence = Average (Area) * Dangle (3)
However, the present invention is not limited to this, and other combinations are also possible. It is also possible to use geometric feature amounts other than the angle DA Angle, the minimum value of the area Area, and the average value of the area Area.

  By the way, the process by the attention area calculation unit 12 and the process by the deformation influence calculation unit 13 may be performed first or may be performed simultaneously. When the original image which is a mesh image is acquired by the mesh acquisition unit 11, the non-mesh image is obtained at an arbitrary timing before the processing of extracting the basic attention region from the non-mesh image by the attention region calculation unit 12 is started. May be generated. Further, when a non-mesh image is acquired by the mesh acquisition unit 11, an arbitrary process before the process of calculating the attention area in the original image by the attention area calculation unit 12 or the process by the deformation influence calculation unit 13 is started. An original image that is a mesh image may be generated at the timing.

<Deformation Influence Area Group Calculation Unit 14>
The deformation influence area group calculation unit 14 sets one or a plurality of deformation influence areas (desirably, a plurality of deformation influence areas) surrounding the attention area.

  Hereinafter, this process will be described in detail with reference to FIGS. 9 and 10.

  FIG. 9A illustrates a state in which the attention area 400 is generated by the attention area calculation unit 12.

  First, the deformation affected area group calculating unit 14, for example, out of the patch group having, as constituent vertices, vertices that constitute the boundary between the attention area 400 and other areas (areas outside the attention area 400). Is selected as a deformation-affected region candidate patch group (step S104).

  Next, in step S105 (described later in detail), if the selected deformation-affected region candidate patch group is directly adopted as a deformation-affected region, as illustrated in FIG. The region 401 is the innermost deformation influence region. Further, as will be described later, the importance of the deformation affected area 401 is obtained.

  Next, when it is determined in step S107 (to be described in detail later) that a deformation affected area is to be generated, for example, the boundary between the deformation affected area candidate patch group 401 and an area outside the patch group 401 Is selected as a new deformation influence area candidate patch group (step S104). For example, as shown in FIG. 9C, the patch group indicated by 402 is selected as a new deformation influence area candidate patch group.

  Next, the deformation affected area group calculation unit 14 performs the process of step S105 in FIG. Hereinafter, this process will be described in detail with reference to FIG.

  FIG. 10A is a diagram in which an arrow is added to FIG. 9C.

  First, the deformation-affected region group calculation unit 14 selects any one patch in the deformation-affected region candidate patch group 402 (for example, a triangular patch indicated by a dotted edge on the left side of FIG. 10A). select.

  From the selected patches, the patches in the deformation affected area candidate patch group 402 are searched in the order of adjacency as indicated by the arrow in FIG. In the example of FIG. 10, the search is performed counterclockwise, but the search may be performed clockwise.

  Hereinafter, the currently selected patch is defined as “optimization target patch”.

  The deformation-affected region group calculation unit 14 is a region where the configuration edge of the optimization target patch is outside the current deformation-affected region candidate patch group 402 (for example, an unhatched patch group outside the patch group 402 in FIG. 10). (For example, when the optimization target patch is the patch 441 in FIG. 10B), the current modification-affected region candidate patch group 402 is passed through as it is (in the drawing, from the patch 441 to the patch 442). (When passing through as indicated by arrow 451) and when passing through the outer deformation area and the outer region (when passing from patch 441 to patch 442 as indicated by arrow 452). And the total value of the deformation influence degree of the edges passing through the. Then, when these total values are compared, and the former total value is less than or equal to the latter total value, the current deformation-affected region candidate patch group 402 is maintained, and the latter total value is lower than the former total value. In this case, the patch 403 and 444 related to the internal path 451 (patches surrounded by the broken line 450) 443 and 444 are updated by replacing the patch 403 and 444 related to the external path 452 with the patch 443 and 444 related to the external path 452. This is used as a new deformation influence area candidate patch group 402.

  For example, when the patch 441 in FIG. 10B is an optimization target patch, out of the vertices constituting the edge shared with the patch searched next to the patch, the vertex that is in contact with the outer periphery is used as a reference. The deformation influence degree is added until the area outside the current deformation influence area candidate patch group 402 returns to the patch included in the current deformation influence area candidate patch group 402 (patch 442 in FIG. 10B) again. The total obtained as a result is used as the first correction evaluation value. Thus, after the patch 442 is specified, the inside of the deformation influence area candidate patch group 402 is searched again from the patch 441 to the patch 442 while adding the deformation influence degree, and the total obtained as a result is obtained as the second sum. A corrected evaluation value is used. Here, when the first correction evaluation value is smaller than the second correction evaluation value, the deformation-affected region candidate patch group 402 is corrected as shown in FIG.

  For example, this correction occurs when a characteristic object exists in the patch surrounded by the broken line 450. Accordingly, it is possible to provide a deformation-affected region area so as to avoid a region portion that is greatly affected when deformed.

  When correction is made after the path comparison from the start point patch 441 to the end point patch 442 is performed, for example, a new search is continued from the end point patch 442.

  On the other hand, if the correction is not performed after the path comparison from the start point patch 441 to the end point patch 442 is performed, the next patch adjacent to the patch 441 is searched in order.

  This is repeated until the patch selected at the start of the search is returned to one set, and is repeated within a predetermined set (for example, one set) until correction processing is not performed. However, if the patch selected at the start of the search is not included in the deformation-affected region candidate patch group 402 due to correction, the target patch is always included when obtaining the above-described second correction evaluation value. Assume that one set has been completed.

  In the above description, when the correction evaluation value is obtained, the total deformation influence degree is used, but other calculations such as an average value, a maximum value, and a minimum value may be used instead of the total.

  In the above description, the search is started from the patch 442 after the correction process, but may be started from the patch 441. In this case, sufficient correction is performed by performing only one set process. In addition, other methods such as a method of continuing a new search from the patch 443 adjacent to the start point patch 441 among the patches newly added in the correction process are possible.

  The deformation affected area candidate patch group 402 corrected by the processing so far is set as a new deformation affected area 402 ((c) in FIG. 10). Then, the importance of the deformation affected area 402 is calculated. For example, the deformation influence degrees of the edges included in the deformation influence area 402 may be summed, and the total value may be used as the importance. Here, the edge included in the region indicates that both of the two patches that share the edge are included in the deformation influence region 402.

  When the newly determined deformation influence area does not include a patch constituting the boundary of the image, the patch group outside the newly added deformation influence area is set as a new deformation influence area candidate patch group, The above correction processing is repeated (steps S106, S105, S107).

  As a method for determining a new deformation-affected area candidate patch group, for example, a newly added deformation-affected area among patch groups having vertices constituting a circumscribed boundary of a newly added deformation-affected area as a constituent vertex. There are ways to select things that are not included.

  If the newly determined deformation-affected region includes a patch that forms the boundary of the image, the process goes out of the loop of the iterative process and proceeds to step S108.

<Deformation Influence Area Selection Unit 15>
Next, the deformation influence area selection unit 15 selects one deformation influence area to be compressed (distorted) by deformation from the generated deformation influence area group (step S108).

  As a selection method, for example, there may be various methods such as a method of selecting in ascending order of area numbers, a method of selecting in descending order of area numbers, and a method of selecting in descending order of importance calculated for each deformation affected area. .

  It should be noted that the deformation-affected region selected here is excluded from selection candidates in the subsequent selection in step S108. That is, in the current step S108, the deformation affected area selected in the previous step S108 (that is, already deformed) is not selected. However, for example, in the method of selecting in ascending order of area numbers or the method of selecting in order of increasing area numbers, the deformation-affected areas that have already been selected (deformed) are automatically excluded from the selection candidates.

  In the above description, the deformation-affected area that has already been selected (deformed) is excluded from the selection candidates. However, for example, when selecting in order of decreasing importance, even if deformation is performed, the deformation-affected area is not affected. Since the area remains, it is possible to calculate the importance again. Therefore, the degree of importance may be calculated again for a deformation-affected area that has already been selected (deformed) without removing it from the selection candidates.

  If the importance level is not used here, the calculation of the importance level by the deformation affected area group calculation unit 14 may be omitted.

<Deformation information calculation unit 16>
Next, the deformation information calculation unit 16 obtains an enlargement center and an enlargement magnification used when executing enlargement (step S109).

  As a method for obtaining the enlargement center, for example, an average value of the coordinates of the vertices constituting the inscribed boundary of the selected deformation-affected area is obtained and used as the enlargement center, or the inscribed boundary of the selected deformation-affected area. Various methods are possible, such as a method of obtaining the center of gravity of the patch group included in the inside and using this as the enlargement center.

  There are various methods for obtaining the magnification.

  For example, after obtaining each vertex constituting the inscribed boundary of the selected deformation-affected area and the distance on the XY plane of the enlargement center, the maximum value is set as the original distance L, and is determined in advance with respect to L. There is a method of calculating L ′ / L (= 1 + a / L) after obtaining L ′ obtained by adding the obtained value a, and using this as an enlargement magnification.

Other methods are also possible. For example, as shown in FIG. 11, a half line from the expansion center 802 to the vertex 803 constituting the inscribed boundary of the selected deformation-affected area 801 is considered, and an intersection 806 where the half line further intersects the circumscribed boundary is obtained. After the processing for obtaining the distance between 803 and the intersection 806 is performed for all the vertices constituting the inscribed boundary, the minimum value LI _min is obtained. In addition, for example, a line segment from the expansion center 802 to the vertex 804 constituting the circumscribed boundary is considered, and an intersection 805 where the line segment further intersects the inscribed boundary is obtained, and the distance between the vertex 804 and the intersection 805 is determined. After performing the process for obtaining all the vertices constituting the circumscribed boundary, the minimum value LO _min is obtained. The smaller value of the obtained LI _min and LO _min is defined as a. When LI _min is the minimum value, the distance to the vertex 803 when the minimum value is taken is L, and when LO _min is the minimum value, the distance to the intersection 805 when the minimum value is taken Let L be the same as above to determine the magnification.

  In addition to this, for example, the enlargement magnification is increased stepwise from 1.0, and each time it is confirmed whether the circumscribed boundary and the inscribed boundary intersect each other, an enlargement immediately before the intersection is made. Various methods are possible, such as a method of employing a magnification.

<Mesh deformation part 17>
The mesh deforming unit 17 expands the deformed vertex group related to the selected deformation-affected area in the mesh image with the enlargement center and the enlargement factor obtained by the deformation information processing unit 16 (step S110). However, components such as luminance I or color information RGB are not enlarged, and only the XY coordinate values are enlarged.

  Note that the mesh deforming unit 17 may reduce the mesh inside the selected deformation-affected area according to the expansion of the deformed vertex group.

  Up to the above, the enlargement processing related to one deformation-affected area has been executed. However, when elements exist in the deformation-affected area group and the magnification has not been enlarged to a predesignated magnification (step). S111), returning to step S108, the same processing is repeated.

  If there is no element in the deformation-affected area group or if the specified enlargement magnification is reached (step S111), the process goes out of the loop of repetitive processing and proceeds to step S112.

  The enlargement magnification may be fixedly set in advance in the image processing apparatus, or may be designated from the outside to the image processing apparatus (for example, by a user or another process) each time. May be set autonomously by a predetermined method.

  The determination as to whether or not the image has been enlarged to a pre-specified enlargement magnification is based on, for example, whether or not the value obtained by multiplying all the enlargement magnifications in the previous enlargement processing is equal to or greater than the pre-specified enlargement magnification. Various determination methods are possible, such as.

  In addition, various other methods are possible for determining whether or not to perform further iterative processing in step S111.

<Mesh presentation unit 18>
The mesh presentation unit 18 presents a thumbnail image (for example, on the display screen) (step S112).

  Prior to this presentation, the original image (mesh image) transformed by the processing so far is converted into a general image representation (non-mesh image) that is not represented by a mesh, and this is used as a thumbnail image. This can be achieved, for example, by calculating the luminance I (calculated for each channel in the case of a color image) from the XY coordinates of each pixel and a patch including the coordinates on the XY plane.

  Note that the mesh presentation unit 18 may include a non-mesh image generation unit (not shown) similar to the non-mesh image generation unit 21 described above, and perform this conversion process.

  Further, the above-described non-mesh image generation unit 21 may perform this conversion process. Further, the non-mesh image generation unit 21 described above may be in the mesh presentation unit 18 or in another location.

  Note that thumbnail images include a reduced image of the original image corresponding to the deformation-affected area (hereinafter referred to as a reduced image) inside the portion corresponding to the deformation-affected area where the deformation has been performed. The portion of the reduced image in the thumbnail image may be modified so that the image becomes less conspicuous based on, for example, the pixel values of the surrounding area.

  Note that the mesh presenting unit 18 may output the generated thumbnail image to a predetermined target in addition to or instead of presenting the generated thumbnail image (for example, on the display screen).

  In this case, the mesh presentation unit 18 may store the thumbnail image in a general storage medium such as a ROM, a RAM, or an HDD, or a wired or wireless network such as the Internet or an intranet. The thumbnail image may be transmitted to a predetermined destination via the network, or the thumbnail image may be stored in a nearby fixed or portable device directly connected by wire or wireless.

  According to the present embodiment, the region is visually recognized by a portion in the original image other than the region that is likely to be noticed by the user in the original image or the region of interest that is considered to be characteristic (for example, a building or a person). It is possible to generate a thumbnail image that emphasizes (enlarges) a region of interest while maintaining a certain amount of visual information about the characteristics of the original image (for example, the situation where the original image was captured or the atmosphere of the original image) Become.

(Second Embodiment)
Hereinafter, the second embodiment will be described focusing on differences from the first embodiment.

  FIG. 12 shows a block diagram of a configuration example of the image processing apparatus of the present embodiment.

  As shown in FIG. 12, the configuration example of the present embodiment is a configuration example in which an area designation unit (designation unit) 19 is further added to the configuration example of the first embodiment (FIG. 1).

<Area specifying unit 19>
The area specifying unit 19 is for acquiring area specifying information from the outside so that the user can specify the area of interest to some extent.

  The area designation information is, for example, a closed loop in which straight lines or curves are connected at specific intervals, and the end point positions and order are stored.

  For example, the area designation information may be generated in real time by a user input such as a mouse input, or a file storing the above information may be read. Even with other means, the closed loop as described above may be input.

  FIG. 13 shows a flowchart of an example of a processing procedure of the image processing apparatus of this embodiment. The difference from the first embodiment (FIG. 6) is that “acquisition of area designation information” is added. Note that “acquisition of area designation information” may be performed at an arbitrary timing before the process by the attention area calculation unit 12 is executed.

  In the present embodiment, a region of interest calculation unit 12 is different from the first embodiment except for the region designation unit 19.

  Hereinafter, the difference between the attention area calculation unit 12 and the first embodiment will be described.

  The attention area calculation unit 12 in this embodiment calculates the attention area based on the area designation information acquired from the area designation unit 19.

  For example, the attention area calculation unit 12 calculates all the patches included in the closed loop (area specification information), and sets the patch group as the attention area.

For criteria on whether to be included in the closed loop, for example,
(1) If even a part of the patch is present in the closed loop, the patch is included in the closed loop.
(2) The patch is included in the closed loop only when the ratio obtained by dividing the area of the portion existing in the closed loop of the patch by the total area of the patch exceeds a predetermined threshold.
(3) The patch is included in the closed loop only when the entire surface of the patch is included in the closed loop.
Various standards are conceivable.

  For example, in the example of FIG. 5, when the closed loop indicated by 130 is designated, the attention area indicated by 300 is obtained when the above criterion (3) is used.

  According to this embodiment, in addition to the first embodiment, the attention area can be controlled by the user.

By the way, the instructions shown in the processing procedure shown in the above-described embodiments (first embodiment, second embodiment) can be executed based on a program that is software. The general-purpose computer system stores this program in advance and reads this program, so that the same effect as that obtained by the image processing apparatus according to the above-described embodiment can be obtained. The instructions described in the above-described embodiments are, as programs that can be executed by a computer, magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD). ± R, DVD ± RW, etc.), semiconductor memory, or a similar recording medium. As long as the recording medium is readable by the computer or the embedded system, the storage format may be any form. If the computer reads the program from the recording medium and causes the CPU to execute instructions described in the program based on the program, the same operation as the image processing apparatus of the above-described embodiment can be realized. Of course, when the computer acquires or reads the program, it may be acquired or read through a network.
In addition, the OS (operating system), database management software, MW (middleware) such as a network, etc. running on the computer based on the instructions of the program installed in the computer or embedded system from the recording medium implement this embodiment. A part of each process for performing the processing may be executed.
Furthermore, the recording medium in the present embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.
Further, the number of recording media is not limited to one, and when the processing in the present embodiment is executed from a plurality of media, it is included in the recording medium in the present embodiment, and the configuration of the media may be any configuration.

The computer or the embedded system in the present embodiment is for executing each process in the present embodiment based on a program stored in a recording medium. The computer or the embedded system includes a single device such as a personal computer or a microcomputer. The system may be any configuration such as a system connected to the network.
In addition, the computer in this embodiment is not limited to a personal computer, but includes an arithmetic processing device, a microcomputer, and the like included in an information processing device, and is a generic term for devices and devices that can realize the functions in this embodiment by a program. ing.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 11 ... Mesh acquisition part, 12 ... Attention area calculation part, 13 ... Deformation influence degree calculation part, 14 ... Deformation influence area group calculation part, 15 ... Deformation influence area selection part, 16 ... Deformation information calculation part, 17 ... Mesh deformation part , 18... Mesh presentation unit, 19... Region designation unit, 21 .. non-mesh image generation unit, 22.

Claims (16)

A first region processing unit for obtaining a first region to be deformed so as to be enlarged from a target image;
A second region processing unit for obtaining a second region to be deformed so as to be reduced as the first region or the region including the first region is deformed so as to expand;
A deformation information calculation unit for obtaining deformation information indicating a deformation method of the first area or the area including the first area;
A deformation processing unit that performs deformation processing to deform so as to enlarge the first region or the region including the first region based on the deformation information, and to deform the second region according to the deformation; An image processing apparatus comprising: For each of the predetermined unit portions of the target image, a deformation influence degree calculating unit that calculates a deformation influence degree indicating a degree of influence on the appearance of the portion when the predetermined unit portion is deformed. In addition,
The image processing apparatus according to claim 1, wherein the second area processing unit obtains the second area based on the deformation influence degree.   The second area processing unit obtains the second area based on the degree of deformation influence so as to exclude an area having a greater influence on the appearance when the deformation is applied. 2. The image processing apparatus according to 2.   The image processing apparatus according to claim 1, wherein the second area processing unit obtains a plurality of the second areas. A second region selection unit that sequentially selects all or a part of the plurality of second regions as a target of the deformation process;
When the second region is selected by the second region selection unit, the deformation information calculation unit obtains the deformation information and the deformation processing unit performs the deformation processing on the selected second region. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   The second area processing unit performs the sequential selection based on an arrangement relation of a plurality of the second areas or based on a magnitude relation of importance of each of the second areas. The image processing apparatus according to claim 5.   The image processing apparatus according to claim 6, wherein the importance is calculated based on a deformation influence degree indicating a degree of influence on appearance when the second area is deformed.   The image processing apparatus according to claim 4, wherein each of the plurality of second regions constitutes a closed loop that does not intersect with each other.   The image processing apparatus according to claim 1, wherein the second region constitutes a closed loop including the first region. The second region has a first boundary closer to the first region and a second boundary farther from the first region of the second region,
The deformation processing unit enlarges a region included in the first boundary of the second region in the range up to the second boundary of the second region in the change processing. The image processing apparatus according to 9. The target image is represented by a mesh composed of polygonal patches,
The first area processing unit obtains the first area with the polygon patch as a minimum unit,
The second area processing unit obtains the second area with the polygon patch as a minimum unit,
The image processing apparatus according to claim 1, wherein the deformation processing unit performs the deformation processing by deforming the polygonal patch.   The image processing apparatus according to claim 1, wherein the deformation information is an enlargement center and an enlargement magnification when the first area or the area including the first area is deformed. .   The said 1st area | region process part calculates | requires a said 1st area | region by extracting the characteristic part or the part which attracts attention from the said image used as the object. Image processing apparatus. A designation unit for inputting designation information for designating the first area;
The image processing apparatus according to claim 1, wherein the first area processing unit obtains the first area based on the designation information. An image processing method of an image processing apparatus comprising a first region processing unit, a second region processing unit, a deformation information calculation unit, and a deformation processing unit,
The first region processing unit obtaining a first region to be deformed so as to be enlarged from a target image;
The second area processing unit obtaining a second area to be deformed so as to be reduced as the first area or the area including the first area is deformed to be enlarged; and
The deformation information calculating unit obtaining deformation information indicating a deformation method of the first region or the region including the first region;
A deformation process in which the deformation processing unit deforms to enlarge the first area or the area including the first area based on the deformation information, and deforms to reduce the second area accordingly. An image processing method comprising the steps of: A program for causing a computer to function as an image processing apparatus,
A first region processing unit for obtaining a first region to be deformed so as to be enlarged from a target image;
A second region processing unit for obtaining a second region to be deformed so as to be reduced as the first region or the region including the first region is deformed so as to expand;
A deformation information calculation unit for obtaining deformation information indicating a deformation method of the first area or the area including the first area;
A deformation processing unit that performs deformation processing to deform so as to enlarge the first region or the region including the first region based on the deformation information, and to deform the second region according to the deformation; A program to make a computer realize.

Images