JP2009245404A - Image processor, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create a partial image by automatically extracting an image portion to be noticed in an image, on the basis of a trimming rule. <P>SOLUTION: A partial image Q reflecting user preference or habit is created by: storing an original image P; storing a trimming rule R obtained by storing the original image P and learning the configuration information of a predetermined partial image cut out from a predetermined original image; obtaining the degrees to which the features of parts in the original image P are different from the features of parts, existing in the periphery thereof, on the basis of colors and brightness of the original image P and the direction of a straight line component appearing in the original image P; extracting an area to be noticed from the original image P according to the degree; and cutting out an image within a predetermined range, including the extracted area to be noticed from the original image P on the basis of the stored trimming rule R. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, method, and program for creating a partial image by cutting out an image within a predetermined range including a desired region of interest from an original image from the original image.

  Traditionally, I have enjoyed works that cut out photos and decorated them with text and illustrations. In particular, in recent years, similar processing has been performed on a personal computer using digital images.

  Specifically, this kind of processing is performed by cutting out the object part that becomes the material from the digital image as a partial image, but the creation of such a partial image that becomes the material is the object area that humans want to use visually Is extracted from the image and necessary parts are cut out. However, it takes a very long time to select an appropriate area from a large number of images and extract an optimal object area.

Therefore, when a specific object such as a human face is cut out, whether or not it is a specific object (for example, a face portion) from an image using a detection technique (for example, a face detection technique) corresponding to the object. There is a proposal of a method of cutting out an object as an independent image based on an index indicating (for example, Patent Document 1).
Japanese Patent Application Laid-Open No. 2004-228995

  However, even if the object is cut out as an independent image based on the index indicating whether or not the object is a specific object, a partial image cannot be created with a cut-out range reflecting the user's preference and habit.

  Therefore, the present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus, method, and program capable of creating a partial image reflecting user preferences and habits. is there.

  An image processing apparatus according to the present invention includes an original image storage unit that stores an original image, and a trimming rule storage that stores a trimming rule obtained by learning composition information of a predetermined partial image cut out from the predetermined original image. Based on the means, the color and brightness of the original image, and the direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the part located around the part is determined. The attention area extracting means for extracting the attention area from the original image according to these degrees, and a predetermined area including the attention area extracted by the attention area extraction means based on the trimming rule stored in the trimming rule storage means Partial image that creates a partial image by cutting out an image within the range of the original image It is characterized in further comprising a forming unit.

  The “attention area” is an area that is visually noticed on the original image, but the “attention area” is visually noticed regardless of whether or not it is truly visually noticed. It includes areas determined to be highly likely. The area that is visually noticeable is often noticed because the color, brightness, and direction of the linear component that appeared in the original image are different from the surrounding image. A region where the appearing linear component is different from the surrounding is extracted as a “region of interest”.

  “Cut out an image within a predetermined range including a region of interest from the original image” may be performed by cutting out the image so that at least a part of the region of interest is included. Further, it may be cut out so that a plurality of attention areas are included in one cut out image.

  The “trimming rule” refers to data indicating information for cutting out an image in a predetermined range from an original image by a numerical value or the like. For example, the trimming rule may be a trimming rule based on one or more pieces of information on the type, position, size, and orientation of an object appearing in a predetermined partial image.

  The image processing apparatus of the present invention may further include display means for displaying the partial image created by the partial image creation means.

  The image processing apparatus according to the present invention further includes trimming rule updating means for learning the composition information of a predetermined partial image and the composition information of the partial image created by the partial image creating means, and updating the trimming rule. May be.

  In the image processing apparatus of the present invention, the attention area extraction unit calculates the attention level for each extracted attention area, and the display means sequentially displays the partial images including the attention area having a high attention level. There may be.

  The image processing apparatus of the present invention further includes instruction means for instructing the partial image creation means to change the clipping range of the partial image created by cutting out from the original image, and the partial image creation means is A partial image in which the designated cutout range is changed may be created.

  In the image processing apparatus according to the present invention, the attention area extraction unit may further include an object determination unit that analyzes the original image and determines the presence or absence of a specific type of object appearing in the original image.

  The “attention area extraction unit” calculates the attention level for each extracted attention area, and the “display unit” displays the partial images including the attention area with the higher attention level in order. Good.

  The image processing method of the present invention stores an original image, stores the original image, stores a trimming rule obtained by learning composition information of a predetermined partial image cut out from the predetermined original image, and stores the original image. Based on the color and brightness of the image, and the direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the parts located around the part is obtained. In response, an attention area is extracted from the original image, and a partial image is created by cutting out an image within a predetermined range including the extracted attention area from the original image based on a stored trimming rule. It is characterized by.

  The program of the present invention has a function of storing an original image in a computer, a function of storing a trimming rule obtained by learning composition information of a predetermined partial image cut out from the predetermined original image, and an original image Based on the color, brightness, and direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the part located around the part is obtained. In accordance with the function of extracting the attention area from the original image and the trimming rule stored, the image within the predetermined range including the extracted attention area is cut out from the original image. The function to be created is executed.

  According to the present invention, the original image is stored, the original image is stored, the trimming rule obtained by learning the composition information of the predetermined partial image cut out from the predetermined original image is stored, and the original image is stored. Based on the color, brightness, and the direction of the linear component appearing in the original image, the degree of difference between the features of each part in the original image is different from the characteristics of the parts located around the part, and according to these degrees Extract the attention area from the original image, and based on the stored trimming rules, cut out the image within the specified range including the extracted attention area from the original image to reflect the user's preference and habit Partial images can be created.

  The first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a schematic configuration diagram of an image processing apparatus of the present invention.

  The image processing apparatus 1 is realized by executing an application program read into an auxiliary storage device on a computer (for example, a workstation). The image processing program may be distributed as a recording medium such as a CD-ROM and installed on the computer from the recording medium, or may be installed after being downloaded from a server connected via a network such as the Internet.

  The image processing apparatus 1 is obtained by learning original image storage means 10 for storing an original image P and composition information of a plurality of partial images cut out from a plurality of original images used for partial image creation. A region of interest is extracted from the original image P based on the trimming rule storage means 15 for storing the trimming rule R, the color and brightness of the original image P, and the direction of the linear component appearing in the original image P. An image within a predetermined range including the attention area extracted by the attention area extraction means 11 based on the attention area extraction means 11 and the trimming rule R stored in the trimming rule storage means 15 from the original image P. The partial image creating means 12 for creating the partial image Q by cutting out and the partial image Q are displayed. And a indicates means 13.

  The original image storage means 10 is a mass storage device such as a hard disk or a recording medium connected to the image processing device 1. The original image storage means 10 stores a large number of original images P taken by digital still cameras or digital video cameras, or illustration images created using image creation software such as an illustrator. This original image P refers to a still image. Hereinafter, the original image P will be described as a still image.

  The attention area extracting unit 11 extracts a portion that is noticed when the original image P is visually confirmed as an attention area. For example, a part of the color on the original image P is different from the surrounding color, a part on the original image P is very bright compared to the surroundings, a straight line part appearing on a flat screen, etc. It becomes an area that attracts attention. Therefore, based on the color and brightness of the original image P, and the direction of the linear component appearing in the original image P, the degree to which the characteristics of each part in the original image P differ from the characteristics of the parts located around that part. In search of these, a region having a large degree of difference is extracted as a region of interest.

  In this way, the region of interest that is visually noticed has elements that make up the image, such as color, brightness, and linear components that appear in the image, different from the surroundings. Therefore, using the color of the original image P (Color), the intensity of the original image P (Intensity), and the direction of the linear component that appears in the original image P (Orientation), the characteristics of each part in the image The degree of difference from the feature of the part located at is obtained, and the part with a large degree of difference is considered to be the attention area that is visually noticed. Specifically, using the method proposed by Itti and Koch et al., It is possible to automatically extract a visually noticeable area of interest (for example, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 20 , NO. 11, NOVEMBER 1998 “A Model of Saliency-Based Visual Action for Rapid Scene Analysis”, Laurent Itti, Christof Koch and Emst Neibur).

  A flow of processing for extracting a region of interest using this method will be described with reference to FIG.

  First, the original image P is subjected to filtering processing to generate an image representing brightness and an image of color components divided into a plurality of color components (Step 1).

  First, a lightness image I is generated from the original image P, and further, a Gaussian pyramid of the lightness image I is generated. An image of each layer of this Gaussian pyramid is assumed to be I (σ) (σ represents a pixel scale, and σ∈ [0..8]).

  Next, the original image P is divided into four color component images R (red), G (green), B (blue), and Y (yellow). Further, four types of Gaussian pyramid are generated from these images R, G, B, and Y, and the images of each layer are defined as R (σ), G (σ), B (σ), and Y (σ).

  Therefore, to what extent the characteristics of each part on the original image P differ from the surrounding parts from these images I (σ), R (σ), G (σ), B (σ), Y (σ). A feature amount map to be expressed is generated (Step 2).

A place where the brightness of a certain part on the screen is different from the brightness of the surrounding area is a place where a dark part exists in a place where the surroundings are bright, or a place where a bright part exists in a place where the surroundings are dark. Therefore, how much the lightness of the central portion is different from the surrounding lightness is obtained using the image I (c) represented by fine pixels and the image I (s) represented by rough pixels. Since the value of one pixel of the rough image I (s) represents a value obtained by collecting a plurality of pixels of the fine image I (c), the value of the pixel (center portion) of the image I (c) And the value of the pixel of the image I (s) at the position corresponding to this pixel (peripheral lightness) to determine how much each part on the image differs from the surrounding (for example, , Center-Surround method). For example, the scale of the image I (c) represented by fine pixels is cε {2, 3, 4}, and the scale of the image I (s) represented by rough pixels is s = c + δ (δε {3, 4}), a brightness feature map M _I (c, s) is obtained. The lightness feature map M _I (c, s) is expressed by the following equation (1).

Similarly, for each color component, a feature map is generated from R (σ), G (σ), B (σ), and Y (σ). A place where the color of a certain part on the screen is detected as different from the surrounding color can be found from a combination of colors (opposite colors) positioned in opposite directions in the hue circle. For example, the feature map M _RG (c, s) is acquired from the combination of red / green and green / red, and the feature map M _BY (c, s) is acquired from the combination of blue / yellow and yellow / blue. This color feature map is expressed by the following equations (2) and (3).

Further, with respect to the direction of the linear component appearing on the image, the part where the difference between the direction of the linear component appearing in each part and the surrounding linear component is detected detects the direction of the linear component from the brightness image I. It can be found using a Gabor filter. A Gabor filter is used for images in each layer of I (σ) to detect a linear component in each direction of θ∈ {0 °, 45 °, 90 °, 135 °}, and a feature map M _O (c, s, θ). The feature map in this direction is expressed as the following equation (4).

  If c∈ {2, 3, 4} and s = c + δ (δ∈ {3,4)}, there are 6 brightness feature maps, 12 color feature maps, and direction feature maps. Twenty-four maps are obtained, and these maps are viewed comprehensively to extract a region of interest that is visually noted.

These 42 pieces of feature maps _{M I, M RG, M BY} , M O is due to differences in information differences and extraction of the dynamic range, which does not appear the parts and so large that those surrounding the difference appears greater There is. Therefore, if the determination is made using the values of the 42 feature maps M _I , M _RG , M _BY , and M _O as they are, the feature map information with a small difference is reflected by the feature map having a large difference. May not be. Therefore, these 42 amino feature maps _{M I, M RG, M BY} , combined after normalizing _{M O,} preferably so as to extract a region of interest.

Specifically, for example, a brightness element-specific attention map M ^C _I obtained by standardizing and combining six brightness characteristic maps M _I (c, s) is acquired, and a color feature map M _RG (c, _{s), M bY (c,} 12 pieces to get the color element by saliency map ^M _{C C} of the combination by normalizing the s), feature map _M O (c relates direction, s, 24 pieces of theta) of to obtain the orientation of the elemental saliency map ^M _{C O} which is combined normalized (Step3). Further, the attention level map M ^S representing the distribution of the attention level of each part of the original image P is obtained by linearly combining the element-specific attention level maps M ^C _I , M ^C _C and M ^C _O (Step 4). A region where the degree of attention exceeds a predetermined threshold is extracted as a region of interest (Step 5).

Further, when extracting the region of interest, the color, brightness, and original image of the original image P are changed so that the influence of the color, brightness, and inclination of the linear component that appears in the original image P is different from the surroundings. The attention area to be extracted can be changed by changing the degree of inclination of the linear component appearing in P and the weighting degree weighted for each degree. For example, the region of interest ROI extracted can be changed by changing the weight when linearly combining the element-specific attention maps M ^C _I , M ^C _C , and M ^C _O. Alternatively, when the element-specific attention maps M ^C _I , M ^C _C , and M ^C _O are acquired, the feature map M _I (c, s) for each brightness, the color feature map M _RG (c, s), M _BY (c, s), to alter the effect of each map feature with respect to the direction map _{M O (c, s, θ} ), each feature map _{M I (c, s),} M RG (c, s), The weights for M _BY (c, s) and M _O (c, s, θ) may be changed.

  The attention area extraction unit 11 may further include an object determination unit (not shown).

  The object discriminating means is an object based on the original image P (landscape, night view, ground, sea, snow, mountain, flower, sunset, party, person (including face), baby, sports, pet, fish, structure, etc.) Is to analyze the type of.

  As an example of the object analysis method in the object discrimination means, for example, the method of Japanese Patent Application Laid-Open No. 2005-310123 (hereinafter referred to as Reference Document 1) can be used for the image processing apparatus of the present invention. The method of Reference 1 considers the tendency of the arrangement of characteristic portions corresponding to a desired specific object in the target original image, and also considers the variation of their position and area ratio, and various specific objects. A plurality of classifiers are used for selecting a specific object for selecting an image, and the recognition of a specific object is indicated by the threshold of the specific classifier or the sum of the thresholds. It is possible to determine the certainty of an object that is a specific object calculated from the threshold of a specific classifier or the sum of a plurality of thresholds.

  Also, the technique disclosed in Japanese Patent Application Laid-Open No. 2005-122720 (hereinafter referred to as Reference Document 2) can be used for the image processing apparatus of the present invention. In the technique of Reference 2, reference data that defines the type of feature quantity used for identifying an object and the identification condition is stored in a memory for each of a plurality of objects that can be designated as specific objects. All feature quantities used for identification of a plurality of objects that can be designated as specific objects are derived from the original image to be selected, referring to the type of feature quantity defined by the reference data according to the designated specific object, Those feature values are derived from the original image to be selected. The identification condition in the reference data is referred to based on the derived feature amount, and it is shown whether or not the original image to be identified includes the image of the specific object based on whether or not it is a predetermined threshold value.

  In addition, the certainty factor of an object that is a specific object calculated from the predetermined threshold or the identification condition can be obtained.

  Furthermore, the object discriminating means can also detect a person's face from the original image P. Specifically, it is possible to detect a region having a facial feature included in the face (for example, having skin color, having eyes, or having a facial shape) as the facial region.

  For example, the technique disclosed in Japanese Patent Application Laid-Open No. 2006-202276 (hereinafter referred to as Reference Document 3) can be used for the image processing apparatus of the present invention. In the method of Reference 3, the face tracking is performed by updating the weights sequentially when re-sampling a known method or learning data such as motion vector and feature point detection. It is conceivable that a machine learning method based on Adaboost, which is a method of creating an integrated learning machine by adding weights to learning machines and adding them, is used. For example, the average frame model is fitted into an image showing the face of the actual original image P, and the position of each landmark on the average frame model is moved so as to match the position of the corresponding landmark detected from the face. When constructing a face frame model by deforming the average frame model, a luminance profile at points on a plurality of sample images that are known to be predetermined landmarks by a machine learning method, and the landmarks The landmark is shown from the image showing the face using the discriminator obtained by performing the learning on the luminance profiles at the points on the plurality of sample images that are not known and the discrimination condition for each discriminator. This is a method for detecting the position of a point.

  Further, the certainty factor of the object that is the specific object calculated from the identification condition can be obtained.

  Further, the technique disclosed in Japanese Patent Application Laid-Open No. 2004-334836 (hereinafter referred to as Reference Document 4) can be used for the image processing apparatus of the present invention. The technique of Reference 4 cuts out image data of a certain size from the original image P, compares the cut-out image data with the matching data of the feature portion image data, and the feature portion image is included in the processing target image. This is a technique that uses a feature part extraction method of image data for detecting whether or not it exists. Note that the technique disclosed in Japanese Patent Application Laid-Open No. 2007-11970 (hereinafter referred to as Reference Document 5) is applied to the image processing apparatus of the present invention, and an animal face or the like is detected as a specific subject in addition to the human face area in the through image data. May be.

  The method for detecting the face described above can detect not only the front face but also the side face by the above method.

  The attention area extraction unit 11 may calculate a new attention level from the certainty factor calculated by the object determination unit and the attention level. For example, a new attention level may be calculated from the average value of the certainty level and the attention level, or a new attention level may be calculated from a value obtained by combining the certainty level and the attention level at a predetermined ratio.

  Also, when an object on the original image P is designated (for example, clicked with the mouse) by the instruction means 14, the degree of attention of the object may be set high.

  Further, the object may be analyzed by the object discriminating means with the peripheral area including the attention degree extracted by the attention area extracting means 11 described above as a search range.

  Thereby, the calculation load of the object discrimination means can be reduced.

  The trimming rule storage unit 15 stores a trimming rule obtained by learning composition information of a predetermined partial image cut out from a predetermined original image.

  The trimming rule storage means 15 stores trimming rules obtained by learning composition information of a predetermined partial image cut out from a predetermined original image in the past, for each partial image or a predetermined number or more. This is obtained by learning composition information for each partial image Q.

  When the trimming rule storage unit 15 determines to approximate the learning unit (not shown) that learns the cut out partial image Q, the object that appears in the original image P to be used, and the object that appears in the predetermined original image, the above approximation Then, a search means (not shown) for searching for a trimming rule R related to a partial image cut out from the predetermined original image determined to be included is provided.

  When a plurality of trimming rules R are extracted by the search means, the trimming rules R may be generated by combining the trimming rules R with an average or a predetermined ratio.

  As a trimming rule, there are rules based on one or more pieces of information on the type, position, size, and orientation of an object appearing in a predetermined partial image.

  For example, an image including an attention area in which the above-described attention level / new attention level is equal to or less than a predetermined value may be specified as the partial image Q.

  In addition, the trimming rule adaptively defines the cutout range on the original image P based on the composition, the above-described attention level / new attention level, and the type of the object determined by the object determination unit. You may prescribe how to create.

  In addition, the trimming rule sets the margin above the head when the margin of the person determined by the object determination unit is greatly reflected on how much margin is left outside the attention region extracted by the attention region extraction unit 11. It may be defined how the partial image Q is created with respect to how much is left, the orientation of the face determined by the object determining means, the balance of the left and right margins, and the like.

  Further, the trimming rule determines that the partial image Q is related to how to leave the upper, lower, left, and right margins as the partial image Q when it is determined that the face is determined by the object determining means exceeds a predetermined number. You may prescribe how to create.

  Further, the trimming rule determines whether the partial image Q is placed at the end point or the center of the predetermined point of the partial image Q where the predetermined point of the attention area / new attention area having a high attention degree described above is placed. You may prescribe how to create.

  Also, the trimming rule determines how much empty area is left from the person and the sky determined by the object determining means, or how much empty area is left from the original image P that is only the landscape determined by the object determining means. Or whether the area of the flower field remains or how much of the body area of the person determined by the object determination means remains. It may be defined how the partial image Q is created.

  In addition, the trimming rule determines how much the image is enlarged when the face discriminated by the object discriminating means is small, how to create the partial image Q in consideration of the cut-out aspect ratio, etc. It may be specified.

  The trimming rule R may be prepared for each user who uses the trimming rule R.

  When accessing the image processing apparatus 1, an access management means (not shown) is provided, and the user can be identified by managing which user has accessed using an ID card or a password. With this function, the trimming rule R for each user can be stored.

  The partial image creation means 12 cuts out an image within a predetermined range including the attention area extracted by the attention area extraction means 11 from the original image P based on the trimming rule R stored in the trimming rule storage means 15. Thus, the partial image Q is created.

  The range of the image to be cut out is determined according to the image size of the partial image Q to be prepared, the resolution of the display means, and the like. For example, when the partial image Q is prepared with 150 × 150 pixels, the trimming rule R is referred to, and an image cut out with 150 × 150 pixels so as to include the region of interest from the original image P is set as the partial image Q. When extracting the attention area, it is preferable to be able to extract various attention areas by changing the influence of the color, brightness, and inclination of the linear component appearing in the original image that are different from the surroundings.

  For example, the partial image creation means 12 refers to the trimming rule R and calculates the region to be cut out from the arrangement of the attention area in the original image P, the attention level / new attention level, and the object type. When a plurality of cutout areas can be calculated, priority may be given based on the arrangement of the attention area, the attention level / new attention level, the object type, and the like. A partial image Q is created based on the priority.

  Further, the partial image creating means 12 is based on the trimming rule R as the order in which the partial image Q is cut out, and attentions corresponding to the attention level / new attention level in descending order of attention level and new attention level. A partial image including a region may be cut out.

  Further, the partial image creating means 12 cuts out the partial images Q including the attention area in the order of the attention areas located from the upper left to the lower right of the original image P based on the trimming rule R as the order of cutting out the partial images Q. Also good.

  Further, the partial image creation means 12 may cut out the partial images Q including the attention area in the order of the attention areas located from the center to the periphery of the original image P based on the trimming rule R as the order of cutting out the partial images Q. good.

  Further, the partial image creating unit 12 sequentially selects the partial images Q from the region including the face or the attention region determined by the object type determining unit of the original image P based on the trimming rule R as the cutout order of the partial image Q. May be cut out.

  The partial image creating means 12 not only cuts out the partial image Q with a rectangular frame, but also sets the shape of a free-form frame that can be deformed by the instruction means 14 and cuts out with the set frame. Good.

  Further, the partial image creating unit 12 may store an optimum frame for the object determined by the object type determining unit based on the trimming rule R and cut out the frame corresponding to the determined object.

  Based on the trimming rule R, the partial image creating unit 12 may adjust the display area in consideration of the display range / resolution defined by the display unit 13.

  The display means 13 displays the partial image Q created by the partial image creation means 12 as described above, and examples thereof include a CRT and a liquid crystal monitor.

  Further, the display means 13 may display only the partial image Q that seems to be the most important based on the trimming rule R in addition to the attention level or the new attention level.

  When the image processing apparatus 1 is incorporated in a printer or digital camera, the display unit 13 may be a monitor mounted on the printer or digital camera.

  When the image processing apparatus 1 is incorporated in a mobile phone, the display unit 13 may be a monitor mounted on the mobile phone.

  Further, when the image processing apparatus 1 is incorporated in a computer, the display unit 13 may be an external monitor connected to the computer.

  Further, the display unit 13 may display the partial images Q in order from the partial image Q including the attention area / the attention area having a new attention degree.

  The instruction unit 14 described above can set the cutout range of the partial image creation unit 12 and can also instruct switching of the original image P and the partial image Q.

  Next, a flow of processing for generating the partial image Q and a flow of processing for displaying the partial image Q in the image processing apparatus 1 of the present invention will be described using the flowchart of FIG.

  First, the image processing apparatus 1 reads an original image P such as a photographed image or an illustration image and temporarily stores it in the hard disk (original image storage means 10) (S100).

The attention area extracting means 11 generates an image I representing brightness and images R, G, B, and Y divided into color components from the original image P stored in the hard disk, and further, from these, a Gaussian pyramid image I ( σ), R (σ), G (σ), B (σ), and Y (σ) are generated. Using these Gaussian pyramid image, color, brightness, based on the direction of the linear component of the original image P, and generates a saliency map M ^S calculated attention degree of each part of the original image P (S101). Attention Attention map M ^S extracts the region of or greater than a predetermined threshold value as a target area (S102).

For example, as shown in FIG. 4A, in an image in which a red road sign is photographed near the center of the image, the surrounding mountains and roads are mostly brown to gray and the color of the road sign is significantly different from the surroundings. degree of attention on the degree map M ^S appears high. Therefore, as shown in FIG. 4B, a portion having a predetermined threshold value or more is extracted as a region of interest ROI. As shown in FIG. 5 or FIG. 5A, when one red rectangle and many green rectangles are lined up in various directions, as shown in FIG. The attention degree of the portion of the rectangle of which the inclination is greatly inclined as compared with other rectangles is high, and the region with the high attention degree is extracted as the attention region ROI. The trimming rule R stored in the trimming rule storage unit 15 is read (S103). Based on the read trimming rule R, the priority of the extracted attention area ROI is assigned or finely adjusted, and the final attention is paid. A region is determined, and a rectangular region is cut out as a partial image Q so as to include the region of interest (S104).

  The partial image Q is displayed on the display means 13 (S105). For example, in an image as shown in FIG. 6A, a range surrounded by a rectangle is cut out as a partial image Q. The size of the partial image and the position of the region of interest ROI in the partial image may not always be the same. In addition, various ranges of images including the same region of interest ROI may be prepared as partial images. In this way, a part including the attention area ROI is cut out as a partial image Q from the original image P.

  Further, the attention area may be calculated by setting a new attention degree as described above from the certainty factor calculated by the object discrimination means and the attention degree.

  Note that, after calculating these attention areas, priority is given or fine adjustment is made with reference to the trimming rule R to determine the attention area.

  Here, for example, in the case of an image as shown in FIG. 6, an example in which a range surrounded by a rectangle is cut out as a partial image Q will be described.

  FIG. 6 is a diagram showing a wedding candle service, and it is possible to cut out a partial image Q as shown in ((2-1) to (2-4)) of FIG. .

Furthermore, for the partial image Q containing a high region of interest ROI and the object of attention / new attention of the saliency map M ^S, depending on the order of cut as described above, the partial image Q on the display unit 13 It may be displayed.

  If necessary, the partial image Q can be switched and displayed by receiving a response from the instruction means 14. In addition to simply changing the display, it may be controlled to move the display range, and it is also possible to switch between enlargement and reduction using zoom or the like.

  Further, as display switching, special display by fading out / fade in, a scroll function, dynamic switching, or the like may be possible.

  The above-described display switching makes it easier for the user to grasp the overall situation of the original image P.

  Here, the partial image creation unit 12 receives a response from the instruction unit 14 to the partial image Q2 cut out as shown in (2-2) of FIG. 6, thereby finely adjusting the position / magnification of the cutout range. It can be performed. For example, the partial image Q2 in which the cutout range is set as shown in (1-1) of FIG. 7 is changed to a partial image Q2 ′ shifted in the horizontal direction as shown in (1-2) of FIG. Adjustments can be made. Similarly, it is possible to shift in the vertical direction.

  Further, the partial image Q2 in which the cutout range is set as shown in (1-1) of FIG. 7 becomes the partial image Q2 ″ whose cutout range is extended as shown in (1-3) of FIG. Fine adjustments can be made.

  Further, the partial image creation means 12 receives a response from the instruction means to the partial images Q1 to Q4 cut out as shown in (2-1) to (2-4) in FIG. Can be instructed whether or not will be treated preferentially.

  As described above, the trimming rule update unit 15 receives the response instruction received from the instruction unit 14, and the composition information of the partial image created by the partial image creation unit 12 and the trimming rule R stored in the trimming rule storage unit 15. And a new trimming rule R can be updated.

  As described above in detail, the original image P is stored, the original image P is stored, and the trimming rule R obtained by learning the composition information of the predetermined partial image cut out from the predetermined original image is obtained. Based on the color and brightness of the original image P stored, and the direction of the linear component appearing in the original image P, the degree to which the feature of each part in the original image P differs from the feature of the part located around the part The attention area is extracted from the original image P according to these degrees, and based on the stored trimming rule R, an image within a predetermined range including the extracted attention area is converted into the original image P. It is possible to create a partial image Q reflecting user preferences and habits.

Schematic configuration diagram of an image processing device The figure for explaining the extraction method of the attention area A flowchart showing the flow of processing for extracting a region of interest from an original image Example of original image (part 1) Example of attention level map (1) Example of original image (Part 2) Example of attention level map (2) Examples of original images (part 3) Example of partial image (1)

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Original image storage means 11 Attention area extraction means 12 Partial image creation means 13 Display means 14 Instruction means 15 Trimming rule storage means 16 Trimming rule update means P Original image Q Partial image ROI Attention area R Trimming rule

Claims (9)

Original image storage means for storing the original image;
Trimming rule storage means for storing a trimming rule obtained by learning composition information of a predetermined partial image cut out from a predetermined original image;
Based on the color and brightness of the original image and the direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the part located around the part is obtained. , Attention area extraction means for extracting the attention area from the original image according to these degrees;
A part that creates a partial image by cutting out an image within a predetermined range including the attention area extracted by the attention area extraction means from the original image based on the trimming rules stored in the trimming rule storage means An image processing apparatus comprising an image creating unit.   2. A trimming rule update unit that learns composition information of the predetermined partial image and composition information of the partial image created by the partial image creation unit, and updates the trimming rule. The image processing apparatus described.   3. The image according to claim 1, wherein the trimming rule is a trimming rule based on at least one of information on a type, a position, a size, and an orientation of an object appearing in the predetermined partial image. Processing equipment.   The image processing apparatus according to claim 1, further comprising display means for displaying the partial image created by the partial image creation means. The attention area extracting means calculates a degree of attention to each of the extracted attention areas;
The image processing apparatus according to claim 4, wherein the display unit sequentially displays partial images including a region of interest having a high degree of attention. An instruction means for instructing to change the cutout range of the partial image created by cutting out from the original image by the partial image creating means;
6. The image processing apparatus according to claim 1, wherein the partial image creation unit creates a partial image in which a clipping range designated by the instruction unit is changed. The region of interest extraction means
The image processing apparatus according to claim 1, further comprising: an object determination unit that analyzes the original image and determines whether or not there is a specific type of object that appears in the original image. Remember the original image,
Storing a trimming rule obtained by learning composition information of a predetermined partial image cut out from a predetermined original image;
Based on the color and brightness of the original image and the direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the part located around the part is obtained. , To extract a region of interest from the original image according to these degrees,
An image processing method comprising: creating a partial image by cutting out an image within a predetermined range including the extracted attention area from the original image based on the stored trimming rule. On the computer,
The ability to store original images,
A function for storing a trimming rule obtained by learning composition information of a predetermined partial image cut out from a predetermined original image;
Based on the color and brightness of the original image and the direction of the linear component appearing in the original image, the degree of difference between the characteristics of each part in the original image and the characteristics of the part located around the part is obtained. , A function of extracting a region of interest from the original image according to these degrees,
A program for executing a function of creating a partial image by cutting out an image within a predetermined range including the extracted region of interest from the original image based on the stored trimming rule .

Images