CN107146198B - Intelligent photo cutting method and device - Google Patents
Intelligent photo cutting method and device Download PDFInfo
- Publication number
- CN107146198B CN107146198B CN201710256861.XA CN201710256861A CN107146198B CN 107146198 B CN107146198 B CN 107146198B CN 201710256861 A CN201710256861 A CN 201710256861A CN 107146198 B CN107146198 B CN 107146198B
- Authority
- CN
- China
- Prior art keywords
- aesthetic
- image
- sliding window
- photo
- optimal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000008569 process Effects 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000011156 evaluation Methods 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract description 3
- 238000013210 evaluation model Methods 0.000 description 24
- 238000013527 convolutional neural network Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 230000003796 beauty Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 238000009966 trimming Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent photo cutting method and device. The method comprises the following steps: acquiring a photo image of the intelligent equipment; generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores; and cutting the photo image according to the coordinates of the optimal aesthetic feeling cutting area. The intelligent photo cutting method and the intelligent photo cutting device realize intelligent cutting of the photo through the image aesthetic evaluation technology, so that the photo which is more suitable for an application scene is cut from the aesthetic angle; the invention is more universal in the type of the applicable photos, convenient in operation and high in efficiency, so that the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an intelligent photo clipping method and device.
Background
Along with the continuous enhancement of the photographing performance of the intelligent terminal equipment, people are interested in recording beauty moments in life at will and share the beauty moments in a social network. But generally, the pictures taken by people are not suitable for being directly uploaded to the internet, and some post-processing is needed. For example, when taking a picture, the finger carelessly blocks the edge of the lens or the edge of the lens is mistakenly entered by a person; only part of the picture in the photo is required to be intercepted for highlighting; there is a nice rectangle self-photograph, while the micro-letter head portrait requires a square. Under the conditions, photo cutting is usually needed, and at present, a photo cutting function provided by mainstream photo/picture trimming applications is to manually edit a screenshot through a free/fixed ratio, but as a common user usually does not have professional photographic knowledge such as composition, the aesthetic quality of the manually cut photo cannot be guaranteed, and the ratio of the manually cut photo may not meet the application requirements. The invention with the application number of CN201410681308.7 carries out face detection on the shot image to obtain a face detection frame; taking the central point of the face detection frame as an original point, and according to the area ratio of 10: 7 producing an aspect ratio of 4:3, a rectangular cutting frame; and cutting out the parts except the rectangular cutting frame in the image. The invention has better effect on cutting the main photos of the face such as the identification photo and the like from the angle of face detection, but can not realize intelligent cutting for a large number of common photos.
Disclosure of Invention
In view of the above problems, the present invention provides a method and an apparatus for intelligently cropping a photo.
The invention provides an intelligent photo clipping method, which comprises the following steps:
acquiring a photo image of the intelligent equipment;
generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores;
and cutting the photo image according to the coordinates of the optimal aesthetic feeling cutting area.
The invention also provides an intelligent photo cutting device, which comprises: the image acquisition module, the calculation module and the cutting module are as follows:
the image acquisition module is used for acquiring a photo image of the intelligent equipment;
the calculation module is used for generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores;
and the cutting module is used for cutting the photo image according to the coordinates of the optimal aesthetic feeling cutting area.
The invention has the following beneficial effects:
the intelligent photo cutting method and the intelligent photo cutting device realize intelligent cutting of the photo through the image aesthetic evaluation technology, so that the photo which is more suitable for an application scene is cut from the aesthetic angle; the invention is more universal in the type of the applicable photo, convenient in operation and high in efficiency. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
Drawings
FIG. 1 is a flow chart of a photo intelligent cropping method of an embodiment of the method of the present invention;
FIG. 2 is a schematic structural diagram of an intelligent photo cropping device according to an embodiment of the present invention;
FIG. 3 is a flowchart of a photo intelligent cropping method of example 1 of the present invention;
FIG. 4 is a flowchart of a photo intelligent cropping method of example 2 of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problem that the aesthetic quality of a photo cut by the prior art cannot be guaranteed or a large number of common photos except for photos with a human face as a main part cannot be intelligently cut, the invention provides an intelligent photo cutting method and an intelligent photo cutting device, and the invention is further described in detail below by combining the attached drawings and the embodiment. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
According to an embodiment of the method of the present invention, an intelligent photo cropping method is provided, fig. 1 is a flowchart of the intelligent photo cropping method according to the embodiment of the method of the present invention, and as shown in fig. 1, the intelligent photo cropping method according to the embodiment of the method of the present invention includes the following processing:
step 101, obtaining a photo image of the intelligent device.
Specifically, the smart device includes a personal computer, a smart phone, a tablet computer, smart glasses, and other devices having a camera and a data processing function.
Step 102, generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores. Specifically, generating a plurality of sub-regions in the photo image includes traversing the photo image with a preset sliding window to obtain a plurality of sub-regions.
Wherein, step 102 includes the following two technical solutions. The first technical scheme comprises the following steps: traversing the photo image by using a first sliding window, obtaining a plurality of aesthetic scores corresponding to the first sliding window in the traversing process by using a preset aesthetic evaluation model, selecting the highest aesthetic score, and taking the position corresponding to the highest aesthetic score as the best aesthetic cutting area; wherein, the first sliding window selects one or more of the following: an original proportion sliding window, a standard proportion sliding window and a preset proportion sliding window.
More specifically, when the first sliding window includes the sliding window with the preset ratio, traversing the photo image using the first sliding window, and obtaining a plurality of aesthetic scores corresponding to the first sliding window in the traversing process by using a preset aesthetic evaluation model, including the following steps:
traversing the photo image by using a sliding window with a preset proportion, and obtaining a plurality of aesthetic scores corresponding to the first sliding window in the traversing process by using a preset aesthetic evaluation model.
More specifically, when the first sliding window comprises an original proportion sliding window and a standard proportion sliding window, traversing the photo image by using the first sliding window, obtaining a plurality of aesthetic scores corresponding to the first sliding window in the traversing process by using a preset aesthetic evaluation model, and selecting the highest aesthetic score, wherein the scheme comprises a scheme A and a scheme B.
Specifically, the scheme A comprises the following steps:
traversing the photo image by using an original proportional sliding window, and obtaining a plurality of aesthetic scores corresponding to the original proportional sliding window in the traversing process by using a preset aesthetic evaluation model;
traversing the photo image by using a standard proportion sliding window, and obtaining a plurality of aesthetic scores corresponding to the standard proportion sliding window in the traversing process by using a preset aesthetic evaluation model;
and selecting the highest aesthetic score from a plurality of aesthetic scores corresponding to the original proportion sliding window and a plurality of aesthetic scores corresponding to the standard proportion sliding window.
Specifically, the scheme B comprises the following steps:
traversing the photo image by using an original proportional sliding window, and obtaining a plurality of aesthetic scores corresponding to the original proportional sliding window in the traversing process by using a preset aesthetic evaluation model;
selecting the highest aesthetic score in a plurality of aesthetic scores corresponding to the original proportion sliding window, and obtaining the optimal aesthetic sense cutting area under the original proportion according to the coordinate of the original proportion sliding window corresponding to the highest aesthetic score;
calculating the maximum intersection ratio of the standard proportion sliding window and the optimal aesthetic feeling cutting area under the original proportion, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the standard proportion;
and selecting one of the optimal aesthetic feeling cutting area under the original proportion or the optimal aesthetic feeling cutting area under the standard proportion as the optimal aesthetic feeling cutting area of the image according to the instruction.
More specifically, in the scheme a and the scheme B, the original proportional sliding window and the standard proportional sliding window may be selected as several scales. For example, when the original scale sliding window selects 3 scales, the scaling ratio of each scale is 1.2, and the length of the horizontal side of the minimum scale sliding window is 1/4 of the long side of the horizontal side of the original image, it means that the original scale image selects the following three scales: the size is 0.25 times the size of the original image, 0.25 × 1.2 times the size of the original image, and 0.25 × 1.2 × 1.2 times the size of the original image.
The second technical solution of step 102 comprises the following steps: traversing the photo image by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process, taking the aesthetic scores as aesthetic quality values of coordinates of center points of the second sliding windows in the traversing process, and generating an aesthetic quality map according to the aesthetic quality values; the second sliding window is the original proportion of the photo image, the second sliding window is smaller than or equal to the first sliding window, and the step length of the second sliding window is smaller than or equal to the first sliding window;
obtaining a highest energy region in the aesthetic quality map by using a preset clustering algorithm, and taking a minimum circumscribed rectangle of the highest energy region as an optimal aesthetic region of the image;
and calculating the maximum intersection ratio of the window of the required clipping proportion and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling clipping area.
Specifically, when the first sliding window includes an original proportional sliding window and a standard proportional sliding window, calculating a maximum intersection ratio of the first sliding window and the optimal aesthetic feeling area of the image, and taking a position where the maximum intersection ratio is generated as the optimal aesthetic feeling clipping area, including the following steps:
calculating the maximum intersection ratio of the original proportion sliding window and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the original proportion;
calculating the maximum intersection ratio of the standard proportion sliding window and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the standard proportion;
and selecting one of the optimal aesthetic feeling cutting areas under the original proportion and the standard proportion, wherein the center of the optimal aesthetic feeling cutting area is closest to the center of the optimal aesthetic feeling cutting area of the image, and the selected one is used as the optimal aesthetic feeling cutting area of the image.
As another embodiment of the method of the present invention, the method further includes the following steps: acquiring a seed point of a designated target cutting area; further, traversing the photo image by using a preset sliding window to obtain a plurality of aesthetic scores corresponding to the preset sliding window in the traversing process, including the following steps:
traversing the seed point neighborhood of the target cutting area by using a preset sliding window to obtain a plurality of aesthetic scores corresponding to the preset sliding window in the traversing process.
And 103, cutting the photo image according to the coordinates of the optimal aesthetic feeling cutting area to obtain the optimal cutting area of the photo image.
The intelligent photo cutting method provided by the embodiment of the invention realizes intelligent cutting of the photo through the image aesthetic evaluation technology, so that the photo which is more suitable for an application scene is cut from an aesthetic angle; the invention is more universal in the type of the applicable photo, convenient in operation and high in efficiency. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
Corresponding to the embodiment of the method of the present invention, the present invention further provides an intelligent photo clipping device, fig. 2 is a schematic structural diagram of the intelligent photo clipping device according to the embodiment of the present invention, and as shown in fig. 2, the intelligent photo clipping device according to the embodiment of the present invention includes: the image acquisition module 20, the calculation module 22, and the cropping module 24 are described in detail below.
Specifically, the image obtaining module 20 is configured to obtain a photo image of the smart device.
The calculating module 22 is configured to generate a plurality of sub-regions in the photo image, calculate an aesthetic score of the plurality of sub-regions, and obtain an optimal aesthetic sense clipping region according to the aesthetic score.
Specifically, generating a plurality of sub-regions in the photo image includes traversing the photo image with a preset sliding window to obtain a plurality of sub-regions.
The calculation module is specifically configured to: traversing the photo image by using a first sliding window, obtaining a plurality of aesthetic scores corresponding to the first sliding window in the traversing process by using a preset aesthetic evaluation model, selecting the highest aesthetic score, and taking the position corresponding to the highest aesthetic score as the best aesthetic cutting area; wherein, the first sliding window selects one or more of the following: an original proportion sliding window, a standard proportion sliding window and a preset proportion sliding window;
or traversing the photo image by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process, taking the aesthetic scores as the aesthetic quality values of the coordinates of the center point of the second sliding window in the traversing process, and generating an aesthetic quality map according to the aesthetic quality values; the second sliding window is the original proportion of the photo image, the second sliding window is smaller than or equal to the first sliding window, and the step length of the second sliding window is smaller than or equal to the first sliding window;
obtaining a highest energy region in the aesthetic quality map by using a preset clustering algorithm, and taking a minimum circumscribed rectangle of the highest energy region as an optimal aesthetic region of the image;
and calculating the maximum intersection ratio of the window of the required clipping proportion and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling clipping area.
More specifically, when the first sliding window includes an original proportional sliding window and a standard proportional sliding window, the calculating module is specifically configured to:
traversing the photo image by using an original proportional sliding window, and obtaining a plurality of aesthetic scores corresponding to the original proportional sliding window in the traversing process by using a preset aesthetic evaluation model;
traversing the photo image by using a standard proportion sliding window, and obtaining a plurality of aesthetic scores corresponding to the standard proportion sliding window in the traversing process by using a preset aesthetic evaluation model;
selecting the highest aesthetic score from a plurality of aesthetic scores corresponding to the original proportion sliding window and a plurality of aesthetic scores corresponding to the standard proportion sliding window;
or traversing the photo image by using an original proportional sliding window, and obtaining a plurality of aesthetic scores corresponding to the original proportional sliding window in the traversing process by using a preset aesthetic evaluation model;
selecting the highest aesthetic score in a plurality of aesthetic scores corresponding to the original proportion sliding window, and obtaining the optimal aesthetic sense cutting area under the original proportion according to the coordinate of the original proportion sliding window corresponding to the highest aesthetic score;
calculating the maximum intersection ratio of the standard proportion sliding window and the optimal aesthetic feeling cutting area under the original proportion, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the standard proportion;
selecting one of the optimal aesthetic feeling cutting area under the original proportion or the optimal aesthetic feeling cutting area under the standard proportion as an image optimal aesthetic feeling cutting area according to the instruction;
or calculating the maximum intersection ratio of the first sliding window and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area, wherein the method comprises the following steps:
calculating the maximum intersection ratio of the original proportion sliding window and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the original proportion;
calculating the maximum intersection ratio of the standard proportion sliding window and the optimal aesthetic feeling area of the image, and taking the position generating the maximum intersection ratio as the optimal aesthetic feeling cutting area under the standard proportion;
and selecting one of the optimal aesthetic feeling cutting areas under the original proportion and the standard proportion, wherein the center of the optimal aesthetic feeling cutting area is closest to the center of the optimal aesthetic feeling cutting area of the image, and the selected one is used as the optimal aesthetic feeling cutting area of the image.
And the cropping module 24 is configured to crop the photo image according to the coordinates of the optimal aesthetic feeling cropping area, so as to obtain an optimal cropping area of the photo image.
The preset aesthetic evaluation model is obtained through training, and the used machine learning method comprises a convolutional neural network, a limited Boltzmann machine, a deep confidence network and the like. Furthermore, the embodiment of the apparatus of the present invention further includes a model training module, where the model training module is configured to:
acquiring a number of images of known aesthetic scores;
and training the preset aesthetic evaluation model by using a plurality of images with known aesthetic scores to obtain the trained aesthetic evaluation model.
The intelligent photo clipping method and system have the following beneficial effects:
1. the invention can intelligently cut the photos with standard proportion from the angle of optimal aesthetics, solves the problems that the photos are cut manually and the proportion of the cut photos is not standard and the photos are possibly low in aesthetic feeling, and meets the requirement of practical application.
2. And the interactive function is supported, and the optimal aesthetic photo can be cut out in the neighborhood range of the manually specified point so as to better meet the customized cutting requirement of the user.
To illustrate embodiments of the present invention in more detail, examples 1 and 2 are given.
As shown in fig. 3, example 1 includes the following steps:
step S1: and acquiring a photo image from the intelligent equipment photo album.
Step S2: the system judges whether a certain coordinate in the image is manually appointed to be used as a seed point of the target cutting area.
Step S3: if the target clipping region seed point is not manually designated, the image is traversed by using multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) sliding windows, and aesthetic scores in the windows are calculated by using a convolutional neural network aesthetic evaluation model. Selecting a position corresponding to the highest aesthetic score of the original proportion and the standard proportion window as an optimal aesthetic cutting area under the original proportion and the standard proportion;
and if the manually-specified seed points of the target clipping region exist, traversing the neighborhood of the seed points by using multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) sliding windows to ensure that each sliding window contains the seed points, and calculating by using a convolutional neural network aesthetic evaluation model to obtain an aesthetic score in each window. And selecting the position corresponding to the highest aesthetic score of the original proportion and the standard proportion window as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
Step S4: and the intelligent equipment cuts the original image according to the coordinates of the optimal aesthetic feeling cutting area under the original proportion and the standard proportion respectively to obtain the optimal cutting area of the image under the original proportion and the standard proportion. And (4) providing the multi-scale image cutting result for the user to select. If the optimal aesthetic feeling clipping area coordinate is the original image coordinate, clipping is not needed.
In step S1, a photo image is acquired from the smart device album.
In step S2, the system determines whether a certain coordinate in the image is manually designated as a target clipping region seed point.
In step S3, if a coordinate in the image is not manually designated as a seed point of the target clipping region, the image is traversed by a step size using multi-scale original scale and standard scale (1:1, 5:3, 4:3, 5:4, 16:9, 3:2, 7:5) sliding windows, and an aesthetic score (1-10 points) in each window is calculated by using a convolutional neural network aesthetic evaluation model. And selecting the position corresponding to the highest aesthetic score of the original proportion and the standard proportion window as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
It should be noted that the aesthetic evaluation model is obtained through training, and the subjectivity and one-sidedness of the manual rule model are avoided. When the optimal aesthetic feeling cutting area under the original proportion and the multiple standard proportions (1:1, 5:3, 4:3, 16:9, 3:2 and 7:5) is detected, the sliding window under all the proportions is used for traversing the image, the image can also be only traversed once by using the original proportion window to obtain the optimal aesthetic feeling cutting area under the original proportion, then the maximum intersection ratio of the multiple standard proportion windows and the optimal aesthetic feeling cutting area under the original proportion is calculated, and the position generating the maximum intersection ratio is used as the optimal aesthetic feeling cutting area under the standard proportion.
Preferably, the sliding window has 7 dimensions, each scaling ratio is 1.2, the length of the horizontal side of the minimum dimension sliding window is 1/4 of the long side of the original image in the horizontal direction, the horizontal direction step size of the sliding window is 1/16 of the length of the horizontal side of the original image, and the vertical direction step size of the sliding window is 1/16 of the length of the vertical side of the original image.
In step S3, if a coordinate in the image is manually designated as a seed point of the target clipping region, the sliding windows of the multi-scale original scale and the standard scale (1:1, 5:3, 4:3, 5:4, 16:9, 3:2, 7:5) are used to traverse the seed point neighborhood, and each sliding window is ensured to contain the seed point. And (3) calculating by using a convolutional neural network aesthetic evaluation model to obtain aesthetic scores (1-10 points) in each window, and selecting the position corresponding to the highest aesthetic score of the original proportion and the standard proportion window as the optimal aesthetic sense cutting area under the original proportion and the standard proportion.
It should be noted that the aesthetic evaluation model is obtained through training, and the subjectivity and one-sidedness of the manual rule model are avoided. When the optimal aesthetic feeling cutting area under the original proportion and the multiple standard proportions (1:1, 5:3, 4:3, 16:9, 3:2 and 7:5) is detected, the sliding window under all the proportions is used for traversing the image, the image can also be only traversed once by using the original proportion window to obtain the optimal aesthetic feeling cutting area under the original proportion, then the maximum intersection ratio of the multiple standard proportion windows and the optimal aesthetic feeling cutting area under the original proportion is calculated, and the position generating the maximum intersection ratio is used as the optimal aesthetic feeling cutting area under the standard proportion.
Preferably, the sliding window has 7 dimensions, each scaling ratio is 1.2, the length of the horizontal side of the minimum dimension sliding window is 1/4 of the long side of the original image in the horizontal direction, the horizontal direction step size of the sliding window is 1/16 of the length of the horizontal side of the original image, and the vertical direction step size of the sliding window is 1/16 of the length of the vertical side of the original image. Preferably, an object recognition algorithm is used in the seed point neighborhood to ensure that the seed point neighborhood contains the contour of the seed point object. Many mature object recognition algorithms exist in the prior art, and are not described herein.
In step S4, the smart device clips the original image according to the coordinates of the optimal aesthetic feeling clipping area at the original scale and the standard scale, respectively, to obtain the optimal clipping area of the image at the original scale and the standard scale. And (4) providing the multi-scale image cutting result for the user to select. If the optimal aesthetic feeling clipping area coordinate is the original image coordinate, clipping is not needed.
It should be noted that, the user can also specify the required cropping ratio, and the method and the system will crop out the optimal image area according to the cropping ratio specified by the user.
As shown in fig. 4, example 2 includes the following steps:
step S1: and acquiring a photo image from the intelligent equipment photo album.
Step S2: an aesthetic quality map of the photographic image is generated using a convolutional neural network aesthetic evaluation model.
Step S3: the system judges whether a certain coordinate in the image is manually appointed to be used as a seed point of the target cutting area.
Step S4: if a certain coordinate in the image is not manually designated as a seed point of the target cutting area, a clustering algorithm is used for obtaining the area with the highest energy in the aesthetic quality map, and the coordinate of the minimum circumscribed rectangle of the area with the highest energy is obtained and corresponds to the coordinate of the optimal aesthetic area of the image. And respectively calculating the coordinate position of the maximum intersection ratio of the multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) window and the optimal aesthetic feeling area of the image, and using the coordinate position as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
If a certain coordinate in the image is manually appointed to serve as a seed point of the target cutting area, a clustering algorithm is used for obtaining the highest energy area in the aesthetic quality map in the neighborhood of the seed point, the range of the highest energy area is ensured to contain the seed point, and the coordinate of the minimum circumscribed rectangle of the highest energy area is obtained and corresponds to the coordinate of the optimal aesthetic feeling area of the image. And respectively calculating the coordinate position of the maximum intersection ratio of the multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) window and the optimal aesthetic feeling area of the image as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
Step S5: and the intelligent equipment cuts the original image according to the coordinates of the optimal aesthetic feeling cutting area under the original proportion and the standard proportion respectively to obtain the optimal cutting area of the image under the original proportion and the standard proportion. And (4) providing the multi-scale image cutting result for the user to select. If the optimal aesthetic feeling clipping area coordinate is the original image coordinate, clipping is not needed.
In step S1, a photo image is acquired from the smart device album.
In step S2, an aesthetic quality map of the photo image is generated using the convolutional neural network aesthetic evaluation model. Traversing the image by sliding windows with certain size and step length, calculating an aesthetic score (1-10 points) of each sliding window by using a convolutional neural network aesthetic evaluation model as an aesthetic quality value of the coordinates of the center point of the sliding window, and generating an aesthetic quality map as the sliding window is traversed.
It should be noted that the aesthetic evaluation model is obtained through training, and the subjectivity and one-sidedness of the manual rule model are avoided.
Preferably, the aspect ratio of the sliding window is the same as the original image, the length of the horizontal side of the sliding window is 1/4 of the horizontal side of the original image, the length of the vertical side of the sliding window is 1/4 of the vertical side of the original image, the horizontal step size of the sliding window is 1/32 of the horizontal side of the original image, and the vertical step size of the sliding window is 1/32 of the vertical side of the original image.
In step S3, the system determines whether a certain coordinate in the image is manually designated as a target clipping region seed point.
In step S4, if a certain coordinate in the image is not manually designated as a seed point of the target clipping region, a clustering algorithm is used to obtain a region with the highest energy in the aesthetic quality map, and a coordinate of a minimum bounding rectangle of the region with the highest energy is obtained, which corresponds to a coordinate of an optimal aesthetic region of the image. And respectively calculating the coordinate position of the maximum intersection ratio of the multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) window and the optimal aesthetic feeling area of the image as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
It should be noted that, when the coordinate positions of the maximum intersection ratio generated by the multi-scale original ratio and standard ratio window and the optimal aesthetic feeling area of the image are not unique, the window with the center closest to the center of the optimal aesthetic feeling area of the image is selected as the optimal aesthetic feeling clipping area.
In step S4, if a certain coordinate in the image is manually designated as a seed point of the target clipping region, a clustering algorithm is used to obtain a region with the highest energy in the aesthetic quality map in the neighborhood of the seed point, and it is ensured that the region with the highest energy includes the seed point, and a coordinate of a minimum bounding rectangle of the region with the highest energy is obtained, which corresponds to a coordinate of the region with the best aesthetic feeling of the image. And respectively calculating the coordinate position of the maximum intersection ratio of the multi-scale original proportion and standard proportion (1:1, 5:3, 4:3, 5:4, 16:9, 3:2 and 7:5) window and the optimal aesthetic feeling area of the image as the optimal aesthetic feeling cutting area under the original proportion and the standard proportion.
It should be noted that, when the coordinate positions of the maximum intersection ratio generated by the multi-scale original ratio and standard ratio window and the optimal aesthetic feeling area of the image are not unique, the window with the center closest to the center of the optimal aesthetic feeling area of the image is selected as the optimal aesthetic feeling clipping area. Preferably, an object recognition algorithm is used in the seed point neighborhood to ensure that the seed point neighborhood contains the contour of the seed point object. Many mature object recognition algorithms exist in the prior art, and are not described herein.
In step S5, the smart device clips the original image according to the coordinates of the optimal aesthetic feeling clipping area at the original scale and the standard scale, respectively, to obtain the optimal clipping area of the image at the original scale and the standard scale. And (4) providing the multi-scale image cutting result for the user to select. If the optimal aesthetic feeling clipping area coordinate is the original image coordinate, clipping is not needed.
It should be noted that, the user can also specify the required cropping ratio, and the method and the system will crop out the optimal image area according to the cropping ratio specified by the user.
The invention can automatically cut out the beauty area in the photo, provides an operation mode of user interaction, can cut out the beauty photo near the appointed position of the user, and simultaneously carries out scene classification on the photo album photo in advance, thereby improving the aesthetic evaluation accuracy. The method and the system have wide applicable range and can be widely applied to intelligent equipment.
The above description is only an example of the present invention, and is not intended to limit the present invention, and it is obvious to those skilled in the art that various modifications and variations can be made in the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (2)
1. An intelligent photo clipping method is characterized by comprising the following steps:
acquiring a photo image of the intelligent equipment;
generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores;
clipping the photo image according to the coordinates of the optimal aesthetic feeling clipping area;
the method comprises the following steps of generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores, wherein the method comprises the following steps:
traversing the photo image by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process, taking the aesthetic scores as aesthetic quality values of coordinates of center points of the second sliding windows in the traversing process, and generating an aesthetic quality map according to the aesthetic quality values; the second sliding window is the original proportion of the photo image;
obtaining a highest energy region in the aesthetic quality map by using a preset clustering algorithm, and taking a minimum circumscribed rectangle of the highest energy region as an optimal aesthetic region of the image;
calculating the maximum intersection ratio of the window with the required cutting proportion and the optimal aesthetic feeling area of the image, and taking the position with the maximum intersection ratio as the optimal aesthetic feeling cutting area;
further comprising the steps of: acquiring a seed point of a designated target cutting area;
traversing the photo image by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process, wherein the method comprises the following steps:
and traversing the seed point neighborhood of the target cutting area by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process.
2. The utility model provides a device is tailor to photo intelligence which characterized in that, includes image acquisition module, calculation module, tailors the module:
the image acquisition module is used for acquiring a photo image of the intelligent equipment;
the calculation module is used for generating a plurality of sub-regions in the photo image, calculating aesthetic scores of the sub-regions, and obtaining an optimal aesthetic sense cutting region according to the aesthetic scores;
the cutting module is used for cutting the photo image according to the coordinates of the optimal aesthetic feeling cutting area;
the calculation module is used for traversing the photo image by using a second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process, taking the aesthetic scores as the aesthetic quality values of the coordinates of the center points of the second sliding windows in the traversing process, and generating an aesthetic quality map according to the aesthetic quality values; the second sliding window is the original proportion of the photo image;
obtaining a highest energy region in the aesthetic quality map by using a preset clustering algorithm, and taking a minimum circumscribed rectangle of the highest energy region as an optimal aesthetic region of the image;
calculating the maximum intersection ratio of the window with the required cutting proportion and the optimal aesthetic feeling area of the image, and taking the position with the maximum intersection ratio as the optimal aesthetic feeling cutting area;
still include seed point and acquire the module:
the seed point acquisition module is used for acquiring seed points of a specified target cutting area;
the calculation module is configured to:
and traversing the seed point neighborhood of the target cutting area by using the second sliding window to obtain a plurality of aesthetic scores corresponding to the second sliding window in the traversing process.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710256861.XA CN107146198B (en) | 2017-04-19 | 2017-04-19 | Intelligent photo cutting method and device |
| PCT/CN2017/116058 WO2018192244A1 (en) | 2017-04-19 | 2017-12-14 | Shooting guidance method for intelligent device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710256861.XA CN107146198B (en) | 2017-04-19 | 2017-04-19 | Intelligent photo cutting method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107146198A CN107146198A (en) | 2017-09-08 |
| CN107146198B true CN107146198B (en) | 2022-08-16 |
Family
ID=59774749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710256861.XA Active CN107146198B (en) | 2017-04-19 | 2017-04-19 | Intelligent photo cutting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107146198B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107520881B (en) * | 2017-10-19 | 2019-05-07 | 合肥智慧龙图腾知识产权股份有限公司 | A kind of photo automatic cutting machine people |
| CN108062739B (en) * | 2017-11-02 | 2021-07-02 | 广东数相智能科技有限公司 | Intelligent picture clipping method and device based on main body position |
| CN108062755B (en) * | 2017-11-02 | 2020-10-02 | 广东数相智能科技有限公司 | Intelligent picture clipping method and device |
| WO2019114147A1 (en) * | 2017-12-15 | 2019-06-20 | 华为技术有限公司 | Image aesthetic quality processing method and electronic device |
| CN109146892B (en) * | 2018-07-23 | 2020-06-19 | 北京邮电大学 | Image clipping method and device based on aesthetics |
| CN109523503A (en) * | 2018-09-11 | 2019-03-26 | 北京三快在线科技有限公司 | A kind of method and apparatus of image cropping |
| CN110147833B (en) * | 2019-05-09 | 2021-10-12 | 北京迈格威科技有限公司 | Portrait processing method, device, system and readable storage medium |
| CN110188652A (en) * | 2019-05-24 | 2019-08-30 | 北京字节跳动网络技术有限公司 | Processing method, device, terminal and the storage medium of facial image |
| CN113066008A (en) * | 2020-01-02 | 2021-07-02 | 杭州喔影网络科技有限公司 | Jigsaw generating method and equipment |
| CN111460929B (en) * | 2020-03-17 | 2022-07-19 | 稿定（厦门）科技有限公司 | Method and device for filling unspecified frame image |
| CN111696112B (en) * | 2020-06-15 | 2023-04-07 | 携程计算机技术（上海）有限公司 | Automatic image cutting method and system, electronic equipment and storage medium |
| CN113821130A (en) * | 2020-06-20 | 2021-12-21 | 华为技术有限公司 | Method and related device for determining screenshot area |
| CN112839167B (en) * | 2020-12-30 | 2023-06-30 | Oppo（重庆）智能科技有限公司 | Image processing method, device, electronic equipment and computer readable medium |
| CN114911551A (en) * | 2021-02-08 | 2022-08-16 | 花瓣云科技有限公司 | Display method and electronic equipment |
| CN112884781A (en) * | 2021-02-24 | 2021-06-01 | 北京小米松果电子有限公司 | Image cropping method and device, electronic equipment and storage medium |
| CN113205522B (en) * | 2021-04-28 | 2022-05-13 | 华中科技大学 | Intelligent image clipping method and system based on antithetical domain adaptation |
| CN116030280A (en) * | 2023-02-22 | 2023-04-28 | 青岛创新奇智科技集团股份有限公司 | Template matching method, device, storage medium and equipment |
| CN117893988A (en) * | 2024-01-19 | 2024-04-16 | 元橡科技（北京）有限公司 | All-terrain scene pavement recognition method and training method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103996186A (en) * | 2014-04-29 | 2014-08-20 | 小米科技有限责任公司 | Image cutting method and image cutting device |
| CN104717413A (en) * | 2013-12-12 | 2015-06-17 | 北京三星通信技术研究有限公司 | Shooting assistance method and equipment |
| CN105528757A (en) * | 2015-12-08 | 2016-04-27 | 华南理工大学 | Content-based image aesthetic quality improvement method |
| CN105787966A (en) * | 2016-03-21 | 2016-07-20 | 复旦大学 | An aesthetic evaluation method for computer pictures |
-
2017
- 2017-04-19 CN CN201710256861.XA patent/CN107146198B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104717413A (en) * | 2013-12-12 | 2015-06-17 | 北京三星通信技术研究有限公司 | Shooting assistance method and equipment |
| CN103996186A (en) * | 2014-04-29 | 2014-08-20 | 小米科技有限责任公司 | Image cutting method and image cutting device |
| CN105528757A (en) * | 2015-12-08 | 2016-04-27 | 华南理工大学 | Content-based image aesthetic quality improvement method |
| CN105787966A (en) * | 2016-03-21 | 2016-07-20 | 复旦大学 | An aesthetic evaluation method for computer pictures |
Non-Patent Citations (3)
| Title |
|---|
| Aesthetic photo composition by optimal crop-and-warp;Yong Jin 等;《Computers & Graphics》;20120821;全文 * |
| Optimised image retargeting using aesthetic-based cropping and scaling;Yun Liang 等;《IET Image Processing》;20130228;第7卷(第1期);第61–69页 * |
| Sensation-based Photo Cropping;Masashi Nishiyama 等;《Proceedings of the 17th ACM international conference on Multimedia》;20091031;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107146198A (en) | 2017-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107146198B (en) | Intelligent photo cutting method and device | |
| EP3338217B1 (en) | Feature detection and masking in images based on color distributions | |
| US11321385B2 (en) | Visualization of image themes based on image content | |
| US9547908B1 (en) | Feature mask determination for images | |
| US9811933B2 (en) | Image editing using selective editing tools | |
| US9262696B2 (en) | Image capture feedback | |
| EP3077898B1 (en) | Editing options for image regions | |
| US10523916B2 (en) | Modifying images with simulated light sources | |
| US8938100B2 (en) | Image recomposition from face detection and facial features | |
| US20180007259A1 (en) | Photo-taking prompting method and apparatus, an apparatus and non-volatile computer storage medium | |
| US20160203360A1 (en) | Systems and methods for performing actions in response to user gestures in captured images | |
| CN109274891B (en) | Image processing method, device and storage medium thereof | |
| WO2015123605A1 (en) | Photo composition and position guidance in an imaging device | |
| US20130108171A1 (en) | Image Recomposition From Face Detection And Facial Features | |
| CN105912126B (en) | A kind of gesture motion is mapped to the adaptive adjusting gain method at interface | |
| US20130108170A1 (en) | Image Recomposition From Face Detection And Facial Features | |
| US20130108165A1 (en) | Image Recomposition From Face Detection And Facial Features | |
| CN114930798A (en) | Shooting object switching method and device, and image processing method and device | |
| CN113591562A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| WO2018192244A1 (en) | Shooting guidance method for intelligent device | |
| CN108109115B (en) | Method, device and equipment for enhancing character image and storage medium | |
| JP2014170978A (en) | Information processing device, information processing method, and information processing program | |
| WO2023283894A1 (en) | Image processing method and device | |
| CN112839167B (en) | Image processing method, device, electronic equipment and computer readable medium | |
| CN111368674B (en) | Image recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |