WO2018192244A1 - 一种智能设备的拍摄引导方法 - Google Patents

一种智能设备的拍摄引导方法 Download PDF

Info

Publication number
WO2018192244A1
WO2018192244A1 PCT/CN2017/116058 CN2017116058W WO2018192244A1 WO 2018192244 A1 WO2018192244 A1 WO 2018192244A1 CN 2017116058 W CN2017116058 W CN 2017116058W WO 2018192244 A1 WO2018192244 A1 WO 2018192244A1
Authority
WO
WIPO (PCT)
Prior art keywords
aesthetic
image
sliding window
region
level
Prior art date
Application number
PCT/CN2017/116058
Other languages
English (en)
French (fr)
Inventor
刘弋锋
吕东岳
许忠雄
廖勇
Original Assignee
中国电子科技集团公司电子科学研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201710256861.XA external-priority patent/CN107146198B/zh
Priority claimed from CN201710257194.7A external-priority patent/CN107018330A/zh
Application filed by 中国电子科技集团公司电子科学研究院 filed Critical 中国电子科技集团公司电子科学研究院
Publication of WO2018192244A1 publication Critical patent/WO2018192244A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to a method for photographing and guiding a smart device.
  • the camera function has become one of the most important functions of smart devices. When encountering beautiful scenery, users hope to take beautiful photos and hope to leave their own figure in the landscape. However, users are using existing ones. When the camera or camera device is used, the beauty of the captured photo is determined only based on the individual's feelings. It is easy for the user to know how to get the best viewfinder image, and it is not known where the viewfinder picture is the best photographing position (for example, when shooting people) The best position to shoot the best position for a single object). Therefore, there is a need for a method that intelligently guides shooting framing and guiding a close-up object to take a picture. There are a number of invention patents that can guide photographing.
  • the related composition data is generated according to the acquired image information, and the composition data is sent to the cloud server; the cloud server searches for the matched composition data, and obtains the optimal through the operation processing.
  • the composition plan data is sent, and the optimal composition plan data is sent to the mobile terminal; the mobile terminal displays a corresponding optimal composition position prompt on the photo preview interface, and the user adjusts the position to take a photo according to the prompt.
  • the invention realizes the composition guidance when photographing, but cannot intelligently recommend the best position of the person to be photographed.
  • the invention of the application number CN201510599253.X can obtain the face pose information of the user in the framing process; compare the face pose information of the preset photograph template with the face pose information of the user; The result prompts the user to perform face gesture adjustment.
  • the invention realizes the photographing instruction for the face gesture of the user, but the face photographing template cannot perform adaptive photographing guidance according to the scene, and can not recommend a plurality of photographing stations.
  • An object of the present invention is to provide a method for photographing and guiding a smart device, the method comprising:
  • Training a first aesthetic evaluation model inputting the current view image and the plurality of different positions of the view image into the first aesthetic evaluation model, and outputting a best aesthetic region, that is, a view recommendation region;
  • the second aesthetic evaluation model is trained, and different types of images with the best aesthetic regions are input to the second aesthetic evaluation model, and the close-up object is photographed and recommended.
  • obtaining the best aesthetic region through the smart clipping algorithm comprises:
  • the spliced large image generates a plurality of sub-regions, calculates a first-level aesthetic score of the plurality of sub-regions, and obtains a best aesthetic cropping region according to the first-level aesthetic score.
  • the spliced large image generates a plurality of sub-regions, calculates a first-level aesthetic score of the plurality of sub-regions, and obtains a best aesthetic cropping region according to the first-level aesthetic score, comprising the following steps:
  • first sliding window to traverse the large image using the preset first aesthetic evaluation model to obtain a plurality of first-level aesthetic scores corresponding to the first sliding window during the traversal process, and selecting the highest first-level aesthetic score, The position corresponding to the highest first-level aesthetic score is used as the best aesthetic cropping area;
  • the first sliding window selects one or more of the following: an original proportional sliding window, a standard proportional sliding window, and a preset proportional sliding window.
  • the spliced large image generates a plurality of sub-regions, calculates a first-level aesthetic score of the plurality of sub-regions, and obtains a best aesthetic cropping region according to the first-level aesthetic score, comprising the following steps:
  • the method further includes the step of: acquiring a specified target cropping region seed point;
  • the traversing the large image by using a preset sliding window, and obtaining a plurality of first-level aesthetic scores corresponding to the preset sliding window during the traversing process including the following steps:
  • the target crop region seed point neighborhood is traversed using a preset sliding window to obtain a plurality of first-level aesthetic scores corresponding to the preset sliding window during the traversal process.
  • the first sliding window comprises an original proportional sliding window and a standard proportional sliding window
  • the first sliding window comprises an original proportional sliding window and a standard proportional sliding window
  • Calculating a maximum cross ratio of the first sliding window to the optimal preselected area, and generating a position of the maximum cross ratio as the best aesthetic cropping area comprising the following steps:
  • the closest one of the center and the best preselected area center is selected as the best aesthetic cropping area in the best aesthetic cropping area at the original scale and the best aesthetic cropping area in the standard scale.
  • the current view image and the plurality of different positions of the view image are input to the first aesthetic evaluation model, and the best aesthetic region is output, including:
  • the current view image and the plurality of different positions of the view image are identified by the classification algorithm and the scene is classified, and the image information of the view image is input into the first aesthetic evaluation model for scoring. And output the best aesthetic area.
  • the image information includes an image scene classification result and the image itself.
  • the image information includes an image scene classification result, an image photo EXIF, and the image itself.
  • generating a plurality of virtual locations in the optimal aesthetic region comprises:
  • the position information corresponding to the region where the background object is far away is selected as the candidate virtual position information during the sliding window traversal.
  • the image having the best aesthetic region is separately subjected to the second-level aesthetic score in combination with the plurality of virtual positions, and the position of the photographing guide recommendation is obtained according to the second-level aesthetic score and the display includes:
  • the recommended location selects one or more of the plurality of virtual locations where the best location or aesthetic score is greater than or equal to the lowest threshold.
  • the virtual position corresponding to the highest aesthetic score is selected as the optimal position; and the plurality of virtual positions whose aesthetic score is greater than or equal to the lowest threshold are arranged in descending order of the aesthetic score.
  • the image having the best aesthetic region is combined with the plurality of virtual locations to perform the second-level aesthetic score, and the location of the photographing guide recommendation according to the second-level aesthetic score includes:
  • the second aesthetic evaluation model comprises a single model and a multi-model, and for the image having the best aesthetic region in which the EXIF information cannot be read, the second aesthetic evaluation model is trained as follows:
  • Predicting a second aesthetic evaluation model based on different scene labels to which the image belongs using images of the best aesthetic regions with known second-level aesthetic scores, image scene category labels, person or animal or plant position coordinates Multiple models corresponding to multiple scenes are trained separately.
  • the second aesthetic evaluation model comprises a single model and a multi-model, and the second aesthetic evaluation model having the best aesthetic region capable of reading the EXIF information is trained as follows:
  • the two aesthetic evaluation models respectively train multiple models corresponding to multiple scenes.
  • the image information having the best aesthetic region includes the image scene classification result and the image itself, and the image information having the best aesthetic region is input to the second image.
  • the aesthetic evaluation model scores and outputs a second-level aesthetic score and position coordinates of the person or animal or plant.
  • the image information having the best aesthetic region includes the image scene classification result and the image itself, and the images of the different category scenes are obtained according to the scene classification result of the image.
  • the second aesthetic evaluation model is input for scoring, and the second-level aesthetic score and the position coordinates of the person or animal or plant are output.
  • the image information having the best aesthetic region includes the image scene classification result, the image photo EXIF, the image itself, and the image information having the best aesthetic region.
  • the score is input to the second aesthetic evaluation model, and the second-level aesthetic score and the position coordinates of the person or animal or plant are output.
  • the image information having the best aesthetic region includes the image scene classification result, the image photo EXIF, and the image itself, which are different according to the scene classification result of the image.
  • the image photo EXIF of the category scene and the image itself are input to the second aesthetic evaluation model for scoring, and the second-level aesthetic score and the position coordinates of the person or animal or plant are output, that is, the position at which the guide guide is taken.
  • the invention intelligently recognizes the shooting scene and gives the recommended station position, so that the user who does not have the professional photography knowledge can also easily capture the high-feeling image, and the invention integrates the photos of the plurality of scenes in advance through the image aesthetic evaluation technology. Intelligent cutting allows you to tailor the best aesthetics from an aesthetic point of view and guide the user through the shooting.
  • the invention adopts an image scene classification algorithm to perform adaptive aesthetic evaluation on different scenes, and scores by training the first aesthetic evaluation model to output the best aesthetic region and guide the user to shoot.
  • the invention utilizes the photo EXIF information, can more accurately obtain various parameters of the photo shooting, and ensures that the first-level aesthetic score is more accurate.
  • the invention utilizes the position information of the close-up main body of the image, and takes into account the composition factor of the main body, so the first-level aesthetic score of the photo image mainly composed of the above object is more accurate.
  • the first aesthetic evaluation model in the present invention is trained by the machine learning method, and the subjectivity and one-sidedness of the manual rule evaluation are avoided.
  • FIG. 1 is a flow chart schematically showing a method for photographing and guiding a smart device of the present invention
  • FIG. 2 is a schematic diagram of a smart device viewfinder lens for acquiring a view image according to the present invention
  • 3 is a flow chart showing the splicing-cutting of the smart device of the present invention to obtain an optimal aesthetic region
  • FIG. 4 is a flow chart showing the selection of the best aesthetic region in addition to the first aesthetic evaluation model of the present invention.
  • FIG. 5 is a flowchart showing a single model training for evaluating a view image in which EXIF information cannot be read;
  • FIG. 6 is a flow chart showing a multi-model training for evaluating a view image in which EXIF information cannot be read;
  • FIG. 7 is a flow chart showing a single model training for evaluating a view image capable of reading EXIF information according to the present invention.
  • FIG. 8 is a flow chart showing a multi-model training for evaluating a view image capable of reading EXIF information according to the present invention
  • FIG. 9 is a flowchart showing a single model shooting guide of a view image in which EXIF information cannot be read;
  • FIG. 10 is a flowchart showing a multi-model shooting guidance of a view image in which EXIF information cannot be read;
  • Figure 11 is a flow chart showing a single model shooting guide of a view image capable of reading EXIF information
  • FIG. 12 is a flowchart showing a multi-model shooting guidance of a view image capable of reading EXIF information
  • Figure 13 is a flowchart showing a single model training for evaluating an image having the best aesthetic region in which EXIF information cannot be read;
  • Figure 14 is a flowchart showing a multi-model training for evaluating an image having the best aesthetic region in which EXIF information cannot be read;
  • Figure 15 is a flowchart showing a single model training for evaluating an image having the best aesthetic region capable of reading EXIF information
  • Figure 16 is a flowchart showing a multi-model training in addition to the present invention for evaluating an image having the best aesthetic region capable of reading EXIF information;
  • 17 shows a single model shooting guide flow chart of an image having the best aesthetic region in which EXIF information cannot be read
  • Figure 18 is a flowchart showing a multi-model shooting guidance of an image having the best aesthetic region in which EXIF information cannot be read;
  • Figure 19 is a flow chart showing a single mode shooting of an image having the best aesthetic area in which EXIF information can be read;
  • Figure 20 shows a multi-model shooting guide flow chart of an image having the best aesthetic region capable of reading EXIF information
  • Figure 21 is a diagram showing the smart device of the present invention guiding a person to shoot.
  • the invention provides an automatic evaluation method based on aesthetic evaluation in a computing device, including but not limited to personal personal computers, smart phones, tablets, smart glasses, and other functions with camera, data storage and data processing functions. device.
  • the present invention provides a photographing and guiding method for the smart device.
  • the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
  • a method for photographing and guiding a smart device is provided in the embodiment.
  • FIG. 1 a flowchart of a method for photographing and guiding a smart device of the present invention, the method for photographing and guiding a smart device includes:
  • the image splicing algorithm is used to splicing the current framing image and the framing images of the plurality of different positions into a large image, and obtaining an optimal aesthetic region by using an intelligent clipping algorithm, that is, a framing recommendation region.
  • S105 Acquire a close-up object photographing recommended position in the optimal aesthetic region, including generating a plurality of virtual positions in the optimal aesthetic region, and the image having the optimal aesthetic region is combined with the plurality of virtual positions to perform the second level.
  • An aesthetic score, the virtual position of the best aesthetic score based on the second aesthetic score, that is, the close-up object is photographed and displayed, or
  • the second aesthetic evaluation model is trained, and different types of images with the best aesthetic regions are input to the second aesthetic evaluation model, and the close-up object is photographed and recommended.
  • FIG. 2 is a schematic diagram of a smart device framing lens of the present invention for acquiring a framing image.
  • the smart device of the present invention includes a personal computer, a smart phone, a tablet computer, smart glasses, and other devices having a camera and a data processing function, which are exemplified in the embodiment.
  • the smart device mobile phone With the mobile phone as a carrier, the smart device mobile phone has a viewfinder lens (camera), a memory and a processor chip.
  • the image acquired by the framing lens is stored in the memory, and is processed by the processor chip to select the best aesthetic area to guide the user to shoot.
  • the computing device can be a camera that takes images directly or other smart devices known to those skilled in the art. As shown in FIG.
  • the front end display screen 101 of the mobile phone 100 displays a framing image.
  • the framing lens is disturbed according to a certain rule, and information of each motion sensor (such as a gyroscope or an altitude meter) of the mobile phone is recorded to obtain a plurality of different positions adjacent to the shooting environment.
  • the framing image For example, taking the framing images of four different positions as an example in the embodiment, the framing framing lens sequentially acquires the framing images of the four positions a, b, c, and d in the direction of the arrow.
  • the obtained current view image 101 and the framing images at different positions are obtained by stitching-trimming to obtain the best aesthetic region, and the best aesthetic region is obtained by training the first aesthetic evaluation model.
  • the process of obtaining the best aesthetic region will be described in detail in the following embodiments.
  • the smart device of the present invention splicing-cutting and obtaining a best aesthetic region, in the embodiment, using an image stitching algorithm to splicing the current view image and the plurality of different positions of the viewfinder into a large image. Get the best aesthetic area with the smart clipping algorithm. specifically:
  • the current view image and the plurality of different positions of the view image are spliced into a large picture.
  • the current view image 101 and the four positions of the view image (a, b, c, d four view images) are stitched together.
  • a big picture Into a big picture.
  • S202 Generate a plurality of sub-regions in the spliced large image, calculate a first-level aesthetic score of the plurality of sub-regions, and obtain a best aesthetic clipping region according to the calculated first-level aesthetic score. Specifically, generating a plurality of sub-regions in the large image includes traversing the spliced large image with a preset sliding window to obtain a plurality of sub-regions.
  • the best aesthetically tailored area in the embodiment is implemented by two schemes, wherein
  • the first option is:
  • the first sliding window uses the preset first aesthetic evaluation model to obtain a plurality of first-level aesthetic scores corresponding to the first sliding window during the traversal process, and selecting the highest first-level aesthetic score, The position corresponding to the highest first-level aesthetic score is used as the best aesthetic cropping area.
  • the first sliding window selects one or more of the following: a raw proportional sliding window, a standard proportional sliding window, and a preset proportional sliding window.
  • the first sliding window includes a preset proportional sliding window
  • the first sliding window is used to traverse the spliced large image
  • the preset first aesthetic evaluation model is used to obtain the first sliding window corresponding to the traversing process.
  • the spliced large image is traversed using a preset proportional sliding window, and a plurality of first-level aesthetic scores corresponding to the first sliding window during traversal are obtained by using a preset first aesthetic evaluation model.
  • the first sliding window includes the original proportional sliding window and the standard proportional sliding window
  • the first sliding window is used to traverse the spliced large image
  • the preset first aesthetic evaluation model is used to obtain the traversing process.
  • a number of first-level aesthetic scores corresponding to the first sliding window are selected, and the highest first-level aesthetic score is selected, including the following scheme A and scheme B.
  • scenario A includes the following steps:
  • the spliced large image is traversed using the original proportional sliding window, and a plurality of first-level aesthetic scores corresponding to the original proportional sliding window during the traversal process are obtained by using the preset first aesthetic evaluation model.
  • the spliced large image is traversed using a standard proportional sliding window, and a plurality of first-level aesthetic scores corresponding to the standard proportional sliding window during the traversal process are obtained by using the preset first aesthetic evaluation model.
  • the highest first-level aesthetic score is selected from a plurality of first-level aesthetic scores corresponding to the original proportional sliding window and a plurality of first-level aesthetic scores corresponding to the standard proportional sliding window.
  • the solution B includes the following steps:
  • the spliced large image is traversed using the original proportional sliding window, and a plurality of first-level aesthetic scores corresponding to the original proportional sliding window during the traversal process are obtained by using the preset first aesthetic evaluation model.
  • the highest first-level aesthetic score of the first-level aesthetic scores corresponding to the original proportional sliding window is selected, and the optimal aesthetic cropping area under the original scale is obtained according to the coordinates of the original proportional sliding window corresponding to the highest first-level aesthetic score.
  • one of the best aesthetic cropping area under the original scale or the best aesthetic cropping area under the standard scale is selected as the best aesthetic cropping area of the image.
  • the original proportional sliding window and the standard proportional sliding window can be selected in several dimensions.
  • the scale ratio of each scale is 1.2
  • the horizontal side length of the minimum scale sliding window is 1/4 of the horizontal direction of the original image (large image stitched into), that is,
  • the original scale images of the following three scales are selected: the size is 0.25 times the original image size, the size is 0.25 ⁇ 1.2 times the original image size, and the size is 0.25 ⁇ 1.2 ⁇ 1.2 times the original image size.
  • the second option is:
  • the second sliding window to traverse the spliced large image, obtaining a plurality of first-level aesthetic scores corresponding to the second sliding window during the traversing process, and using the first-level aesthetic scores as the traversal process
  • An aesthetic quality value of a plurality of second sliding window center point coordinates, and an aesthetic quality map is generated based on a number of aesthetic quality values.
  • the second sliding window is the original proportion of the photo image, and the second sliding window is smaller than or equal to the first sliding window, and the second sliding window step is less than or equal to the first sliding window;
  • the first sliding window includes an original proportional sliding window and a standard proportional sliding window
  • calculating a maximum cross ratio of the first sliding window to the optimal preselected area, and generating a maximum cross ratio is the most Jiamei feels the cropping area, including the following steps:
  • the closest one of the center and the center of the best preselected region is selected as the best aesthetic cropping region of the spliced large image.
  • the method further includes the following steps: acquiring a specified target cropping region seed point; further, the traversing the spliced large graph by using a preset sliding window to obtain the traversing
  • a plurality of first-level aesthetic scores corresponding to the preset sliding window in the process include the following steps:
  • the target crop region seed point neighborhood is traversed using a preset sliding window to obtain a plurality of first-level aesthetic scores corresponding to the preset sliding window during the traversal process.
  • S203 obtain the coordinates of the best aesthetic cropping area, and cut the spliced large image to obtain an optimal cropping area of the spliced large image, and the best tailoring area is obtained.
  • the best aesthetic area is obtained.
  • the current view image obtained by the embodiment of the invention and the images of the plurality of different positions are spliced into a large picture, and the large picture is intelligently cut to obtain the best aesthetic area, and the best aesthetic area is used to guide the user to take the image, so that the shooting is performed.
  • the image is more in line with the application scene and is more aesthetically pleasing.
  • the present invention trains the first aesthetic evaluation model to obtain a flow chart of the best aesthetic region.
  • the current view image and the plurality of different positions of the view image are input into the first aesthetic evaluation model to obtain the best aesthetic sense.
  • Areas including:
  • the image scene classification algorithm is used to identify and classify the current view image and the plurality of different positions of the view image.
  • the framing image referred to in step S304 includes the current framing image and the framing images of the plurality of different positions, specifically, the current framing image 101 and the framing images of four different positions (a, b, c, d in the embodiment). A framing image of four positions).
  • the first aesthetic evaluation model of the present invention includes a character class model and a scene class single model, and a character class multi-model and a scene class multi-model.
  • the first aesthetic evaluation model classifies and trains whether the photo to be scored is a close-up photo, and the training process Trained by machine learning methods, the machine learning methods used include convolutional neural networks, restricted Boltzmann machines, deep confidence networks, and the like.
  • the first aesthetic evaluation model is trained to obtain the character class model and the scene class.
  • the single model, as well as the multi-model of the character class and the multi-model of the scene class, the first aesthetic evaluation model is obtained through machine learning training.
  • the present invention is a single model training flowchart for evaluating a view image in which EXIF information cannot be read, and a single model training for evaluating a view image in which EXIF information cannot be read is trained as follows:
  • S402. Train the preset first aesthetic evaluation model by using a plurality of images of the known first-level aesthetic score, the image scene category label, and the location coordinates of the region of interest to obtain a single model.
  • the present invention is a multi-model training flowchart for evaluating a view image in which EXIF information cannot be read, and a multi-model training for evaluating an image in which EXIF information cannot be read is trained as follows:
  • the model includes a multi-model training for a character-like image photograph and a scene-like image photograph to obtain a model for aesthetic evaluation of the character-like image and a model for aesthetic evaluation of the scene-like image, and those skilled in the art understand that it is not limited thereto.
  • the first aesthetic evaluation model is trained to obtain a single model and a multi-model.
  • the single model training flowchart for evaluating a view image capable of reading EXIF information is used for training a single model for evaluating a view image capable of reading EXIF information according to the following method:
  • the multi-model training flowchart for evaluating a view image capable of reading EXIF information is used for training a multi-model training for evaluating a view image capable of reading EXIF information according to the following method:
  • the multi-model includes a multi-model training for the character-like image photograph and the scene-like image photograph to obtain a model for the aesthetic evaluation of the character-like image and a model for the aesthetic evaluation of the scene-like image, and those skilled in the art should understand that it is not limited thereto.
  • the shooting guidance process of the smart device of the present invention is specifically described in the following embodiments: the framing images are automatically scored by the different kinds of first aesthetic evaluation models obtained by the above training, and the shooting guidance is performed according to the score. specifically,
  • the image scene classification algorithm is used to perform image recognition and scene classification on the image captured by the smart device, and determine whether the photo is a close-up photo.
  • the image scene classification algorithm is used to identify the photos, and the photos are divided into landscape, night scene, architecture, dynamic, static, backlight, portrait, animal, and plant according to the recognition result.
  • the scene classification algorithm includes the following steps to implement scene classification of the framing image:
  • An object recognition algorithm is used to identify objects in the image.
  • the association between objects in the image identified by the object recognition algorithm is analyzed according to the context semantic model.
  • the images can be classified into landscape, night scene, architecture, dynamic, static, backlight, portrait, animal, and plant categories, that is, the scene classification result of the image is obtained.
  • the framing image is a close-up photo, otherwise it is a non-close-up photo.
  • the single-model shooting guidance flowchart of the view image of the EXIF information cannot be read.
  • the image information of the framing image includes the image scene classification result and the image itself.
  • the image information of the framing image is input to the single model obtained in step S402 for scoring, and the first-level aesthetic score and the region of interest position coordinate of the framing image are output, and the location coordinates of the region of interest is used as the best aesthetic region to guide the user to shoot. For example, for a scene of a person or an animal or a plant, the coordinates of the position of the region of interest corresponding to the person or animal or plant are respectively output as the corresponding optimal aesthetic region.
  • the result of the scene classification in the input single model is a category label of a different class of the image scene.
  • the multi-model shooting guidance flowchart of the view image of the EXIF information cannot be read.
  • the image information of the view image includes the image scene classification result and the image itself.
  • the images of the different types of scenes are input into step S404 to obtain a first aesthetic evaluation model of the multi-model corresponding scene, and the first-level aesthetic score of the framing image and the coordinates of the region of interest are output, which will be of interest.
  • the area position coordinates are used as the best aesthetic area.
  • the character type view image input into the character class image aesthetic evaluation model and the scene type view image are input into the model of the scene object image aesthetic evaluation, and the position coordinates of the region of interest and the scene type view image of the character class view image are respectively obtained.
  • the location coordinates of the region of interest Those skilled in the art will understand that it is not limited thereto.
  • the single-model shooting and shooting flowchart of the framing image of the EXIF information can be read.
  • the image information includes the image scene classification result, the image photo EXIF, and the image itself.
  • the image information of the framing image is input to the single model obtained in step S406 for scoring, the first-level aesthetic score of the framing image and the coordinates of the region of interest are output, and the coordinates of the region of interest are guided as the best aesthetic region to guide the user to shoot. For example, for a scene of a person or an animal or a plant, the coordinates of the position of the region of interest corresponding to the person or animal or plant are respectively output as the corresponding optimal aesthetic region.
  • the result of the scene classification in the input single model is a category label of a different class of the image scene.
  • a multi-model shooting guide flow chart capable of reading the view image of the EXIF information
  • the image information of the view image includes the image scene classification result, the image photo EXIF, and the image itself.
  • the scene classification result of the image the image photo EXIF of the different category scenes and the image itself are input into step S408 to obtain a first aesthetic evaluation model of the multi-model corresponding scene, and the first-level aesthetic score and the location coordinates of the region of interest are output.
  • the location coordinates of the region of interest is taken as the best aesthetic region.
  • the model image of the person type image is input into the model of the aesthetic evaluation of the character image and the image of the scene image is input into the model of the aesthetic evaluation of the scene image, and the coordinates of the region of interest and the scene framing image of the character framing image are respectively obtained.
  • the location of the area of interest Those skilled in the art will understand that it is not limited thereto.
  • the second aesthetic evaluation model is trained, and different types of images having the best aesthetic regions are input to the second aesthetic evaluation model, and the position at which the photographing guide is recommended is output.
  • the virtual location herein includes, but is not limited to, a virtual location of a specific person, a virtual location of a specific plant, and a virtual location of a specific animal. For example, when photographing guidance for a person photographing, a virtual station of a plurality of personal objects is generated.
  • generating a plurality of virtual stations in the best aesthetic region of the image having the best aesthetic region includes the following steps:
  • a plurality of virtual positions are obtained by traversing through the multi-scale sliding window in the optimal aesthetic region; preferably, the position information corresponding to the region where the background object is far away is selected as the candidate virtual station information during the sliding window traversal.
  • the image with the best aesthetic region is combined with the plurality of virtual positions to perform the second-level aesthetic evaluation, and the position of the shooting guidance recommendation is obtained according to the second-level aesthetic score and displayed.
  • the recommended location selects one or more of the plurality of virtual locations where the optimal location or the aesthetic score is greater than or equal to the lowest threshold.
  • the virtual position corresponding to the highest aesthetic score is selected as the optimal position; and the plurality of virtual positions whose aesthetic score is greater than or equal to the lowest threshold are arranged in descending order of the aesthetic score.
  • the image with the best aesthetic region is combined with a plurality of virtual locations to perform a second-level aesthetic score
  • the location of the photographing guide recommendation according to the second-level aesthetic score includes:
  • the location of the shooting guide recommendation is obtained based on the plurality of second-level aesthetic scores.
  • the position of the photographing guide recommendation is obtained by the second aesthetic evaluation model in the embodiment.
  • the position of the specific person, the position of the specific animal, and the position of the specific plant are obtained in the embodiment. For example, it should be understood that it is not limited thereto.
  • the second aesthetic evaluation model of the present invention includes a single model and a multi-model, and the second aesthetic evaluation model classifies whether the image with the best aesthetic region is a close-up photo, and the training process is trained by a machine learning method, and the machine learning is used.
  • Methods include convolutional neural networks, restricted Boltzmann machines, deep confidence networks, and the like.
  • the second aesthetic evaluation model is trained to obtain a single model and a multi-model, and the second aesthetic evaluation model is obtained through machine learning training.
  • the present invention is a single model training flowchart for evaluating an image having the best aesthetic region in which EXIF information cannot be read, for evaluating an image having the best aesthetic region in which EXIF information cannot be read.
  • the single model training is trained as follows:
  • S602. Train a preset second aesthetic evaluation model by using an image with a best aesthetic region of a known second-level aesthetic score, an image scene category label, a person or an animal or plant position coordinate, to obtain a single model.
  • the present invention is a multi-model training flowchart for evaluating an image having the best aesthetic region in which EXIF information cannot be read, for evaluating an image having the best aesthetic region in which EXIF information cannot be read.
  • the multi-model training is trained as follows:
  • the second evaluation model is trained to obtain a single model and multiple models.
  • the present invention is a single model training flowchart for evaluating an image having the best aesthetic region capable of reading EXIF information, for evaluating an image having the best aesthetic region capable of reading EXIF information.
  • the single model training is trained as follows:
  • the present invention is a multi-model training flowchart for evaluating an image having the best aesthetic region capable of reading EXIF information, for evaluating an image having the best aesthetic region capable of reading EXIF information.
  • the multi-model training is trained as follows:
  • the preset is determined according to the scene label to which the image belongs.
  • the second aesthetic evaluation model respectively trains multiple models corresponding to multiple scenes.
  • the photographing and guiding process of the smart device of the present invention is specifically described in the following embodiments: the different kinds of second aesthetic evaluation models obtained by the above training are automatically scored on the image with the best aesthetic region, and the close-up object is obtained in the best aesthetic region. Shoot the recommended location. specifically,
  • the information includes the image scene classification result and the image itself.
  • the image information having the best aesthetic region is input to the single model obtained in step S602 for scoring, and the second-level aesthetic score and the position coordinates of the person or animal or plant are output, that is, the close-up object photographing recommended position.
  • the information includes the image scene classification result and the image itself.
  • the images of the different types of scenes are input into step S604 to obtain a second aesthetic evaluation model of the multi-model corresponding scene, and the second-level aesthetic score and the position coordinates of the person or the animal or the plant are output, that is, the close-up object.
  • Shoot the recommended location
  • a single-model shooting and shooting flowchart capable of reading an image of an EXIF information having an optimal aesthetic region, when the image having the best aesthetic region is capable of reading EXIF information, has the best aesthetic region.
  • the image information includes an image scene classification result, an image photo EXIF, and an image itself.
  • the image information having the best aesthetic region is input to the single model obtained in step S606 for scoring, and the second-level aesthetic score and the position coordinates of the person or animal or plant are output, that is, the close-up object photographing recommended position.
  • a multi-model shooting guide flow chart capable of reading an image of an EXIF information having an optimal aesthetic region as shown in FIG. 20, and an image having the best aesthetic region when the image having the best aesthetic region is capable of reading EXIF information.
  • the information includes image scene classification results, image photo EXIF, and the image itself.
  • the scene classification result of the image the image photo EXIF of the different category scenes and the image itself are input into step S608 to obtain a second aesthetic evaluation model of the multi-model corresponding scene, and the second-level aesthetic score and the position coordinates of the person or the animal or the plant are output. , that is, a close-up object to shoot the recommended position.
  • the smart device of the present invention guides a person's photographing.
  • the person photographing guiding process is exemplarily given, and the photographing guide recommendation in the best aesthetic region photographed by the person is displayed in the display interface 101 of the mobile phone 100.
  • the photographing guide recommendation in the best aesthetic region photographed by the person is displayed in the display interface 101 of the mobile phone 100.
  • a person who needs to be photographed is placed in a position 102 of the photographing guidance recommended by the display interface 101 to perform photographing.
  • the invention intelligently recognizes the shooting scene and gives the recommended station position, so that the user who does not have the professional photography knowledge can also easily capture the high-feeling image, and the invention integrates the photos of the plurality of scenes in advance through the image aesthetic evaluation technology. Intelligent cutting allows you to tailor the best aesthetics from an aesthetic point of view and guide the user through the shooting.
  • the recommended position is photographed using the best-featured area image close-up object, and in other embodiments, the image may be used to generate a close-up object to capture the recommended position (person, animal in the background).
  • the best aesthetic region may also be generated separately (eg, when taking a landscape picture).
  • the invention adopts the image scene classification algorithm to adaptively evaluate the different scenes, evaluates the model by training aesthetic evaluation model, outputs the best aesthetic area, and guides the user to shoot.
  • the invention utilizes the photo EXIF information, can more accurately obtain various parameters of the photo shooting, and ensures that the aesthetic score is more accurate.
  • the invention utilizes the position information of the close-up main body of the image, and takes into account the composition factor of the main body, so the aesthetic score of the image with the object as the main body is more accurate.
  • the aesthetic evaluation model in the present invention is trained by the machine learning method, and the subjectivity and one-sidedness of the manual rule evaluation are avoided.
  • the photographing and guiding method of the smart device helps the user to take photos of landscapes and people, and further recommends the best shooting guide position for different types of photos, thereby making the photograph more beautiful. Image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供了一种智能设备的拍摄引导方法包括：获取智能设备中的当前取景图像；按照一定的规律扰动智能设备的取景镜头，获取拍摄环境邻近的多个不同位置的取景图像；使用图像拼接算法将当前取景图像和多个不同位置的取景图像拼接成一幅大图，通过智能剪裁算法获取最佳美感区域；或者训练第一美学评价模型，将当前取景图像和多个不同位置的取景图像输入到第一美学评价模型中，输出最佳美感区域。在最佳美感区域图像中结合多个虚拟位置进行第二级美学评分，获得最佳美学评分的虚拟位置；或者训练第二美学评价模型，将不同类的具有最佳美感区域的图像输入到第二美学评价模型，输出特写物体拍摄推荐位置。本发明指导用户进行取景及特写物体拍摄。

Description

一种智能设备的拍摄引导方法
专利的交叉引用
本申请要求2017年04月19日提交的，申请号CN201710256861.X、申请号CN201710257194.7的中国发明专利申请的优选权。
技术领域
本发明涉及图像处理技术领域，特别涉及一种智能设备的拍摄引导方法。
背景技术
拍照功能已经成为目前智能设备的必备重要功能之一，遇到美丽的风景时，用户希望可以拍出具有美感的照片，也希望在风景中留下自己的身影，然而用户在使用现有的相机或拍照设备时，仅根据个人的感受来确定拍摄照片的美感，其容易造成用户不知如何获得最佳的取景画面，也不知取景画面中的何处才是最佳拍照位置(例如拍摄人时的最佳站位，拍摄单个物体的最佳位置)。因此，需要一种能智能引导拍摄取景和引导特写物体拍照站位的方法。目前已有一些可以对拍照进行指导的发明专利。申请号为CN201410180886.2的发明进入拍照预览模式时，根据获取到的图像信息生成相关构图数据，并发送所述构图数据至云端服务器；云端服务器搜索匹配的构图数据，并通过运算处理获取最优构图方案数据，并将所述最优构图方案数据发送至移动终端；移动终端在拍照预览界面上显示相应的最佳构图位置提示，用户根据提示调整位置拍照。该发明实现了拍照时进行构图指导，但是无法智能推荐出被拍照人的最佳站位。申请号为CN201510599253.X的发明可以实现在取景过程中获取所述用户的人脸姿态信息；将预设的拍照模板的人脸姿态信息与所述用户的人脸姿态信息进行比对；根据比对结果提示所述用户进行人脸姿态调整。该发明实现了对用户的人脸姿态进行拍照指导，但人脸拍照模板不能根据场景进行自适应的拍照指导，且不能推荐多种拍照站位。
因此，为了解决上述问题，需要智能识别拍摄场景，给出推荐站位，使得不具备专业摄影知识的用户也能轻松拍出高美感的图像的一种智能设备的拍摄引导方法。
发明内容
本发明的目的在于提供一种智能设备的拍摄引导方法，所述方法包括：
a)获取智能设备中的当前取景图像；
b)按照一定的规律扰动所述智能设备的取景镜头，记录扰动时各个运动传感器信息，获取拍摄环境邻近的多个不同位置的取景图像；
c)使用图像拼接算法将所述当前取景图像和所述多个不同位置的取景图像拼接成一幅大图，通过智能剪裁算法获取最佳美感区域，即为取景推荐区域；或者
训练第一美学评价模型，将所述当前取景图像和所述多个不同位置的取景图像输入到所述第一美学评价模型中，输出最佳美感区域，即为取景推荐区域；
d)在所述最佳美感区域中获取特写物体拍摄推荐位置，包括
在所述最佳美感区域中生成多个虚拟位置，具有所述最佳美感区域的图像结合多个虚拟位置进行第二级美学评分，根据所述第二美学评分得到最佳美学评分的虚拟位置，即特写物体拍摄推荐位置并显示，或者
训练第二美学评价模型，将不同类的具有最佳美感区域的图像输入到第二美学评价模型，输出特写物体拍摄推荐位置并显示。
优选地，通过智能剪裁算法获取最佳美感区域包括：
将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域。
优选地，将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域，包括如下步骤：
使用第一滑动窗口遍历所述大图，利用预设的第一美学评价模型得 到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，以最高第一级美学评分对应的位置作为最佳美感裁剪区域；
其中，第一滑动窗口选取以下中的一种或几种：原始比例滑动窗口、标准比例滑动窗口、预设比例滑动窗口。
优选地，将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域，包括如下步骤：
使用第二滑动窗口遍历所述大图，得到在遍历的过程中所述第二滑动窗口对应的若干个第一级美学评分，以所述若干个第一级美学评分作为遍历过程中若干个第二滑动窗口中心点坐标的美学质量值，根据若干个美学质量值生成美学质量图谱；其中，所述第二滑动窗口为大图原始比例，且第二滑动窗口小于或等于第一滑动窗口，第二滑动窗口步长小于或等于第一滑动窗口；
使用预设的聚类算法得到所述美学质量图谱中的能量最高区域，以所述能量最高区域的最小外接矩形作为最佳预选区域；
计算所需裁剪比例的窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域。
优选地，还包括以下步骤：获取指定的目标裁剪区域种子点；
所述使用预设的滑动窗口遍历所述大图，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分，包括以下步骤：
使用预设的滑动窗口遍历所述目标裁剪区域种子点邻域，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分。
优选地，当所述第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时；
使用第一滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，包括以下步骤：
使用原始比例滑动窗口遍历所述大图，利用预设的第一美学评价模 型得到在遍历的过程中所述原始比例滑动窗口对应的若干个第一级美学评分；
使用标准比例滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述标准比例滑动窗口对应的若干个第一级美学评分；
在所述原始比例滑动窗口对应的若干个第一级美学评分、及所述标准比例滑动窗口对应的若干个第一级美学评分中选取最高第一级美学评分；
或者使用原始比例滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述原始比例滑动窗口对应的若干个第一级美学评分；
选取原始比例滑动窗口对应的若干个第一级美学评分中最高的第一级美学评分，根据最高的第一级美学评分对应的原始比例滑动窗口的坐标得到原始比例下的最佳美感裁剪区域；
计算所述标准比例滑动窗口与所述原始比例下的最佳美感裁剪区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域；
按照指令选择原始比例下的最佳美感裁剪区域或标准比例下的最佳美感裁剪区域中的一个作为最佳美感裁剪区域。
优选地，当所述第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时；
计算第一滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域，包括以下步骤：
计算原始比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为原始比例下的最佳美感裁剪区域；
计算标准比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域；
在所述原始比例下的最佳美感裁剪区域、及标准比例下的最佳美感裁剪区域中选取中心与所述最佳预选区域中心最近的一个作为最佳美感 裁剪区域。
优选地，所述当前取景图像和所述多个不同位置的取景图像输入到所述第一美学评价模型，输出最佳美感区域，包括：
采用图像场景采用分类算法对所述当前取景图像和所述多个不同位置的取景图像识别并进行场景分类，并将所述取景图像的图像信息输入到所述第一美学评价模型中进行评分，并输出最佳美感区域。
优选地，若所述取景图像不能读取EXIF信息，则所述图像信息包括图像场景分类结果和图像本身。
优选地，若所述取景图像可以读取EXIF信息，则所述图像信息包括图像场景分类结果、图像照片EXIF、图像本身。
优选地，在所述最佳美感区域中生成多个虚拟位置包括：
在最佳美感区域中通过多尺度滑动窗口遍历，得到若干个虚拟位置。
优选地，滑动窗口遍历过程中选取背景物体距离较远的区域对应的位置信息作为候选的虚拟位置信息。
优选地，具有最佳美感区域的图像结合若干个虚拟位置分别进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置并显示包括：
推荐位置选取最佳位置或美学评分大于等于最低阈值的多个虚拟位置中的一个或多个。
优选地，选取最高美学评分对应的虚拟位置作为最佳位置；美学评分大于等于最低阈值的多个虚拟位置按照美学评分由高至低的顺序进行排列。
优选地，具有最佳美感区域的图像分别结合若干个虚拟位置进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置包括：
将具有最佳美感区域的图像与任意一个虚拟位置输入到预设的美学评价模型中对虚拟位置进行第二级美学评分，遍历所述若干个虚拟位置，得到若干个第二级美学评分；根据所述若干个第二级美学评分得到拍摄引导推荐的位置。
优选地，所述第二美学评价模型包括单模型和多模型，针对不能读取EXIF信息的具有最佳美感区域的图像，所述第二美学评价模型按如下 方法训练：
获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像；
利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型，或者
利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
优选地，所述第二美学评价模型包括单模型和多模型，针对能读取EXIF信息的具有最佳美感区域的所述第二美学评价模型按如下方法训练：
获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像。
利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型，或者
利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
优选地，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果和图像本身，将具有最佳美感区域的图像信息输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
优选地，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果和图像本身，根据图像的场景分类结果，将不同类别场景的图像本身输入第二美学评价模型 进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
优选地，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF、图像本身，将具有最佳美感区域的图像信息输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
优选地，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF和图像本身，根据图像的场景分类结果，将不同类别场景的图像照片EXIF和图像本身输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即拍摄引导推荐的位置。
本发明智能识别拍摄场景，给出推荐站位，使得不具备专业摄影知识的用户也能轻松拍出高美感的图像，本发明通过图像美学评价技术预先对多个场景的照片拼接成的大图进行智能裁剪，使得从美学角度裁剪出最佳美感区域，进而指导用户进行拍摄。
本发明采用图像场景分类算法对不同场景进行自适应地美学评价，通过训练第一美学评价模型进行评分，输出最佳美感区域，指导用户进行拍摄。
本发明利用了照片EXIF信息，可以更准确的获取照片拍摄时的各项参数，保证第一级美学评分更加准确。
本发明利用了图像特写主体物体位置信息，考虑到了主体的构图因素，因此对以上述物体为主体的照片图像的第一级美学评分更加准确。
本发明中的第一美学评价模型通过机器学习方法训练得到，避免了人工规则评价的主观性和片面性。
应当理解，前述大体的描述和后续详尽的描述均为示例性说明和解释，并不应当用作对本发明所要求保护内容的限制。
附图说明
参考随附的附图，本发明更多的目的、功能和优点将通过本发明实施方式的如下描述得以阐明，其中：
图1示意性示出了本发明智能设备拍摄引导方法的流程图；
图2示出了本发明智能设备取景镜头获取取景图像的示意图；
图3示出了本发明智能设备拼接-裁剪获取最佳美感区域的流程图；
图4示除了本发明训练第一美学评价模型获取最佳美感区域的流程图，
图5示出了本发明用于对不能读取EXIF信息的取景图像进行评价的单模型训练流程图；
图6示出了本发明用于对不能读取EXIF信息的取景图像进行评价的多模型训练流程图；
图7示出了本发明用于对能读取EXIF信息的取景图像进行评价的单模型训练流程图；
图8示出了本发明用于对能读取EXIF信息的取景图像进行评价的多模型训练流程图；
图9示出了不能读取EXIF信息的取景图像的单模型拍摄引导流程图；
图10示出了不能读取EXIF信息的取景图像的多模型拍摄引导流程图；
图11示出了能读取EXIF信息的取景图像的单模型拍摄引导流程图；
图12示出了能读取EXIF信息的取景图像的多模型拍摄引导流程图；
图13示出了本发明用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训练流程图；
图14示出了本发明用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训练流程图；
图15示出了本发明用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训练流程图；
图16示除了本发明用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训练流程图；
图17示出了不能读取EXIF信息的具有最佳美感区域的图像的单模型拍摄引导流程图；
图18示出了不能读取EXIF信息的具有最佳美感区域的图像的多模型拍摄引导流程图；
图19示出了图能读取EXIF信息的具有最佳美感区域的图像的单模 型拍摄拍摄流程图；
图20示出了能读取EXIF信息的具有最佳美感区域的图像的多模型拍摄引导流程图；
图21示出了本发明智能设备引导人物拍摄的示意图。
具体实施方式
通过参考示范性实施例，本发明的目的和功能以及用于实现这些目的和功能的方法将得以阐明。然而，本发明并不受限于以下所公开的示范性实施例；可以通过不同形式来对其加以实现。说明书的实质仅仅是帮助相关领域技术人员综合理解本发明的具体细节。
在下文中，将参考附图描述本发明的实施例，相关技术术语应当是本领域技术人员所熟知的。在附图中，相同的附图标记代表相同或类似的部件，或者相同或类似的步骤，除非另有说明。下面通过具体的实施例对本发明的内容进行说明。
在下文中解决具体的实施例对本发明提供的一种基于美学评价的照片自动评分方法进行详细说明，现有技术中对照片图像的处理缺乏根据不同场景对照片图像进行评分，使得在面对海量质量参差不齐的照片时，无法实现从美学角度进行照片质量的分类。
本发明提供的一种基于美学评价的照片自动评分方法执行在计算设备，计算设备包括但不限于个人个人电脑、智能手机、平板电脑、智能眼镜，以及其他具有摄像头、数据存储和数据处理功能的设备。
为了解决现有技术拍摄照片时美学质量不能保证的问题，本发明提供了一种智能设备的拍摄引导方法，以下结合附图以及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不限定本发明。
根据本发明，实施例中提供了一种智能设备的拍摄引导方法，如图1所示，本发明智能设备拍摄引导方法的流程图，智能设备的拍摄引导方法包括：
S101、获取智能设备中的当前取景图像。
S102、按照一定的规律扰动所述智能设备的取景镜头，记录扰动时 各个运动传感器信息，获取拍摄环境邻近的多个不同位置的取景图像。
S103、使用图像拼接算法将所述当前取景图像和所述多个不同位置的取景图像拼接成一幅大图，通过智能剪裁算法获取最佳美感区域，即为取景推荐区域。
S104、训练第一美学评价模型，将所述当前取景图像和所述多个不同位置的取景图像输入到所述第一美学评价模型中，输出最佳美感区域，即为取景推荐区域。
S105、在所述最佳美感区域中获取特写物体拍摄推荐位置，包括在所述最佳美感区域中生成多个虚拟位置，具有所述最佳美感区域的图像结合多个虚拟位置进行第二级美学评分，根据所述第二美学评分得到最佳美学评分的虚拟位置，即特写物体拍摄推荐位置并显示，或者
训练第二美学评价模型，将不同类的具有最佳美感区域的图像输入到第二美学评价模型，输出特写物体拍摄推荐位置并显示。
如图2所示本发明智能设备取景镜头获取取景图像的示意图，本发明智能设备包括个人电脑、智能手机、平板电脑、智能眼镜，以及其他具有摄像头及数据处理功能的设备，实施例中示例性的以手机作为载体，智能设备手机具有取景镜头(摄像头)、存储器和处理器芯片。取景镜头获取的图像存储至存储器，由处理器芯片处理后选出最佳美感区域，以指导用户进行拍摄。在一些实施例中，计算设备可以是直接进行拍摄图像的相机或其他本领域技术人员所公知的智能设备。如图2所示，手机100前端显示屏幕101显示取景图像，根据本发明按照一定规律扰动取景镜头，记录手机各个运动传感器(例如陀螺仪或海拔仪)信息，获取拍摄环境邻近的多个不同位置的取景图像。实施例中示例性的以四个不同位置的取景图像为例，扰动取景镜头按照箭头方向依次获取a、b、c、d四个位置的取景图像。对获取的当前取景图像101和不同位置的取景图像通过拼接-剪裁的方法获取最佳美感区域，获取通过训练第一美学评价模型输出最佳美感区域。在下面实施例中将详细说明获取最佳美感区域的过程。
拼接-剪裁获取最佳美感区域
如图3所示本发明智能设备拼接-裁剪获取最佳美感区域的流程图，实施例中使用图像拼接算法将所述当前取景图像和所述多个不同位置的取景图像拼接成一幅大图，通过智能剪裁算法获取最佳美感区域。具体地：
S201、将当前取景图像和多个不同位置的取景图像拼接成一幅大图，实施例中，将当前取景图像101和四个位置的取景图像(a、b、c、d四个取景图像)拼接成一幅大图。
S202、在拼接成的大图中生成若干子区域，计算若干子区域的第一级美学评分，根据计算得到的第一级美学评分得到最佳美感剪裁区域。具体的，在大图中生成若干子区域包括用预设的滑动窗口遍历拼接成的大图，得到若干个子区域。
根据本发明，实施例中得到最佳美感剪裁区域通过两个方案实现，其中
第一种方案为：
使用第一滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，以最高第一级美学评分对应的位置作为最佳美感裁剪区域。第一滑动窗口选取以下中的一种或几种：原始比例滑动窗口、标准比例滑动窗口、预设比例滑动窗口。
当第一滑动窗口中包括预设比例滑动窗口时，使用第一滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，包括以下步骤：
使用预设比例滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分。
更加具体的，当第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时，使用第一滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，包括以下方案A和方案B。
具体的，方案A包括以下步骤：
使用原始比例滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中原始比例滑动窗口对应的若干个第一级美学评分。
使用标准比例滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中标准比例滑动窗口对应的若干个第一级美学评分。
在所述原始比例滑动窗口对应的若干个第一级美学评分、及所述标准比例滑动窗口对应的若干个第一级美学评分中选取最高第一级美学评分。
具体的，方案B包括以下步骤：
使用原始比例滑动窗口遍历所述拼接成的大图，利用预设的第一美学评价模型得到在遍历的过程中原始比例滑动窗口对应的若干个第一级美学评分。
选取原始比例滑动窗口对应的若干个第一级美学评分中最高的第一级美学评分，根据最高的第一级美学评分对应的原始比例滑动窗口的坐标得到原始比例下的最佳美感裁剪区域。
计算所述标准比例滑动窗口与所述原始比例下的最佳美感裁剪区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域。
按照指令选择原始比例下的最佳美感裁剪区域或标准比例下的最佳美感裁剪区域中的一个作为图像最佳美感裁剪区域。
更加具体的，在方案A和方案B中，原始比例滑动窗口和标准比例滑动窗口可选用为几个尺度。例如当原始比例滑动窗口选用3个尺度，各尺度的缩放比为1.2，最小尺度滑动窗口的水平方向边长为原始图像(拼接成的大图)水平方向长边的1/4时，即表示选用以下三种尺度的原始比例图像：大小为原始图像大小的0.25倍、大小为原始图像大小的0.25×1.2倍、大小为原始图像大小的0.25×1.2×1.2倍。
第二种方案为：
使用第二滑动窗口遍历所述拼接成的大图，得到在遍历的过程中所述第二滑动窗口对应的若干个第一级美学评分，以所述若干个第一级美学评分作为遍历过程中若干个第二滑动窗口中心点坐标的美学质量值，根据若干个美学质量值生成美学质量图谱。其中，所述第二滑动窗口为照片图像原始比例，且第二滑动窗口小于或等于第一滑动窗口，第二滑动窗口步长小于或等于第一滑动窗口；
使用预设的聚类算法得到所述美学质量图谱中的能量最高区域，以所述能量最高区域的最小外接矩形作为大图的最佳预选区域；
计算所需裁剪比例的窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域。
具体的，当所述第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时，计算第一滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域，包括以下步骤：
计算原始比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为原始比例下的最佳美感裁剪区域；
计算标准比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域；
在所述原始比例下的最佳美感裁剪区域、及标准比例下的最佳美感裁剪区域中选取中心与所述最佳预选区域中心最近的一个作为拼接成的大图的最佳美感裁剪区域。
作为本发明方法实施例的另一实施例，还包括以下步骤：获取指定的目标裁剪区域种子点；进一步的，所述使用预设的滑动窗口遍历所述拼接成的大图，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分，包括以下步骤：
使用预设的滑动窗口遍历所述目标裁剪区域种子点邻域，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分。
S203、根据第一个方案和/或第二个方案得到最佳美感裁剪区域的坐标裁剪所述拼接成的大图，得到拼接成的大图的最佳裁剪区域，将得到最佳剪裁区域作为最佳美感区域。
本发明实施例获取的当前取景图像和多个不同位置的图像拼接成一幅大图，并对大图进行智能剪裁，获得最佳美感区域，利用最佳美感区域指导用户进行拍摄图像，使得拍摄的图像更加符合应用场景，更加具有美学感受。
训练第一美学评价模型，获取最佳美感区域
如图4所示本发明训练第一美学评价模型获取最佳美感区域的流程图，实施例中，将当前取景图像和多个不同位置的取景图像输入到第一美学评价模型中获取最佳美感区域，包括：
S301、训练第一美学评价模型，具体第一美学评价模型的训练将在下文中详细阐释。
S302、获取当前取景图像和多个不同位置的取景图像。
S303、采用图像场景分类算法对当前取景图像和多个不同位置的取景图像识别并进行场景分类。
S304、将取景图像的图像信息输入到第一美学评价模型中进行评分，并输出最佳美感区域。
应当理解步骤S304中所称的取景图像包括当前取景图像和多个不同位置的取景图像，具体地本实施例中包括当前取景图像101和四个不同位置的取景图像(a、b、c、d四个位置的取景图像)。
本发明的第一美学评价模型包括人物类单模型和景物类单模型，以及人物类多模型和景物类多模型，第一美学评价模型针对需要评分的照片是否为特写照片进行分类训练，训练过程通过机器学习方法进行训练得到，使用的机器学习方法包括卷积神经网络、受限玻尔兹曼机、深度置信网络等。
具体地，对于不能读取EXIF信息的取景图像(包括当前取景图像和多个不同位置的取景图像，下文中将不再累述)，第一美学评价模型训练后得到人物类单模型和景物类单模型，以及人物类多模型和景物类多模型，第一美学评价模型通过机器学习训练得到。
如图5所示本发明用于对不能读取EXIF信息的取景图像进行评价的单模型训练流程图，用于对不能读取EXIF信息的取景图像进行评价的单 模型训练按照如下方法进行训练：
S401、获取已知第一级美学评分、图像场景类别标签、感兴趣区域位置坐标和EXIF信息的若干个图像。
S402、利用已知第一级美学评分、图像场景类别标签和感兴趣区域位置坐标的若干个图像对预设的第一美学评价模型进行训练，得到单模型。
如图6所示本发明用于对不能读取EXIF信息的取景图像进行评价的多模型训练流程图，用于对不能读取EXIF信息的图像进行评价的多模型训练按照如下方法进行训练：
S403、获取已知第一级美学评分、图像场景类别标签、感兴趣区域位置坐标和EXIF信息的若干个图像。
S404、利用已知第一级美学评分和感兴趣区域位置坐标的若干个图像，根据图像所属的场景标签的不同，对预设的第一美学评价模型分别训练出与多个场景对应的多模型，包括对人物类图像照片和景物类图像照片进行多模型训练分别得到人物类图像美学评价的模型和景物类图像美学评价的模型，本领域技术人员应当理解并不限于此。
对于能读取EXIF信息的取景图像，第一美学评价模型训练后得到单模型和多模型。
如图7所示本发明用于对能读取EXIF信息的取景图像进行评价的单模型训练流程图，用于对能读取EXIF信息的取景图像进行评价的单模型训按照如下方法进行训练：
S405、获取已知第一级美学评分、图像场景类别标签、感兴趣区域位置坐标和EXIF信息的若干个图像。
S406、利用所述已知第一级美学评分、图像场景类别标签、感兴趣区域位置坐标和EXIF信息的若干个图像对预设的第一美学评价模型进行训练，得到所述单模型。
如图8所示本发明用于对能读取EXIF信息的取景图像进行评价的多模型训练流程图，用于对能读取EXIF信息的取景图像进行评价的多模型训按照如下方法进行训练：
S407、获取已知第一级美学评分、图像场景类别标签、感兴趣区域位置坐标和EXIF信息的若干个图像。
S408、利用已知第一级美学评分、感兴趣区域位置坐标和EXIF信息的若干个图像，根据图像所属的场景标签的不同，对预设的第一美学评价模型分别训练出与多个场景对应的多模型，包括对人物类图像照片和景物类图像照片进行多模型训练分别得到人物类图像美学评价的模型和景物类图像美学评价的模型，本领域技术人员应当理解并不限于此。
下面实施例中具体说明本发明智能设备的拍摄引导过程：由上文中训练得到的不同种类的第一美学评价模型对取景图像进行自动评分，根据评分进行拍摄引导。具体地，
对由智能设备中采集的取景图像采用图像场景分类算法对进行图像识别并进行场景分类，并判断照片是否为特写照片。
采用图像场景分类算法对照片进行图像识别，根据识别结果将照片分为风景、夜景、建筑、动态、静态、逆光、人像、动物、植物多个类别。
实施例中，场景分类算法包括以下步骤实现取景图像的场景分类：
采用物体识别算法识别图像中的物体。
根据上下文语义模型分析出物体识别算法识别出的图像中物体之间的关联。
根据分析出的结果可将图像归类为风景、夜景、建筑、动态、静态、逆光、人像、动物、植物多个类别，即得到图像所属场景分类结果。
应当理解对于场景分类算，本实施例中只是示例性的进行说明，本领域技术人员可以选择其他所公知的场景分类方法进行场景分类。
取景图像是否为特写照片通过如下方法判断：
通过显著性检测算法检测取景图像中的显著性区域；
当最大显著性区域面积/面积比例大于阈值时，则取景图像为特写照片，否则为非特写照片。
如图9所示不能读取EXIF信息的取景图像的单模型拍摄引导流程图，当取景图像为不能读取EXIF信息时，则取景图像的图像信息包括图 像场景分类结果和图像本身。将取景图像的图像信息输入到步骤S402得到的单模型进行评分，输出取景图像的第一级美学评分和感兴趣区域位置坐标，将感兴趣区域位置坐标作为最佳美感区域引导用户拍摄。例如对于人物或动物或植物的场景，分别输出对应于人物或动物或植物的感兴趣区域位置坐标，并作为对应的最佳美感区域。应当理解，对于输入单模型中的场景分类结果是图像场景不同类的类别标签。
如图10所示不能读取EXIF信息的取景图像的多模型拍摄引导流程图，当取景图像为不能读取EXIF信息时，则取景图像的图像信息包括图像场景分类结果和图像本身。根据图像的场景分类结果，将不同类别场景的图像本身输入步骤S404得到多模型对应场景的第一美学评价模型进行评分，输出取景图像的第一级美学评分和感兴趣区域位置坐标，将感兴趣区域位置坐标作为最佳美感区域。实施例中人物类取景图像输入到人物类图像美学评价的模型和景物类取景图像输入到景物类图像美学评价的模型进行评分，分别得到人物类取景图像的感兴趣区域位置坐标和景物类取景图像的感兴趣区域位置坐标。本领域技术人员应当理解并不限于此。
如图11所示图能读取EXIF信息的取景图像的单模型拍摄拍摄流程图，当取景图像为能读取EXIF信息时，则图像信息包括图像场景分类结果、图像照片EXIF、图像本身。将取景图像的图像信息输入到步骤S406得到的单模型进行评分，输出取景图像的第一级美学评分和感兴趣区域位置坐标，将感兴趣区域位置坐标作为最佳美感区域引导用户拍摄。例如对于人物或动物或植物的场景，分别输出对应于人物或动物或植物的感兴趣区域位置坐标，并作为对应的最佳美感区域。应当理解，对于输入单模型中的场景分类结果是图像场景不同类的类别标签。
如图12所示能读取EXIF信息的取景图像的多模型拍摄引导流程图，当取景图像为能读取EXIF信息时，则取景图像的图像信息包括图像场景分类结果、图像照片EXIF和图像本身。根据图像的场景分类结果，将不同类别场景的图像照片EXIF和图像本身输入步骤S408得到多模型对应场景的第一美学评价模型进行评分，输出取景图像的第一级美学评分和感兴趣区域位置坐标，将感兴趣区域位置坐标作为最佳美感区域。实施 例中人物类图像照片输入到人物类图像美学评价的模型和景物类图像照片输入到景物类图像美学评价的模型进行评分，分别得到人物类取景图像的感兴趣区域位置坐标和景物类取景图像的感兴趣区域位置。本领域技术人员应当理解并不限于此。
上文中详细阐述了获取不同类最佳美感区域的具体方法，下面详细阐释本发明根据得到最佳美感区域获得拍摄引导推荐的位置的过程。
在最佳美感区域中获取拍摄引导位置坐标，包括在最佳美感区域中生成多个虚拟位置，具有所述最佳美感区域的图像结合多个虚拟位置进行第二级美学评分，根据所述第二美学评分得到拍摄引导推荐的位置并显示，或者
训练第二美学评价模型，将不同类的具有最佳美感区域的图像输入到第二美学评价模型，输出拍摄引导推荐的位置。
需要说明的是，这里的虚拟位置包括但不限于具体人物的虚拟位置、具体植物的虚拟位置和具体动物的虚拟位置。例如对人物拍摄进行拍摄引导时，生成若干个人物的虚拟站位。
生成虚拟位置，获得特写物体拍摄推荐位置
S501，在具有最佳美感区域的图像的最佳美感区域中生成若干个虚拟位置。
具体的，在具有最佳美感区域的图像的最佳美感区域中生成若干个虚拟站位包括以下步骤：
在最佳美感区域中通过多尺度滑动窗口遍历，得到若干个虚拟位置；优选地，滑动窗口遍历过程中选取背景物体距离较远的区域对应的位置信息作为候选的虚拟站位信息。
S502，具有最佳美感区域的图像结合若干个虚拟位置分别进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置并显示。
具体的，推荐位置选取最佳位置或美学评分大于等于最低阈值的多个虚拟位置中的一个或多个。
更加具体的，选取最高美学评分对应的虚拟位置作为最佳位置；美学评分大于等于最低阈值的多个虚拟位置按照美学评分由高至低的顺序进行排列。
具体的，具有最佳美感区域的图像分别结合若干个虚拟位置进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置包括：
将具有最佳美感区域的图像与任意一个虚拟位置输入到预设的美学评价模型中对虚拟位置进行第二级美学评分，遍历所述若干个虚拟位置，得到若干个第二级美学评分；
根据所述若干个第二级美学评分得到拍摄引导推荐的位置。
训练第二美学评价模型，获得特写物体拍摄推荐位置
根据本发明，实施例中通过第二美学评价模型获得拍摄引导推荐的位置，为了更加清晰的说明本发明的内容，实施例中以获取具体人物的位置、具体动物的位置和具体植物的位置为例，但应当理解并不限于此。
本发明的第二美学评价模型包括单模型和多模型，第二美学评价模型针对具有最佳美感区域的图像是否为特写照片进行分类训练，训练过程通过机器学习方法进行训练得到，使用的机器学习方法包括卷积神经网络、受限玻尔兹曼机、深度置信网络等。
具体地，对于不能读取EXIF信息的具有最佳美感区域的图像，第二美学评价模型训练后得到单模型和多模型，第二美学评价模型通过机器学习训练得到。
如图13所示本发明用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训练流程图，用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训练按照如下方法进行训练：
S601、获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像。
S602、利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型。
如图14所示本发明用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训练流程图，用于对不能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训练按照如下方法进行训练：
S603、获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像。
S604、利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
对于能读取EXIF信息的具有最佳美感区域的图像，第二学评价模型训练后得到单模型和多模型。
如图15所示本发明用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训练流程图，用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的单模型训按照如下方法进行训练：
S605、获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像。
S606、利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型。
如图16所示本发明用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训练流程图，用于对能读取EXIF信息的具有最佳美感区域的图像进行评价的多模型训按照如下方法进行训练：
S607、获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像。
S608、利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
下面实施例中具体说明本发明智能设备的拍摄引导过程：由上文中训练得到的不同种类的第二美学评价模型对具有最佳美感区域的图像进行自动评分，在最佳美感区域中获得特写物体拍摄推荐位置。具体地，
如图17所示不能读取EXIF信息的具有最佳美感区域的图像的单模型拍摄引导流程图，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果和图像本身。 将具有最佳美感区域的图像信息输入到步骤S602得到的单模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即特写物体拍摄推荐位置。
如图18所示不能读取EXIF信息的具有最佳美感区域的图像的多模型拍摄引导流程图，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果和图像本身。根据图像的场景分类结果，将不同类别场景的图像本身输入步骤S604得到多模型对应场景的第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即特写物体拍摄推荐位置。
如图19所示图能读取EXIF信息的具有最佳美感区域的图像的单模型拍摄拍摄流程图，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF、图像本身。将具有最佳美感区域的图像信息输入到步骤S606得到的单模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即特写物体拍摄推荐位置。
如图20所示能读取EXIF信息的具有最佳美感区域的图像的多模型拍摄引导流程图，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF和图像本身。根据图像的场景分类结果，将不同类别场景的图像照片EXIF和图像本身输入步骤S608得到多模型对应场景的第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即特写物体拍摄推荐位置。
如图21所示本发明智能设备引导人物拍摄的示意图，实施例中示例性的给出人物拍摄引导过程，在手机100的显示界面101中显示人物拍摄的最佳美感区域中的拍摄引导推荐的位置102，将需要拍摄的人物置于显示界面101的拍摄引导推荐的位置102内进行拍摄。
本发明智能识别拍摄场景，给出推荐站位，使得不具备专业摄影知识的用户也能轻松拍出高美感的图像，本发明通过图像美学评价技术预先对多个场景的照片拼接成的大图进行智能裁剪，使得从美学角度裁剪出最佳美感区域，进而指导用户进行拍摄。
应当理解，上述实施例中使用最佳美感区域图像特写物体拍摄推荐位置，在另一些实施例中也可以使用任意图像生成特写物体拍摄推荐位置(人物、动物在背景中的站位)，在再一些实施例中，也可以单独生成最佳美感区域(例如在拍摄风景图片时)。
本发明采用图像场景分类算法对不同场景进行自适应地美学评价，通过训练美学评价模型进行评分，输出最佳美感区域，指导用户进行拍摄。
本发明利用了照片EXIF信息，可以更准确的获取照片拍摄时的各项参数，保证美学评分更加准确。
本发明利用了图像特写主体物体位置信息，考虑到了主体的构图因素，因此对物体为主体的图像的美学评分更加准确。
本发明中的美学评价模型通过机器学习方法训练得到，避免了人工规则评价的主观性和片面性。
本发明的提供的一种智能设备的拍摄引导方法，有助于引导用户进行风景和人物等类照片的拍摄，并针对不同类照片进一步推荐出最佳的拍摄引导位置，从而拍摄出更具美感的图像。
结合这里披露的本发明的说明和实践，本发明的其他实施例对于本领域技术人员都是易于想到和理解的。说明和实施例仅被认为是示例性的，本发明的真正范围和主旨均由权利要求所限定。

Claims (21)

  1. 一种智能设备的拍摄引导方法，其特征在于，所述方法包括：
    a)获取智能设备中的当前取景图像；
    b)按照一定的规律扰动所述智能设备的取景镜头，记录扰动时各个运动传感器信息，获取拍摄环境邻近的多个不同位置的取景图像；
    c)使用图像拼接算法将所述当前取景图像和所述多个不同位置的取景图像拼接成一幅大图，通过智能剪裁算法生成若干个子区域，计算所述若干个子区域的第一级美学评分，获取最佳美感区域；或者
    训练第一美学评价模型，将所述当前取景图像和所述多个不同位置的取景图像输入到所述第一美学评价模型中，输出最佳美感区域；
    d)在所述最佳美感区域中获取特写物体拍摄推荐位置，包括
    在所述最佳美感区域中生成多个虚拟位置，具有所述最佳美感区域的图像结合多个虚拟位置进行第二级美学评分，根据所述第二美学评分得到最佳美学评分的虚拟位置，即特写物体拍摄推荐位置并显示，或者
    训练第二美学评价模型，将不同类的具有最佳美感区域的图像输入到第二美学评价模型，输出特写物体拍摄推荐位置并显示。
  2. 根据权利要求1所述的方法，其特征在于，通过智能剪裁算法获取最佳美感区域包括：
    将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域，将所述最佳美感剪裁区域作为最佳美感区域。
  3. 根据权利要求2所述的方法，其特征在于，将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域，包括如下步骤：
    使用第一滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，以最高第一级美学评分对应的位置作为最佳美感裁剪区域；
    其中，第一滑动窗口选取以下中的一种或几种：原始比例滑动窗口、标准比例滑动窗口、预设比例滑动窗口。
  4. 根据权利要求2所述的方法，其特征在于，将拼接成的所述大图生成若干子区域，计算所述若干子区域的第一级美学评分，根据所述第一级美学评分得到最佳美感裁剪区域，包括如下步骤：
    使用第二滑动窗口遍历所述大图，得到在遍历的过程中所述第二滑动窗口对应的若干个第一级美学评分，以所述若干个第一级美学评分作为遍历过程中若干个第二滑动窗口中心点坐标的美学质量值，根据若干个美学质量值生成美学质量图谱；其中，所述第二滑动窗口为大图原始比例，且第二滑动窗口小于或等于第一滑动窗口，第二滑动窗口步长小于或等于第一滑动窗口；
    使用预设的聚类算法得到所述美学质量图谱中的能量最高区域，以所述能量最高区域的最小外接矩形作为最佳预选区域；
    计算所需裁剪比例的窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域。
  5. 根据权利要求2至4中任一权利要求所述的方法，其特征在于，还包括以下步骤：获取指定的目标裁剪区域种子点；
    所述使用预设的滑动窗口遍历所述大图，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分，包括以下步骤：
    使用预设的滑动窗口遍历所述目标裁剪区域种子点邻域，得到在遍历的过程中所述预设的滑动窗口对应的若干个第一级美学评分。
  6. 根据权利要求3所述的方法，其特征在于，
    当所述第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时；
    使用第一滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述第一滑动窗口对应的若干个第一级美学评分，选取最高第一级美学评分，包括以下步骤：
    使用原始比例滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述原始比例滑动窗口对应的若干个第一级美学 评分；
    使用标准比例滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述标准比例滑动窗口对应的若干个第一级美学评分；
    在所述原始比例滑动窗口对应的若干个第一级美学评分、及所述标准比例滑动窗口对应的若干个第一级美学评分中选取最高第一级美学评分；
    或者使用原始比例滑动窗口遍历所述大图，利用预设的第一美学评价模型得到在遍历的过程中所述原始比例滑动窗口对应的若干个第一级美学评分；
    选取原始比例滑动窗口对应的若干个第一级美学评分中最高的第一级美学评分，根据最高的第一级美学评分对应的原始比例滑动窗口的坐标得到原始比例下的最佳美感裁剪区域；
    计算所述标准比例滑动窗口与所述原始比例下的最佳美感裁剪区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域；
    按照指令选择原始比例下的最佳美感裁剪区域或标准比例下的最佳美感裁剪区域中的一个作为最佳美感裁剪区域。
  7. 根据权利要求4所述的照片智能裁剪方法，其特征在于，
    当所述第一滑动窗口包括原始比例滑动窗口和标准比例滑动窗口时；
    计算第一滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为最佳美感裁剪区域，包括以下步骤：
    计算原始比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为原始比例下的最佳美感裁剪区域；
    计算标准比例滑动窗口与所述最佳预选区域的最大交并比，将产生最大交并比的位置作为标准比例下的最佳美感裁剪区域；
    在所述原始比例下的最佳美感裁剪区域、及标准比例下的最佳美感裁剪区域中选取中心与所述最佳预选区域中心最近的一个作为最佳美感 裁剪区域。
  8. 根据权利要求1所述的方法，其特征在于，所述当前取景图像和所述多个不同位置的取景图像输入到所述第一美学评价模型，输出最佳美感区域，包括：
    采用图像场景采用分类算法对所述当前取景图像和所述多个不同位置的取景图像识别并进行场景分类，并将所述取景图像的图像信息输入到所述第一美学评价模型中进行评分，并输出最佳美感区域。
  9. 根据权利要求8所述的方法，其特征在于，若所述取景图像不能读取EXIF信息，则所述图像信息包括图像场景分类结果和图像本身。
  10. 根据权利要求8所述的方法，其特征在于，若所述取景图像可以读取EXIF信息，则所述图像信息包括图像场景分类结果、图像照片EXIF、图像本身。
  11. 根据权利要求1所述的方法，其特征在于，在所述最佳美感区域中生成多个虚拟位置包括：
    在最佳美感区域中通过多尺度滑动窗口遍历，得到若干个虚拟位置。
  12. 根据权利要求11所述的方法，其特征在于，滑动窗口遍历过程中选取背景物体距离较远的区域对应的位置信息作为候选的虚拟位置信息。
  13. 根据权利要求1所述的方法，其特征在于，具有最佳美感区域的图像结合若干个虚拟位置分别进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置并显示包括：
    推荐位置选取最佳位置或美学评分大于等于最低阈值的多个虚拟位置中的一个或多个。
  14. 根据权利要求13所述的方法，其特征在于，选取最高美学评分对应的虚拟位置作为最佳位置；美学评分大于等于最低阈值的多个虚拟位置按照美学评分由高至低的顺序进行排列。
  15. 根据权利要求1所述的方法，其特征在于，具有最佳美感区域的图像分别结合若干个虚拟位置进行第二级美学评分，根据第二级美学评分得到拍摄引导推荐的位置包括：
    将具有最佳美感区域的图像与任意一个虚拟位置输入到预设的美学 评价模型中对虚拟位置进行第二级美学评分，遍历所述若干个虚拟位置，得到若干个第二级美学评分；根据所述若干个第二级美学评分得到拍摄引导推荐的位置。
  16. 根据权利要求1所述的方法，其特征在于，所述第二美学评价模型包括单模型和多模型，针对不能读取EXIF信息的具有最佳美感区域的图像，所述第二美学评价模型按如下方法训练：
    获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像；
    利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型，或者
    利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
  17. 根据权利要求1所述的方法，其特征在于，所述第二美学评价模型包括单模型和多模型，针对能读取EXIF信息的具有最佳美感区域的所述第二美学评价模型按如下方法训练：
    获取已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像。
    利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像对预设的第二美学评价模型进行训练，得到单模型，或者
    利用已知第二级美学评分、图像场景类别标签、人物或动物或植物位置坐标、EXIF信息位置的若干个具有最佳美感区域的图像，根据图像所属的场景标签的不同，对预设的第二美学评价模型分别训练出与多个场景对应的多模型。
  18. 根据权利要求1所述的方法，其特征在于，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括 图像场景分类结果和图像本身，将具有最佳美感区域的图像信息输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
  19. 根据权利要求1所述的方法，其特征在于，当具有最佳美感区域的图像为不能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果和图像本身，根据图像的场景分类结果，将不同类别场景的图像本身输入第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
  20. 根据权利要求1所述的方法，其特征在于，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF、图像本身，将具有最佳美感区域的图像信息输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标。
  21. 根据权利要求1所述的方法，其特征在于，当具有最佳美感区域的图像为能读取EXIF信息时，则具有最佳美感区域的图像信息包括图像场景分类结果、图像照片EXIF和图像本身，根据图像的场景分类结果，将不同类别场景的图像照片EXIF和图像本身输入到第二美学评价模型进行评分，输出第二级美学评分和人物或动物或植物的位置坐标，即拍摄引导推荐的位置。
PCT/CN2017/116058 2017-04-19 2017-12-14 一种智能设备的拍摄引导方法 WO2018192244A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201710257194.7 2017-04-19
CN201710256861.X 2017-04-19
CN201710256861.XA CN107146198B (zh) 2017-04-19 2017-04-19 一种照片智能裁剪方法及装置
CN201710257194.7A CN107018330A (zh) 2017-04-19 2017-04-19 一种实时拍照指导方法及装置

Publications (1)

Publication Number Publication Date
WO2018192244A1 true WO2018192244A1 (zh) 2018-10-25

Family

ID=63855654

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/116058 WO2018192244A1 (zh) 2017-04-19 2017-12-14 一种智能设备的拍摄引导方法

Country Status (1)

Country Link
WO (1) WO2018192244A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220210334A1 (en) * 2020-12-29 2022-06-30 Industrial Technology Research Institute Movable photographing system and photography composition control method
CN117478825A (zh) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 虚拟会议的视频集锦获取方法、系统、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101674404A (zh) * 2008-09-08 2010-03-17 索尼株式会社 摄影设备和方法及程序
US20100171874A1 (en) * 2009-01-05 2010-07-08 Samsung Electronics Co., Ltd. Apparatus and method for photographing image in portable terminal
CN103945129A (zh) * 2014-04-30 2014-07-23 深圳市中兴移动通信有限公司 基于移动终端的拍照预览构图指导方法及系统
CN104182970A (zh) * 2014-08-08 2014-12-03 浙江大学 一种基于摄影构图规则的留念照人像站位推荐方法
CN104717413A (zh) * 2013-12-12 2015-06-17 北京三星通信技术研究有限公司 拍照辅助方法及设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101674404A (zh) * 2008-09-08 2010-03-17 索尼株式会社 摄影设备和方法及程序
US20100171874A1 (en) * 2009-01-05 2010-07-08 Samsung Electronics Co., Ltd. Apparatus and method for photographing image in portable terminal
CN104717413A (zh) * 2013-12-12 2015-06-17 北京三星通信技术研究有限公司 拍照辅助方法及设备
CN103945129A (zh) * 2014-04-30 2014-07-23 深圳市中兴移动通信有限公司 基于移动终端的拍照预览构图指导方法及系统
CN104182970A (zh) * 2014-08-08 2014-12-03 浙江大学 一种基于摄影构图规则的留念照人像站位推荐方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220210334A1 (en) * 2020-12-29 2022-06-30 Industrial Technology Research Institute Movable photographing system and photography composition control method
US11445121B2 (en) * 2020-12-29 2022-09-13 Industrial Technology Research Institute Movable photographing system and photography composition control method
CN117478825A (zh) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 虚拟会议的视频集锦获取方法、系统、设备及存储介质
CN117478825B (zh) * 2023-12-27 2024-04-16 苏州元脑智能科技有限公司 虚拟会议的视频集锦获取方法、系统、设备及存储介质

Similar Documents

Publication Publication Date Title
CN108764091B (zh) 活体检测方法及装置、电子设备和存储介质
US11321385B2 (en) Visualization of image themes based on image content
CN105981368B (zh) 在成像装置中的照片构图和位置引导
CN107146198B (zh) 一种照片智能裁剪方法及装置
KR101605983B1 (ko) 얼굴 검출을 이용한 이미지 재구성
US8938100B2 (en) Image recomposition from face detection and facial features
US10110868B2 (en) Image processing to determine center of balance in a digital image
US10523916B2 (en) Modifying images with simulated light sources
US20150178592A1 (en) Image capture feedback
US11574424B2 (en) Augmented reality map curation
US9008436B2 (en) Image recomposition from face detection and facial features
US20130108168A1 (en) Image Recomposition From Face Detection And Facial Features
CN109040605A (zh) 拍摄引导方法、装置及移动终端和存储介质
US20130108171A1 (en) Image Recomposition From Face Detection And Facial Features
US20130108170A1 (en) Image Recomposition From Face Detection And Facial Features
US20130108166A1 (en) Image Recomposition From Face Detection And Facial Features
US20200134296A1 (en) Automated image capture based on emotion detection
WO2018192245A1 (zh) 一种基于美学评价的照片自动评分方法
US11954301B2 (en) Systems and methods for analyzing and organizing digital photos and videos
US9025835B2 (en) Image recomposition from face detection and facial features
CN108319937A (zh) 人脸检测方法及装置
US20210012503A1 (en) Apparatus and method for generating image
CN110826487B (zh) 一种人脸表情数据采集方法
WO2018192244A1 (zh) 一种智能设备的拍摄引导方法
CN113781408B (zh) 一种图像拍摄智能指导系统及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17906154

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17906154

Country of ref document: EP

Kind code of ref document: A1