CN110519509A

CN110519509A - Composition evaluation method, method for imaging, device, electronic equipment, storage medium

Info

Publication number: CN110519509A
Application number: CN201910708006.7A
Authority: CN
Inventors: 唐琪
Original assignee: Fantasy Power (shanghai) Culture Communication Co Ltd
Current assignee: Fantasy Power (shanghai) Culture Communication Co Ltd
Priority date: 2019-08-01
Filing date: 2019-08-01
Publication date: 2019-11-29

Abstract

The present invention provides composition evaluation method, method for imaging, device, electronic equipment, storage mediums, comprising: extracts the characteristic information of image in the frame of composition frame, the composition frame includes photographic subjects；The composition frame is evaluated based on the characteristic information, obtains the first assessment of the composition frame；The composition frame is evaluated based on the composition evaluation model of deep learning method, obtains the second assessment of the composition frame；According to the first assessment and the second assessment, the overall merit of the composition frame is obtained.The present invention has done comprehensive evaluation to photography composition from the shallow-layer feature and further feature of image, so that output meets the photo of human aesthetic's requirement.

Description

Composition evaluation method, method for imaging, device, electronic equipment, storage medium

Technical field

The present invention relates to automatic photography technical field, espespecially composition evaluation method, method for imaging, device, electronic equipment, deposit Storage media.

Background technique

In production and living practice, people using the cameras such as camera by optical imaging concept, with obtain image, Image, and record image, image.In general, needing such as photographer by manual operation camera, to obtain default Image, the image of photographic subjects.Typically, the basis of photography is composition, that is to say selection photographic subjects and will shoot mesh It is marked with a certain size, ratio is placed in corresponding composition frame, and finally showed in picture, and then obtain to hold accordingly In the material for carrying photographic subjects relevant information.

Good composition can make photo clearly express artistic conception and site environment atmosphere, however how evaluate the quality of composition It is a very difficult and complicated thing, composition evaluation is a very subjective problem in itself.Traditional composition evaluation side Method is mostly the experience in conjunction with photographer, does aesthetic assessment to composition from bottom visual information.How from the angle of quantization more comprehensively The quality of composition is evaluated on ground, and being one is worth the problem of further exploring.

Summary of the invention

An object of the present invention is to provide composition evaluation side to overcome at least partly deficiency existing in the prior art Method, method for imaging, device, electronic equipment, storage medium do comprehensively photography composition from the shallow-layer feature and further feature of image Evaluation, thus output meet human aesthetic requirement photo.

Technical solution provided by the invention is as follows:

A kind of photography composition evaluation method, comprising: extract the characteristic information of image in the frame of composition frame, the composition frame packet Contain photographic subjects；The composition frame is evaluated based on the characteristic information, obtains the first assessment of the composition frame； The composition frame is evaluated based on the composition evaluation model of deep learning method, obtains commenting for the second time for the composition frame Valence；According to the first assessment and the second assessment, the overall merit of the composition frame is obtained.

It is further preferred that described carry out evaluating it based on the composition evaluation model of deep learning method to the composition frame Before include: the label for obtaining several images for being added to composition frame and each composition frame, obtain pattern data collection；With described The further feature of pattern data collection training deep learning network, the deep learning network is evaluated for composition；When the depth When learning network restrains, composition evaluation model is obtained.

It is further preferred that the further feature of the deep learning network is evaluated for composition further include: described The further feature of deep learning network and at least one shallow-layer feature are combined to be evaluated for composition.

It is further preferred that described be based on the characteristic information to carry out evaluation to the composition frame including: based on the spy The random forest disaggregated model of reference breath evaluates the composition frame.

It is further preferred that the characteristic information of image includes: according in photographic subjects in the frame for extracting composition frame The range information in four crosspoints of nine grids, obtains composition of geometry characterization factor in the heart and composition frame；And/or when shooting mesh When being designated as one, the vision equilibrium characteristic factor is obtained according to the facial orientation of the photographic subjects；When photographic subjects are more people, The vision equilibrium characteristic factor is obtained according to personage's center of gravity of the photographic subjects；And/or when photographic subjects are personage, according to Integrity degree of the human body key node of the photographic subjects in composition frame, obtains the integrity feature factor of human body key point； And/or when photographic subjects are personage, according to the expression of the photographic subjects, obtain the expressive features factor；And/or work as shooting When target is personage, according to the statistics with histogram of the background pixel of composition frame, background color terseness characterization factor is obtained.

The present invention also provides a kind of photography composition evaluation methods, comprising: the characteristic information of image in the frame of composition frame is extracted, The composition frame includes photographic subjects；The composition frame is evaluated based on the characteristic information, obtains the composition frame The first assessment；Judge whether the first assessment meets preset condition；When the first assessment is unsatisfactory for default item When part, the overall merit of the composition frame is obtained according to the first assessment；When the first assessment meets preset condition When, the composition frame is evaluated using the composition evaluation model based on deep learning method, obtains the of the composition frame Second evaluation；And according to the first assessment and the second assessment, obtain the overall merit of the composition frame.

The present invention also provides a kind of automatic photography methods, comprising: generation includes the composition frame of photographic subjects；According to aforementioned Any one of described in photography composition evaluation method the composition frame is evaluated, obtain overall merit；According to the synthesis Evaluation controls the output of the composition frame；When photographic unit receives the composition frame, photograph according to the composition frame.

The present invention also provides a kind of photography composition evaluating apparatus, comprising: characteristic extracting module, for extracting the frame of composition frame The characteristic information of interior image, the composition frame includes photographic subjects；First evaluation module, for being based on the characteristic information pair The composition frame is evaluated, and the first assessment of the composition frame is obtained；Second evaluation module, for being based on deep learning side The composition evaluation model of method evaluates the composition frame, obtains the second assessment of the composition frame；Overall merit module, For obtaining the overall merit of the composition frame according to the first assessment and the second assessment.

It is further preferred that further include: model construction module, for obtain several images for being added to composition frame and The label of each composition frame, obtains pattern data collection；And with the pattern data collection training deep learning network, the depth The further feature for spending learning network is evaluated for composition；When the deep learning network convergence, composition evaluation model is obtained.

It is further preferred that the model construction module, is further used for the pattern data collection training deep learning Network, the further feature of the deep learning network and at least one shallow-layer feature are combined to be evaluated for composition.

It is further preferred that first evaluation module, is further used for the random forest based on the characteristic information point Class model evaluates the composition frame.

It is further preferred that the characteristic extracting module, is further used for according in the center of photographic subjects and composition frame The range information in four crosspoints of nine grids, obtains composition of geometry characterization factor；And/or when photographic subjects is one, root The vision equilibrium characteristic factor is obtained according to the facial orientation of the photographic subjects；When photographic subjects are more people, according to the shooting Personage's center of gravity of target obtains the vision equilibrium characteristic factor；And/or when photographic subjects are personage, according to the photographic subjects Integrity degree of the human body key node in composition frame, obtain the integrity feature factor of human body key point；And/or work as shooting When target is personage, according to the expression of the photographic subjects, the expressive features factor is obtained；And/or when photographic subjects are personage When, according to the statistics with histogram of the background pixel of composition frame, obtain background color terseness characterization factor.

The present invention also provides a kind of photography composition evaluating apparatus, including characteristic extracting module, for extracting the frame of composition frame The characteristic information of interior image, the composition frame includes photographic subjects；First evaluation module, for being based on the characteristic information pair The composition frame is evaluated, and the first assessment of the composition frame is obtained；Whether judgment module judges the first assessment Meet preset condition；Overall merit module, for when the first assessment is unsatisfactory for preset condition, according to the first time Evaluation obtains the overall merit of the composition frame；Second evaluation module, for when the first assessment meets preset condition, The composition frame is evaluated using the composition evaluation model based on deep learning method, obtains second of the composition frame Evaluation；The overall merit module is further used for obtaining the structure according to the first assessment and the second assessment The overall merit of picture frame.

The present invention also provides a kind of automatic photographing devices, comprising: composition frame generation module includes shooting mesh for generating Target composition frame；Photography composition evaluating apparatus described in any one of aforementioned obtains comprehensive for evaluating the composition frame Close evaluation；Composition frame output module, for controlling the output of the composition frame according to the overall merit；Photographic unit is used for When receiving the composition frame, photograph according to the composition frame.

The present invention also provides a kind of electronic equipment, comprising: memory, for storing computer program；Processor, for transporting The row computer program, realize it is any one of aforementioned described in photography composition evaluation method.

The present invention also provides a kind of computer readable storage mediums, are stored thereon with computer program, the computer journey Realized when sequence is executed by processor it is any one of aforementioned described in photography composition evaluation method.

The composition evaluation method that there is provided through the invention, method for imaging, device, electronic equipment, storage medium, can bring Below the utility model has the advantages that

1, photography composition evaluation method provided by the invention, not only from local message or bottom visual information but also from entirety Information or profound information evaluate the quality of composition, traditional characteristic information have both been considered, it is contemplated that the sum of picture Humorous property, the integrality of patterned matter, evaluation is more comprehensively.

2, automatic photography method provided by the invention promotes the promotion of patterning quality by the promotion that composition is evaluated, thus It avoids exporting low-quality photo, and then low quality photo is avoided to occupy memory source, increase loss etc..

3, the present invention can substitute manpower and realize automatic shooting, avoid Human Physiology tired and subjective factor is to the shadow of shooting It rings.

Detailed description of the invention

Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, to a kind of composition evaluation method, takes the photograph Image method, device, electronic equipment, storage medium above-mentioned characteristic, technical characteristic, advantage and its implementation give furtherly It is bright.

Fig. 1 is a kind of flow chart of one embodiment of photography composition evaluation method of the invention；

Fig. 2 is a kind of flow chart of another embodiment of photography composition evaluation method of the invention；

Fig. 3 is a kind of flow chart of another embodiment of photography composition evaluation method of the invention；

Fig. 4 is a kind of flow chart of one embodiment of automatic photography method of the invention；

Fig. 5 is a kind of structural schematic diagram of one embodiment of photography composition evaluating apparatus of the invention；

Fig. 6 is a kind of structural schematic diagram of another embodiment of photography composition evaluating apparatus of the invention；

Fig. 7 is a kind of structural schematic diagram of another embodiment of photography composition evaluating apparatus of the invention；

Fig. 8 is the structural schematic diagram of one embodiment of a kind of electronic equipment of the invention；

Fig. 9 is an example of visual balance in a kind of photography composition evaluation method of the invention；

Figure 10 is a kind of structural schematic diagram of one embodiment of automatic photographing device of the invention；

Figure 11 is a kind of schematic perspective view of one embodiment of automatic photographing device of the invention.

Drawing reference numeral explanation:

100. photography composition evaluating apparatus, 110. characteristic extracting modules, 120. first evaluation modules, 130. second evaluation moulds Block, 140. overall merit modules, 150. model construction modules, 160. judgment modules, 200. composition frame generation modules, 300. compositions Frame output module, 500. automatic photographing devices, 502. photographic units, 400. electronic equipments, 410. memories, 420. processors, 430. computer programs, 440. buses.

Specific embodiment

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, Detailed description of the invention will be compareed below A specific embodiment of the invention.

It is understood that as needed, photographic subjects can be children or adult, be also possible to it is one or more its He is personage, can also be other animals, plant, landscape or building etc., can also be any combination in aforementioned object." composition Frame " refers to the closure frame with certain shapes and size, interior for limiting shape, the size of photographic picture, to realize Selection to the relative position of photographic subjects.In general, " composition frame " is rectangle frame.More specifically, " composition frame " is left and right Width is greater than the rectangle frame of upper-lower height.As needed, when such as camera is rotated by 90 ° realization and erects bat, above and below " composition frame " Height is then greater than left and right width.Correspondingly, when composition frame is selected, and when the images such as output photo, the shape of composition frame Corresponding to picture.

In addition, unless otherwise instructed, " upper and lower " that occurs in the present invention, " left and right ", " forward and backward ", " one, another " etc. It is relative concept.In addition, the term " first " occurred in the present invention, " second " etc. are only used for description conveniently, and it is not understood to Indication or suggestion relative importance clearly limits sequencing.

In one embodiment of the invention, as shown in Figure 1, a kind of photography composition evaluation method, comprising:

Step S200 extracts the characteristic information of image in the frame of composition frame, and the composition frame includes photographic subjects.

Step S300 is based on the characteristic information and evaluates the composition frame, and the first time for obtaining the composition frame comments Valence.

Specifically, as needed, photographic subjects can be adult or children, it can be other one or more personages.It adopts It whether can be identified in composition frame with known face recognition technology comprising photographic subjects.It but include the composition frame of photographic subjects It is not necessarily a good composition, in order to avoid generating low quality photo, output as far as possible meets the photo that human aesthetic requires, The composition to composition frame is needed to evaluate.

Feature information extraction is carried out to image in the frame of composition frame first.Characteristic information is usually to be passed through according to traditional photography It tests, is mostly extracted from the bottom visual information of composition, and can be by limited or simple mathematical knowledge expression, for example, shooting When target is personage, position feature of the photographic subjects in composition frame is (for example, the center and each side of composition frame that pass through photographic subjects Range information reflection), size characteristic of the photographic subjects in composition frame.Position based on photographic subjects in composition frame is special Sign, it can be determined that photographic subjects are the edges in center or composition frame, if being in the edge of composition frame, structure Scheme undesirable；If opposite center, composition meet the aesthetic habit of people in picture；Based on photographic subjects in composition Size characteristic in frame, it can be determined that whether photographic subjects size is too small in composition frame, if size is too small, composition is paid no attention to Think.

Features above information is one or more mathematical expressions of parts of traditional photography experience, and this mathematical expression needs people Work is pre-defined, and machine extracts corresponding characteristic information value according to the mathematical expression predetermined from specific composition frame, The composition of composition frame is evaluated further according to these characteristic information values, obtains the first assessment.

Step S400 evaluates the composition frame based on the composition evaluation model of deep learning method, obtains the structure The second assessment of picture frame.

Specifically, manual features extraction comparison is laborious, and the feature that can manually extract is also than relatively limited, mostly shallow-layer Feature has some features (such as further feature) that can not also express by limited mathematical knowledge, for example image is presented Artistic conception, concordance of picture etc..

Deep learning is a kind of machine learning algorithm, and benefit is not need manually to extract feature, is learnt by supervised Feature can be automatically extracted, shallow-layer feature can not only be extracted, moreover it is possible to extract further feature, pass through complicated mathematical knowledge table Up to out.Deep learning network is a kind of neural network with more hidden layers, is with classical convolutional neural networks Alexnet Example has 1 input layer, 6 hidden layers, 1 output layer, and totally 8 layers, every layer has thousands of a neurons, and every layer of output is the one of data Kind feature, and feature hierarchy is improved with the increase of processing level, for example, first layer is the edge feature of image, the second layer is The color characteristic of image, it is the concordance feature of image that third layer, which is the 8th layer of brightness ... ... of image, network it is former Layer reflects the local feature of image, is called shallow-layer feature, such as the features such as edge, color, brightness of image, network it is last One layer, i.e. the top layer global feature that reflects image, are called further feature, such as the concordance feature of image.

When being trained to deep learning network, only further feature can be evaluated applied to composition, it can also be by deep layer Feature and shallow-layer feature combined application are evaluated in composition.When deep learning network convergence, composition evaluation model is obtained.Use this Composition evaluation model evaluates composition frame, obtains the second assessment of the composition frame.

Step S500 obtains the overall merit of the composition frame according to the first assessment and the second assessment.

Specifically, the composition based on deep learning evaluates (i.e. the second assessment) cannot replace the first assessment completely, i.e., Composition evaluation model is set to consider further feature and shallow-layer feature, also can be problematic, the reason is that: one, deep learning learns out Feature lack interpretation, it is why good, it is why bad, can not be explicitly stated；Two, the spy that deep learning learns out Sign depends greatly on data set, and the distribution of data set and abundant degree will limit its effect；Exactly because above-mentioned two There is uncontrollable situation, i.e. depth in reason, the generalization ability that will lead to the composition evaluation model based on deep learning method Algorithm is practised to lack in common sense.

The first assessment is evaluated from the feature of Manual definition.The feature of Manual definition, i.e. traditional characteristic, mostly shallowly Layer feature, interpretation be it is guaranteed, generalization ability is also more secure than deep learning, all for clearly good feature Bad feature is specified, it is more suitable for going description with traditional characteristic, and can also largely reduce calculation amount.But it passes System feature also has the problem of itself, is manually also difficult to define further feature, such as the concordance of whole picture etc. at present, and this A little high-level things, exactly deep learning neural network are relatively good at.

The second assessment considers the further feature that machine deep learning arrives, and the two, which combines, just can more fully be commented Valence had both considered interpretation feature, it is also considered that not describable further feature (i.e. high-level semantics features), it in this way could be complete The quality for removing evaluation composition in face, existing details also have entirety, and generalization ability is more secure.

Overall merit is obtained according to the first assessment and the second assessment.It, just can be further only when overall merit is high Composition frame is exported, is photographed；The promotion of patterning quality is promoted by the promotion that composition is evaluated, to avoid output low quality Photo, and then avoid low quality photo occupy memory source, increase loss etc..

In another embodiment of the present invention, as shown in Fig. 2, a kind of photography composition evaluation method, comprising:

Step S100 obtains the label of several images for being added to composition frame and each composition frame, obtains pattern data Collection；

The further feature of the step S110 pattern data collection training deep learning network, the deep learning network is used It is evaluated in composition；

Step S130 obtains composition evaluation model when the deep learning network convergence.

Specifically, using large-scale dataset, such as millions of images, composition frame is added to every image, and stamp Label obtains pattern data collection.Since composition evaluation is subjective, there are apparent individual differences, therefore, for composition frame The mark of interior image is generally labeled same piece image using more people, takes proprietary average mark as image later Final label.

Deep learning network structure using classical convolutional neural networks, such as Alexnet, can also using ZFNet or GoogLeNet etc., herein without limitation, wherein by the last layer feature of network, (i.e. top-level feature, it reflects image Further feature) it is evaluated for composition.

With obtained pattern data collection training deep learning network, study is allowed to the mapping between image and label.When (for example, convergence criterion are as follows: pattern data concentrates the evaluation of each composition frame to export and the structure when deep learning network convergence The label of picture frame is consistent；Or pattern data concentrates evaluation output and the ratio of the consistent composition frame of label to reach preset number), it obtains To composition evaluation model.

Preferably, the further feature of network and at least one layer of shallow-layer feature are combined into use in deep learning network structure It is evaluated in composition.Further feature, that is, top-level feature of network, shallow-layer are characterized in for opposite further feature, can be any non-top Layer feature, such as first layer feature or second layer feature etc..Shallow-layer feature includes edge, color, brightness of image etc., deep layer Characterizations integrality of picture material, such as the concordance of picture etc., are fused together shallow-layer feature and further feature Can more complete picture engraving aesthetic feeling, to keep the evaluation of the composition evaluation model obtained based on deep learning method more acurrate.

The characteristic information includes that composition of geometry characterization factor, and/or the vision equilibrium characteristic factor, and/or human body are crucial The integrity feature factor, and/or the expressive features factor, and/or background color terseness characterization factor of point.According to different type Photography, such as personage shine, can choose composition of geometry characterization factor, the vision equilibrium characteristic factor, human body key point it is complete Property characterization factor, the expressive features factor and background color terseness characterization factor；Pure natural land shines, and can choose geometry structure Figure characterization factor.

Nine grids composition method is one of common composition rule of taking pictures, and is meant that, by composition frame with two vertical lines, two After horizontal line is respectively divided into three parts, personage is placed on the crosspoint of four cut-off rules, this four points are exactly picture emphasis. This composition method can make the picture entirety sense of equilibrium preferable, can also emphasize out to shoot the relativity between main body and secondary main body. Based on the rule, according to the range information computational geometry structure in four crosspoints of nine grids in the center of photographic subjects and composition frame Figure characterization factor, to measure whether composition meets nine grids composition rule.

Composition of geometry characterization factor can be further calculated using the following equation:

Wherein, f1 is composition of geometry characterization factor, (C_Rx,C_Ry) be photographic subjects centre coordinate, (P_ix,P_iy) it is target The coordinate in four crosspoints of nine grids in composition frame, X and Y are the width and height of target pattern frame.

The vision equilibrium characteristic factor is mainly used for measuring the visual balance state of composition itself.When photographic subjects are for one When, visual balance is mainly judged according to the facial orientation of photographic subjects；As shown in figure 9, facial orientation is to the right, the body of people exists Position inside composition frame is left side, so being visually balance；If the body of people is on right side, consistent with facial orientation, It is visually then unbalanced.

When photographic subjects are more people, in order to measure the reasonability of more people's space layouts, by human body detecting method, calculate Everyone position of centre of gravity in composition；The position of centre of gravity of more people is obtained according to everyone position of centre of gravity, according to more people's Location information of the position of centre of gravity in composition frame is to calculate the visual balance state of composition.If the position of centre of gravity of more people is in structure The marginal position of picture frame is then non-equilibrium state；It is flat if the position of centre of gravity of more people is located at the central area of composition frame Weighing apparatus state.For example, 3 people are on the left side of composition frame, for 1 people on the right of composition frame, personage's center of gravity is visually uneven on the left side Weighing apparatus, this composition are bad.

The integrity feature factor of human body key point is primarily to butt occur, cutting hand, cut the feelings such as foot in punishment composition Condition.When photographic subjects are personage, according to integrity degree of the human body key node of photographic subjects in composition frame, human body pass is obtained The integrity feature factor of key point.Human body key point is extracted by image recognition, then judges human body key node in composition frame Integrated degree.

The expressive features factor is used to react the expression of photographic subjects.When photographic subjects are personage, pass through recognition of face skill Art determines that the expression of photographic subjects in composition frame, such as expression define three kinds of states: laughing at, is neutral, bad expression, to obtain table Feelings characterization factor.

Background color terseness characterization factor is mainly used for measuring the succinct degree of background color.When photographic subjects are personage When, according to the statistics with histogram of the background pixel of composition frame, obtain background color terseness characterization factor.According to background pixel Statistics with histogram when pixel class is bigger, then shows as background face as a result, count the pixel class greater than preset quantity Color is complicated, not enough succinctly.

For example, the pixel value in tri- channels background area RGB of composition frame is quantified respectively to 16 grades, so sharing 4096 Kind combination of pixels.The statistics with histogram for calculating pixel distribution, can obtain background color terseness characterization factor according to following formula F5:

Wherein, S=i | His (i) >=γ h_maxIndicating that quantity accounting is greater than the background pixel set of preset ratio, γ is Predetermined coefficient, h_maxFor the maximum statistics component of histogram, His (i) is the statistic of pixel i in histogram, | | S | | indicate collection Close the element number of S.Be in when γ takes 0.01, f5 (0,1.5%] when, then background color is succinct；Otherwise not simple for background color It is clean.

Step S310 evaluates the composition frame based on the random forest disaggregated model of the characteristic information, obtains institute State the first assessment of composition frame.

Specifically, random forest classification is also a kind of machine learning algorithm, it is that one kind includes multiple decision trees, is based on majority The classifier of voting mechanism, each decision tree are voted according to some features of input, and this feature needs Manual definition, selection Most classification results vote as last prediction result.For example, a random forest grader comprising M decision tree, New data is put into this M decision tree, every decision tree has a classification results, M classification results are obtained, to point Class result is counted, using the most classification results of poll as last prediction result.

Random forest sorting algorithm is applied in the composition evaluation of composition frame, in conjunction with the feature of Manual definition, by big The sample training of amount obtains the random forest disaggregated model evaluated for composition.The construction method of the model is as follows: obtaining several It is added to the image of composition frame and the label of composition frame, obtains pattern data collection；According to the feature of Manual definition to composition number Corresponding characteristic information is extracted according to each composition frame of collection；The characteristic information of each composition frame of pattern data collection and label is defeated Enter random forest disaggregated model to be trained, is allowed to study to the mapping between characteristic information and label；When the disaggregated model is received (for example, convergence criterion are as follows: the evaluation output for each composition frame is consistent with the label of the composition frame) when holding back, trained Good random forest disaggregated model.

By taking personage is shone as an example, composition of geometry characterization factor, the vision equilibrium characteristic factor, the integrality of human body key point are selected The characteristic informations such as characterization factor, the expressive features factor and background color terseness characterization factor, according to the people of features described above information Work definition carries out corresponding feature information extraction to a composition frame to be evaluated, extracted characteristic information is inputted preparatory The random forest disaggregated model of building obtains the evaluation output of the model, and evaluation output is commented as the first time of composition frame Valence.

The first assessment of the present embodiment is obtained according to the feature and random forest disaggregated model of Manual definition, random gloomy Standing forest class model is obtained by big data training, compared between the characteristic information defined according to artificial experience and composition evaluation Mapping relations, it can obtain more accurate mapping relations, to improve the accuracy of the first assessment, and then improve synthesis The accuracy of evaluation.

In another embodiment of the present invention, as shown in figure 3, a kind of photography composition evaluation method, comprising: with aforementioned reality It applies example to compare, something in common no longer repeats, the difference is that after step S310, comprising:

Step S320 judges whether the first assessment meets preset condition；

Step S330 obtains the structure when the first assessment is unsatisfactory for preset condition, according to the first assessment The overall merit of picture frame；

Step S410 is evaluated when the first assessment meets preset condition using the composition based on deep learning method Model evaluates the composition frame, obtains the second assessment of the composition frame；

Step S510 obtains the overall merit of the composition frame according to the first assessment and the second assessment.

Specifically, if the first assessment is unsatisfactory for preset condition, such as when being lower than some pre-determined threshold, illustrate based on biography The evaluation for feature of uniting is not high, and overall merit is not certainly high, so the second assessment need not be carried out, can reduce system in this way Operand reduces system loading.

In one embodiment of the invention, as shown in figure 4, a kind of automatic photography method, comprising:

Step S600 generation includes the composition frame of photographic subjects.

Specifically, as needed, photographic subjects can be adult or children.It correspondingly, can be according to based on Identification of Images Computer vision technique, generate include photographic subjects composition frame.At this point, with the image of photographic subjects, root in composition frame The images such as the corresponding photo of photographic subjects can be exported according to the composition frame.

Step S700 uses any one photography composition evaluation method in aforementioned to evaluate the composition frame, obtains comprehensive Close evaluation.

Step S800 controls the output of the composition frame according to the overall merit.

Step S900 photographs when photographic unit the receives composition frame according to the composition frame.

The present embodiment only when the overall merit of composition frame is high, just can further export the composition in automatic photography Frame executes photography.It when overall merit is low, needs to regenerate composition frame, adjusts composition.It can be evaluated and be kept away by composition in this way Exempt to export low-quality photo, and then low quality photo is avoided to occupy memory source, increase loss etc..

In order to improve automatic photography efficiency, multiple composition frames can be generated simultaneously, and photography is executed to each composition frame respectively Composition evaluation, obtains the overall merit of each composition frame.If there is the overall merit of multiple composition frames reaches minimum output requirement, The therefrom highest composition frame output of selection overall merit.If reaching minimum output requirement without composition frame, need to give birth to again At composition frame.

In one embodiment of the invention, as shown in figure 5, a kind of photography composition evaluating apparatus 100, comprising:

Characteristic extracting module 110, the characteristic information of image in the frame for extracting composition frame, the composition frame include to clap Take the photograph target.

First evaluation module 120 obtains the composition for evaluating based on the characteristic information the composition frame The first assessment of frame.

Specifically, as needed, photographic subjects can be adult or children.It can be known using known face recognition technology It whether include photographic subjects in other composition frame.But it include that the composition frames of photographic subjects is not necessarily a good composition, in order to It avoids generating low quality photo, output as far as possible meets the photo that human aesthetic requires, and the composition to composition frame is needed to comment Valence.

Second evaluation module 130 comments the composition frame for the composition evaluation model based on deep learning method Valence obtains the second assessment of the composition frame.

Overall merit module 140, for obtaining the composition frame according to the first assessment and the second assessment Overall merit.

In one embodiment of the invention, as shown in fig. 6, a kind of photography composition evaluating apparatus 100, comprising:

Model construction module 150, for obtaining the label of several images for being added to composition frame and each composition frame, Obtain pattern data collection；And with the pattern data collection training deep learning network, the deep layer of the deep learning network is special It takes over for use and is evaluated in composition；When the deep learning network convergence, composition evaluation model is obtained.

First evaluation module 120, for the random forest disaggregated model based on the characteristic information to the composition frame into Row evaluation, obtains the first assessment of the composition frame.

In another embodiment of the present invention, as shown in fig. 7, a kind of photography composition evaluating apparatus 100, comprising: with it is preceding Embodiment to be stated to compare, something in common no longer repeats, the difference is that, comprising:

Judgment module, judges whether the first assessment meets preset condition；

Overall merit module, for when the first assessment is unsatisfactory for preset condition, according to the first assessment Obtain the overall merit of the composition frame；

Second evaluation module, for when the first assessment meets preset condition, using being based on deep learning method Composition evaluation model the composition frame is evaluated, obtain the second assessment of the composition frame；

The overall merit module is further used for obtaining institute according to the first assessment and the second assessment State the overall merit of composition frame.

In one embodiment of the invention, as shown in figure 8, a kind of electronic equipment 400, including memory 410 and processing Device 420.The memory 410 is for storing computer program 430.The processor is realized such as when running the computer program The photography composition evaluation method of foregoing description.

As an example, it is realized according to the step S200 of foregoing description extremely when processor 420 executes computer program S500.Additionally, it is realized when processor 420 executes computer program each in the photography composition evaluating apparatus 100 of foregoing description Module, the function of unit.As another example, processor 420 realized when executing computer program characteristic extracting module 110, The function of first evaluation module 120, the second evaluation module 130 and overall merit module 140.

Optionally, according to specific needs of the invention are completed, the computer program can be divided into one or more Module/unit.Each module/unit can be the series of computation machine program instruction section that can complete specific function.The calculating Machine program instruction section is for describing implementation procedure of the computer program in photography composition evaluating apparatus 100.As an example, The computer program can be divided into modules/unit in virtual bench, for example characteristic extracting module, the first evaluation Module, the second evaluation module and overall merit module.

The processor is used for by executing the computer program to realize that photography composition is evaluated.As needed, institute Stating processor can be central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), dedicated collection At circuit (ASIC), field programmable gate array (FPGA), general processor or other logical devices etc..

The memory can be the internal storage unit and/or external storage that arbitrarily can be realized data, program storage Equipment.For example, the memory can be plug-in type hard disk, intelligent memory card (SMC), secure digital (SD) card or flash card Deng.The memory is used to store other programs and data of computer program, photography composition evaluating apparatus 100.

The electronic equipment 400 can be any computer equipment, for example desktop PC (desktop), portable Computer (laptop), palm PC (PDA) or server (server) etc..As needed, the electronic equipment 400 can be with Including input-output equipment, display equipment, network access equipment and bus 440 etc..The electronic equipment 400 can also be monolithic Machine, or it is integrated with the calculating equipment of central processing unit (CPU) and graphics processing unit (GPU).

It for realizing the unit of corresponding function, the division of module is it will be appreciated by persons skilled in the art that above-mentioned Said units, module are further divided or combined, i.e., according to application demand in convenient explanation, the purpose of narration Device/equipment internal structure is re-started into division, combination, with the function of the above-mentioned record of realization.In above-described embodiment Individual physical unit can be respectively adopted in each unit, module, two or more units, module can also be integrated In a physical unit.Each unit, module in above-described embodiment can be using the realizations of hardware and/or SFU software functional unit Corresponding function.Between multiple units, component, module in above-described embodiment can with direct-coupling, INDIRECT COUPLING or communication Connection can be realized by bus or interface；Coupling, connection between multiple units or device can be electrical, mechanical Or similar mode.Correspondingly, each unit in above-described embodiment, the specific name of module also only to facilitate narration and It distinguishes, and does not have to the protection scope of limitation the application.

In one embodiment of the invention, a kind of computer readable storage medium is stored thereon with computer program, institute The photography composition evaluation method recorded such as previous embodiment can be realized by stating when computer program is executed by processor.It that is to say, when Some or all of technical solution that the aforementioned embodiment of the present invention contributes to the prior art passes through computer software product When mode emerges from, aforementioned computer software product is stored in a computer readable storage medium.The computer can Reading storage medium can be any portable computers program code entity apparatus or equipment.For example, described computer-readable to deposit Storage media can be USB flash disk, mobile disk, magnetic disk, CD, computer storage, read-only memory, random access memory etc..

In one embodiment of the invention, as shown in Figure 10, Figure 11, a kind of automatic photographing device 500, comprising:

Composition frame generation module 200, for generate include photographic subjects composition frame.

Photography composition evaluating apparatus 100 described in aforementioned any embodiment obtains described for evaluating composition frame The overall merit of composition frame.

Composition frame output module 300, for controlling the output of the composition frame according to the overall merit.

Photographic unit 502, for being photographed according to the composition frame when receiving the composition frame.Described the ministry of photography Part 502 is used to optical imagery shooting being fixed on corresponding storage medium.Optionally, the photographic unit 502 can be number Code camera.

In order to promote the application performance of automatic photographing device 500, automatic photographing device 500 further includes pedestal 504 and optics Camera lens 252.Pedestal 504 is used to support the other structures of automatic shooting equipment 500, component etc..Pedestal 504 is generally cylindrical, Its circumferential direction is circumferencial direction.Optical lens 252 is set to the periphery wall of pedestal 504.Optical lens 252 is used for will be to be captured Target carry out optical imagery.The optical lens 252 is wide-angle lens, has the visual field of the camera lens 503 greater than photographic unit 502 Angle.I.e. the field angle of photographic unit 502 is determined by the field angle of its included camera lens 503.The camera lens 503 of photographic unit 502, also It is a kind of optical lens, is arranged on bracket component 506.In order to capture photographic subjects in spatial dimension as big as possible, three Optical lens 252 is intervally arranged along the circumferencial direction of pedestal 504.The field angle of three optical lens 252 is adjacent to each other or again It is folded, so as to obtain the photographic subjects in 360 ° of spaces of level.

Automatic photographing device 500 further includes imaging sensor 254.The quantity of imaging sensor 254 can be with optical lens 252 It is identical, and the assembling cooperation one by one that corresponds to each other.Optical picture of the imaging sensor 254 for that will enter in corresponding optical lens 252 As signal is converted to the picture signal of electronic data, and the picture signal of the electronic data is exported in order to subsequent place Reason.In Figure 11, imaging sensor 254 is arranged in pedestal 504, and explanation dotted line illustrates its substantially installation position for the ease of illustration It sets.

In order to facilitate that the activity to photographic unit 502 is being propped up to higher image quality, the setting of photographic unit 502 is obtained On frame component 506, and it can be rotatably set relative to bracket component 506.Specifically, photographic unit 502 around horizontal extension and hangs down Directly in 602 pitch rotation of the pivot center of shooting direction, the activity of vertical up and down direction can be realized.Photographic unit 502 is opposite It can be rotatably set in the pivot center 603 of bracket component 506 extended along shooting direction.When photographic unit 502 rotates 90 ° When, " perpendicular to clap " may be implemented.Bracket component 506 can be rotatably set relative to the axis 601 that vertical direction extends, so as to Photographic unit 502 is driven to realize the rotation within the scope of 360 ° of horizontal view angle.Wherein, appoint in three pivot centers 601,602,603 Meaning two perpendicular to one another, to be configured to solid space coordinate system.

Photography composition evaluating apparatus 100, composition frame generation module 200, composition frame output module 300 are arranged in pedestal 504 It is interior.Aforementioned electronic devices 400 also can be set in pedestal 504.

The image of higher shooting quality is obtained by aforementioned photography composition evaluation method, automatic shooting method, device Substantially steps are as follows:

Photographic subjects are captured on a large scale by optical lens 252, after finding photographic subjects, obtain the position letter of photographic subjects Breath adjusts photographic unit 502, it is made to be directed at photographic subjects, has one to include shooting mesh in the visual field of photographic unit 502 at this time Target image, in order to further enhance the picture quality shot to photographic subjects, composition frame generation module 200 is to current figure Photographic subjects setting composition frame (preview pane in similar digital camera) as in, obtains initial pictures (the i.e. structure of photographic subjects Image in picture frame).There are many setting means of composition frame, such as centered on photographic subjects, takes the external envelope of photographic subjects, External envelope is arranged by the photographic scale that camera is shot again, obtains a kind of composition frame.Photography composition evaluating apparatus 100 is to this Composition frame is patterned evaluation, obtains the overall merit of composition frame, that is, determines whether the initial pictures meet photography composition requirement； When the overall merit meets photography composition requirement, triggering composition frame output module 300 exports the composition frame, photographic unit 502 When receiving the composition frame, then execute movement of taking pictures.When the overall merit is unsatisfactory for photography composition requirement, repetition pair can be passed through The adjusting of photographic unit 502, the generation and evaluation of composition frame until output meets the composition frame of photography composition requirement, then execute It takes pictures.It takes pictures due to only just will do it when initial pictures meet photography composition requirement, so the figure of better quality can be obtained Picture.

It can one of in the following manner or a variety of adjusting photographic units 502: the adjusting of photographic unit enlargement ratio (zoom), photographic unit with respect to the horizontal plane on pitch regulation and photographic unit in moving in parallel on the water surface and/or phase The spatial position that axis around vertical direction horizontally rotates is adjusted, that is to say and carry out PTZ (Pan/Tilt/Zoom) operation.

In order to improve automatic photography efficiency, multiple composition frames can be generated simultaneously, and photography is executed to each composition frame respectively Composition evaluation, obtains the overall merit of each composition frame.If there is the overall merit of multiple composition frames reaches minimum output requirement, The therefrom highest composition frame output of selection overall merit.If reaching minimum output requirement without composition frame, needs to adjust and take the photograph Shadow component regenerates composition frame.

It should be noted that above-described embodiment can be freely combined as needed.The above is only of the invention preferred Embodiment, it is noted that for those skilled in the art, in the premise for not departing from the principle of the invention Under, several improvements and modifications can also be made, these modifications and embellishments should also be considered as the scope of protection of the present invention.

Claims

1. a kind of photography composition evaluation method characterized by comprising

The characteristic information of image in the frame of composition frame is extracted, the composition frame includes photographic subjects；

The composition frame is evaluated based on the characteristic information, obtains the first assessment of the composition frame；

The composition frame is evaluated based on the composition evaluation model of deep learning method, obtains second of the composition frame Evaluation；

According to the first assessment and the second assessment, the overall merit of the composition frame is obtained.

2. photography composition evaluation method according to claim 1, which is characterized in that the structure based on deep learning method Figure evaluation model includes: before being evaluated the composition frame

The label for obtaining several images for being added to composition frame and each composition frame, obtains pattern data collection；

It is evaluated with the further feature of the pattern data collection training deep learning network, the deep learning network for composition；

When the deep learning network convergence, composition evaluation model is obtained.

3. photography composition evaluation method according to claim 2, which is characterized in that the deep learning network Further feature is evaluated for composition further include:

The further feature of the deep learning network and at least one shallow-layer feature are combined to be evaluated for composition.

4. photography composition evaluation method according to claim 1, which is characterized in that described to be based on the characteristic information to institute It states composition frame and evaluate and include:

The composition frame is evaluated based on the random forest disaggregated model of the characteristic information.

5. photography composition evaluation method according to claim 1, which is characterized in that image in the frame for extracting composition frame Characteristic information include:

According to the range information in four crosspoints of nine grids in the center of photographic subjects and composition frame, composition of geometry feature is obtained The factor；And/or

When photographic subjects is one, the vision equilibrium characteristic factor is obtained according to the facial orientation of the photographic subjects；And when When photographic subjects are more people, the vision equilibrium characteristic factor is obtained according to personage's center of gravity of the photographic subjects；And/or

When photographic subjects are personage, according to integrity degree of the human body key node of the photographic subjects in composition frame, obtain The integrity feature factor of human body key point；And/or

When photographic subjects are personage, according to the expression of the photographic subjects, the expressive features factor is obtained；And/or

When photographic subjects are personage, according to the statistics with histogram of the background pixel of composition frame, it is special to obtain background color terseness Levy the factor.

6. a kind of photography composition evaluation method characterized by comprising

Judge whether the first assessment meets preset condition；

When the first assessment is unsatisfactory for preset condition, commented according to the synthesis that the first assessment obtains the composition frame Valence；

When the first assessment meets preset condition, using the composition evaluation model based on deep learning method to the structure Picture frame is evaluated, and the second assessment of the composition frame is obtained；And it described comments according to the first assessment and for the second time Valence obtains the overall merit of the composition frame.

7. a kind of automatic photography method characterized by comprising

Generation includes the composition frame of photographic subjects；

Photography composition evaluation method according to claim 1-6 evaluates the composition frame, is integrated Evaluation；

The output of the composition frame is controlled according to the overall merit；

When photographic unit receives the composition frame, photograph according to the composition frame.

8. a kind of photography composition evaluating apparatus characterized by comprising

Characteristic extracting module, the characteristic information of image in the frame for extracting composition frame, the composition frame includes photographic subjects；

First evaluation module obtains the of the composition frame for evaluating the composition frame based on the characteristic information Primary evaluation；

Second evaluation module is evaluated the composition frame for the composition evaluation model based on deep learning method, is obtained The second assessment of the composition frame；

Overall merit module, for obtaining the synthesis of the composition frame according to the first assessment and the second assessment Evaluation.

9. photography composition evaluating apparatus according to claim 8, which is characterized in that further include:

Model construction module obtains composition for obtaining the label of several images for being added to composition frame and each composition frame Data set；And with the pattern data collection training deep learning network, the further feature of the deep learning network is used for structure Figure evaluation；When the deep learning network convergence, composition evaluation model is obtained.

10. photography composition evaluating apparatus according to claim 9, it is characterised in that:

The model construction module is further used for the pattern data collection training deep learning network, the deep learning The further feature of network and at least one shallow-layer feature are combined to be evaluated for composition.

11. photography composition evaluating apparatus according to claim 8, it is characterised in that:

First evaluation module is further used for the random forest disaggregated model based on the characteristic information to the composition frame It is evaluated.

12. photography composition evaluating apparatus according to claim 8, it is characterised in that:

The characteristic extracting module is further used for four crosspoints according to nine grids in the center of photographic subjects and composition frame Range information, obtain composition of geometry characterization factor；And/or

13. a kind of photography composition evaluating apparatus characterized by comprising

Judgment module, judges whether the first assessment meets preset condition；

Overall merit module, for being obtained according to the first assessment when the first assessment is unsatisfactory for preset condition The overall merit of the composition frame；

Second evaluation module, for when the first assessment meets preset condition, using the structure based on deep learning method Figure evaluation model evaluates the composition frame, obtains the second assessment of the composition frame；

The overall merit module is further used for obtaining the structure according to the first assessment and the second assessment The overall merit of picture frame.

14. a kind of automatic photographing device characterized by comprising

Composition frame generation module, for generate include photographic subjects composition frame；

The described in any item photography composition evaluating apparatus of claim 8-13 obtain comprehensive for evaluating the composition frame Close evaluation；

Composition frame output module, for controlling the output of the composition frame according to the overall merit；

Photographic unit, for being photographed according to the composition frame when receiving the composition frame.

15. a kind of electronic equipment characterized by comprising

Memory, for storing computer program；

Processor realizes photography composition according to any one of claim 1 to 6 when for running the computer program Evaluation method.

16. a kind of computer readable storage medium, is stored thereon with computer program, it is characterised in that:

The computer program realizes that photography composition according to any one of claim 1 to 6 is commented when being executed by processor Valence method.