CN112256907B - Travel attack editing method, system, equipment and storage medium based on photo library - Google Patents
Travel attack editing method, system, equipment and storage medium based on photo library Download PDFInfo
- Publication number
- CN112256907B CN112256907B CN202011287071.6A CN202011287071A CN112256907B CN 112256907 B CN112256907 B CN 112256907B CN 202011287071 A CN202011287071 A CN 202011287071A CN 112256907 B CN112256907 B CN 112256907B
- Authority
- CN
- China
- Prior art keywords
- pictures
- picture
- keyword
- travel
- attack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013441 quality evaluation Methods 0.000 claims abstract description 17
- 238000011156 evaluation Methods 0.000 claims abstract description 15
- 238000012163 sequencing technique Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 15
- 238000013508 migration Methods 0.000 claims description 12
- 230000005012 migration Effects 0.000 claims description 12
- 238000005065 mining Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 5
- 238000010606 normalization Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Library & Information Science (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a travel attack editing method, a system, equipment and a storage medium based on a photo library, wherein the method comprises the following steps: obtaining at least one keyword with highest association degree from a preset travel keyword set based on the text of the travel attack, and forming a keyword combination; searching pictures meeting the keyword combination to form a photo library; sequencing the quality of pictures in a target photo library through a trained picture quality network; and adding the at least one picture ranked at the front to a preset position in the text of the travel attack. The invention can realize the quality evaluation of the pictures, and carry out the overall evaluation on the pictures in multiple aspects from the angle of the user, so that the most attractive pictures are ranked first, and the user experience is enhanced.
Description
Technical Field
The invention relates to the field of picture evaluation, in particular to a travel attack editing method, a system, equipment and a storage medium based on a photo library.
Background
In the current information explosion environment, the time for people to read information is shortened, the users are required to be attracted, the most attractive part needs to be displayed at the first time, the pictures are the most information-knowing media contacted by the users, but the pictures contain a plurality of information, the effective and forward information is difficult to convey at the first time, and the pictures for travel attack contain official pictures, user pictures and the like, so that the sources are various, the quality is uneven, and the quantity is huge. And the user also needs to search related photos by oneself to match pictures when writing travel strategies, so that the efficiency is reduced, and the user experience is poor.
Accordingly, the present invention provides a travel attack editing method, system, device and storage medium based on photo library.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a travel attack editing method, a system, equipment and a storage medium based on a photo library, which overcome the difficulties in the prior art, can realize the quality evaluation of pictures, carry out multi-aspect overall evaluation on the pictures from the angle of users, quickly and accurately acquire the picture quality, lead the most attractive pictures to be at first, lead users to acquire the best looking pictures at the first time when acquiring information, and enhance the user experience.
The embodiment of the invention provides a travel attack editing method based on a photo library, which comprises the following steps of:
s110, obtaining at least one keyword with highest association degree from a preset travel keyword set based on a travel attack text to form a keyword combination;
s120, searching pictures meeting the keyword combination to form a photo library;
s130, sorting the quality of pictures in a target photo library through a trained picture quality network; and
and S140, adding at least one picture which is ranked at the front into a preset position in the text of the travel attack.
Preferably, in the step S110, the travel keyword set includes a location keyword subset and a topic keyword subset, at least one location keyword is obtained in the location keyword subset and at least one topic keyword is obtained in the topic keyword subset based on the text of the travel attack;
in the step S120, a photo library is searched for the photos satisfying both the location keyword and the subject keyword.
Preferably, in the step S120, the condition that the picture satisfies the location keyword is:
shooting location information of the picture hits the location keyword; or alternatively
And the GPS positioning information of the shooting location of the picture is in the GPS positioning range corresponding to the location keyword.
Preferably, in the step S120, the condition that the picture satisfies the topic keyword is:
the pictures are respectively labeled through a trained picture network, and each picture obtains at least one label;
at least one of the tags of the picture hits the subject keyword.
Preferably, the picture tag network performs training of picture tag classification according to the preset travel keyword set, and outputs at least one travel keyword in the preset travel keyword set to the picture.
Preferably, the pictures meeting the place keywords form a process photo library, and the picture meeting the subject keywords is searched in the process photo library to form a photo library.
Preferably, in step S120, the range of the searched photo is at least one cloud photo library corresponding to the current user account or a photo library stored in the mobile terminal currently used by the user.
Preferably, in the step S110, the text of the travel attack is a paragraph of natural text currently edited by the user;
in the step S140, the selected picture is added before the paragraph of the natural text currently edited by the user.
Preferably, the training of the picture quality network in step S130 includes the following steps:
s131, clustering high-frequency picture types by searching on-line pictures of different types and styles;
s132, establishing an image information mining model based on multi-scale features, and performing multi-dimensional evaluation on the searched attack picture information, wherein the dimensions comprise definition, layout, color and saturation; and
s133, converting the result of the mining model into decimal places and reserving fixed digits, and sorting according to the final quality score.
Preferably, the step S132 includes:
based on preset keywords, respectively finding out picture information under different keywords, and generating a quality evaluation data set, wherein evaluation standards comprise definition, saturation, composition, color and the like;
preprocessing the pictures in the data set, uniformly adjusting the sizes of all the images to be fixed in length and width, and normalizing the image pixels; carrying out batch processing on the adjusted fixed-size pictures and inputting the pictures into a subsequent network; for fractional segments with less partial pictures, horizontally turning the pictures to form new pictures, and putting the new pictures into a data set;
according to the picture characteristics, a corresponding deep convolutional neural network is built, and a multi-scale-based multi-layer complex network (MultiNet) is designed according to the picture characteristics; the whole network consists of two parts, wherein the first part is a backbone network, the backbone network uses a dense convolution network (dense 121 network) and consists of 4 blocks (blocks), each block comprises a plurality of convolution layers, an activation layer and the like, and the total number of the convolution layers is 121, and the characteristic sizes of all the layers are the same, so that each layer can be connected with the characteristics of all the previous layers on a channel, the characteristic reuse is realized, and the efficiency is improved; performing model compression on the last layer of each block, reducing the size of the features, then averaging the features to 1*1, performing the operation on four blocks, and finally combining the four features on a channel to realize multi-scale; the second part is a full-connection network, the extracted characteristics are independently trained, the full-connection network comprises 3 full-connection layers, a plurality of activation layers, a pooling layer and the like, a prediction layer is finally output, 8 categories are output, then probability combination is carried out on the 8 categories, and a final score is output;
wherein a cross entropy loss function is used for class prediction, the formula is defined as follows, where p t For the category prediction probability, y is the image category label
loss＝∑y log(p t )
Dividing an acquired data set into a training set, a verification set and a test set, performing migration learning by using a model obtained based on large-scale scene data set training, performing operations such as zooming, normalization and the like on pictures in the migration learning process to enhance the robustness of the model, performing multi-scale feature extraction by using a network after migration learning, training the extracted features, and iterating the model until the test effect of the model on the verification set reaches the optimal;
forward prediction is carried out on the on-line attack pictures by using the trained model, and the prediction probability of the image on each fractional segment is output;
the prediction probability is combined with the score to obtain a final score, wherein i is the score, and p (i) is the prediction probability of the score;
and outputting the score as a result of the mining model.
The embodiment of the invention also provides a picture quality sorting system based on the photo library, which is used for realizing the travel attack editing method based on the photo library, and comprises the following steps:
the keyword obtaining module is used for obtaining at least one keyword with highest association degree from a preset travel keyword set based on the text of the travel attack, so as to form a keyword combination;
the photo searching module searches the pictures meeting the keyword combination to form a photo library;
the quality sorting module sorts the quality of the pictures in the target photo library through a trained picture quality network; and
and the picture inserting module is used for adding at least one picture which is ranked at the front into a preset position in the text of the travel attack.
The embodiment of the invention also provides a picture quality sorting device based on the photo library, which comprises:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the photo library-based travel attack editing method described above via execution of the executable instructions.
Embodiments of the present invention also provide a computer-readable storage medium storing a program that, when executed, implements the steps of the photo library-based travel attack editing method described above.
The invention aims to provide a travel attack editing method, a system, equipment and a storage medium based on a photo library, which can realize quality evaluation of pictures, carry out multi-aspect overall evaluation on the pictures from the angle of a user, quickly and accurately acquire the picture quality, lead the most attractive pictures to be at first, lead the user to acquire the best-looking pictures in the first time when acquiring information, and enhance the user experience.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings.
FIG. 1 is a flow chart of a travel attack editing method based on photo libraries of the present invention.
Fig. 2 to 4 are process diagrams of a travel attack editing method based on photo libraries for implementing the present invention.
Fig. 5 is a schematic block diagram of a photo library-based picture quality ordering system of the present invention.
Fig. 6 is a schematic structural diagram of a picture quality sorting apparatus based on a photo library of the present invention.
Fig. 7 is a schematic structural view of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus a repetitive description thereof will be omitted.
FIG. 1 is a flow chart of a travel attack editing method based on photo libraries of the present invention. As shown in fig. 1, an embodiment of the present invention provides a travel attack editing method based on a photo library, including the following steps:
s110, obtaining at least one keyword with highest association degree from a preset travel keyword set based on a travel attack text to form a keyword combination;
s120, searching pictures meeting the keyword combination to form a photo library;
s130, sorting the quality of pictures in a target photo library through a trained picture quality network; and
and S140, adding at least one picture which is ranked at the front into a preset position in the text of the travel attack.
According to the invention, an algorithm model is established for the condition that a large number of low-quality pictures are preferentially displayed at present, so that the quality evaluation of the online pictures is realized in time, and the accuracy and reliability of the subsequent head picture selection and picture sorting are ensured.
In a preferred embodiment, in the step S110, the travel keyword set includes a location keyword subset and a topic keyword subset, at least one location keyword is obtained from the location keyword subset and at least one topic keyword is obtained from the topic keyword subset based on the text of the travel attack, respectively;
in the step S120, a photo library is searched for the photos satisfying both the location keyword and the subject keyword.
In a preferred embodiment, in the step S120, the condition that the picture satisfies the location keyword is:
shooting location information of the picture hits the location keyword; or alternatively
And the GPS positioning information of the shooting location of the picture is in the GPS positioning range corresponding to the location keyword.
In a preferred embodiment, in the step S120, the condition that the picture satisfies the topic keyword is:
the pictures are respectively labeled through a trained picture network, and each picture obtains at least one label;
at least one of the tags of the picture hits the subject keyword.
In a preferred embodiment, the picture tag network performs training of picture tag classification according to the preset travel keyword set, and outputs at least one travel keyword in the preset travel keyword set to the picture.
In a preferred embodiment, the pictures satisfying the place keyword are formed into a process photo library, and the picture satisfying the subject keyword is searched in the process photo library to form a photo library.
In a preferred embodiment, the range of the search photo in step S120 is at least one cloud photo library corresponding to the current user account or a photo library stored in the mobile terminal currently used by the user.
In a preferred embodiment, in the step S110, the text of the travel attack is a paragraph of natural text currently edited by the user;
in the step S140, the selected picture is added before the paragraph of the natural text currently edited by the user.
In a preferred embodiment, the training picture quality network in step S130 includes the following steps:
s131, clustering high-frequency picture types by searching on-line pictures of different types and styles;
s132, establishing an image information mining model based on multi-scale features, and performing multi-dimensional evaluation on the searched attack picture information, wherein the dimensions comprise definition, layout, color and saturation; and
s133, converting the result of the mining model into decimal places and reserving fixed digits, and sorting according to the final quality score.
In a preferred embodiment, the step S132 includes:
based on preset keywords, respectively finding out picture information under different keywords, and generating a quality evaluation data set, wherein evaluation standards comprise definition, saturation, composition, color and the like;
preprocessing the pictures in the data set, uniformly adjusting the sizes of all the images to be fixed in length and width, and normalizing the image pixels; carrying out batch processing on the adjusted fixed-size pictures and inputting the pictures into a subsequent network; for fractional segments with less partial pictures, horizontally turning the pictures to form new pictures, and putting the new pictures into a data set;
according to the characteristics of the pictures, a corresponding deep convolutional neural network is built, and multi-scale-based MultiNet is designed according to the characteristics of the attack pictures; the whole network consists of two parts, wherein the first part is a backbone network, the backbone network uses a densnet 121 network and consists of 4 blocks, each block comprises a plurality of convolution layers, an activation layer and the like, and the total of 121 layers of convolution layers are the same in characteristic size, so that each layer can be connected with the characteristics of all the previous layers on a channel, the characteristic reuse is realized, and the efficiency is improved; performing model compression on the last layer of each block, reducing the size of the features, then averaging the features to 1*1, performing the operation on four blocks, and finally combining the four features on a channel to realize multi-scale; the second part is a full-connection network, the extracted characteristics are independently trained, the full-connection network comprises 3 full-connection layers, a plurality of activation layers, a pooling layer and the like, a prediction layer is finally output, 8 categories are output, then probability combination is carried out on the 8 categories, and a final score is output;
wherein a cross entropy loss function is used for class prediction, the formula is defined as follows, where p t For the category prediction probability, y is the image category label
loss＝∑y log(p t )
Dividing an acquired data set into a training set, a verification set and a test set, performing migration learning by using a model obtained based on large-scale scene data set training, performing operations such as zooming, normalization and the like on pictures in the migration learning process to enhance the robustness of the model, performing multi-scale feature extraction by using a network after migration learning, training the extracted features, and iterating the model until the test effect of the model on the verification set reaches the optimal;
forward prediction is carried out on the on-line attack pictures by using the trained model, and the prediction probability of the image on each fractional segment is output;
the prediction probability is combined with the score to obtain a final score, wherein i is the score, and p (i) is the prediction probability of the score;
and outputting the score as a result of the mining model.
As shown in fig. 2 to 4, the implementation of the travel attack editing method based on photo library of the present invention is illustrated. Referring to fig. 2, a user edits a travel scenario of going to the hokkaido in the last year on his mobile phone 1, writes "we go to the hokkaido in winter to ski in the last year. The travel keyword set includes a preset site keyword subset (including, for example, shanghai, beijing, hokkaido, new York, etc.) and a preset topic keyword subset (including, for example, food, hot spring, surfing, skiing, etc.), based on the text of travel attack, "we go to Hokkaido in winter for the past year skiing.
With reference to figure 3 of the drawings, a plurality of photos 21, 22, 23, 24 in a photo library 2 stored in a mobile phone 1 currently used by a user searching pictures meeting the place keywords and the theme keywords in the environment and the like to form a photo library. (wherein the photo 21 is a group photo taken in the North sea, the photo 22 is a ski run taken in the North sea, the photo is taken with the GPS of the mobile terminal obtaining the positioning information of the shooting place at the time of the shooting), the photos 21, 22 are selected by comparing the GPS information of each photo taken by the mobile phone 1 with the GPS positioning range corresponding to the North sea, and when the GPS information of the photo is in the GPS positioning range corresponding to the North sea, the process photo library 3 is formed.
And respectively labeling all the photos 21 and 22 in the process photo library through a trained picture network, wherein each picture obtains at least one label, the label of the photo 21 is a photo, the label of the photo 22 is skiing, the label of the photo 22 meets the theme keyword of skiing, and the photo library such as the photo 21 is formed by analogy (namely, the photos meeting the requirements of 'North sea' and 'skiing' in the mobile phone 1). The label network performs training of picture label classification according to the preset travel keyword set, and outputs at least one travel keyword in the preset travel keyword set to the picture.
And sequencing the quality of the pictures in the target photo library through a trained picture quality network to obtain a sequence with the photo quality from high to low.
The invention discloses a picture quality ordering method, which comprises the following steps:
s01, excavating on-line attack pictures, including but not limited to scenic spots, delicacies, restaurants and the like.
S02, establishing an image information mining model based on multi-scale features, wherein an image information mining model algorithm comprises, but is not limited to, image classification and a picture evaluation model. The information of the mined attack pictures is comprehensively and comprehensively evaluated in a plurality of aspects, including but not limited to definition, layout, color, saturation and the like
S03, converting the model result into decimal, reserving a fixed digit, and sequencing according to the final quality score.
S04, manually intervening and confirming the picture quality, and performing error correction.
For S01, in the implementation of the scheme, the high-frequency picture types including indoor, natural landscape, food and the like are clustered by mining on-line pictures of different types and styles.
The implementation flowchart of the implementation method of the image information extraction in S02 is shown in fig. 2. Specifically, the method comprises the following steps:
step one: data set construction
According to the keywords mentioned in S01, picture information under different keywords is respectively found out, a quality evaluation data set is generated, quality evaluation and sample labeling are respectively carried out according to different subjective experiences of users under different scenes, each scene picture is ensured not to be lower than 1000, wherein the quality evaluation score ranges from 3 minutes to 10 minutes, and evaluation standards are based on various aspects including but not limited to definition, saturation, composition, color and the like. The final result of the picture quality evaluation is based on the same result value of a plurality of persons.
Step two: data preprocessing
Preprocessing the pictures in the data set, uniformly adjusting the sizes of all the images to be fixed in length and width, and normalizing the image pixels. And carrying out batch processing on the adjusted fixed-size pictures and inputting the pictures into a subsequent network. And for fractional segments with fewer partial pictures, horizontally turning the pictures to form new pictures, and putting the new pictures into a data set.
Step three: network construction
According to the picture characteristics, a corresponding deep convolutional neural network is built, and the multi-scale MultiNet based on the picture characteristics is designed. The whole network is composed of two parts, wherein the first part is a backbone network, the backbone network is composed of 4 blocks by using a densnet 121 network, each block comprises a plurality of convolution layers, an activation layer and the like, the total of 121 layers of convolution layers are the same in characteristic size, so that each layer can be connected with the characteristics of all the previous layers on a channel, characteristic reuse is realized, and efficiency is improved. And (3) carrying out model compression on the last layer of each block, reducing the size of the features, then carrying out averaging pooling on the features to 1*1, carrying out the operation on four blocks, and finally carrying out channel combination on the four features to realize multi-scale. The second part is a full-connection network, the extracted features are independently trained, the full-connection network comprises 3 full-connection layers, a plurality of activation layers, a pooling layer and the like, a prediction layer is finally output, 8 categories are output, and then probability combination is carried out on the 8 categories, so that a final score is output.
Wherein a cross entropy loss function is used for class prediction, the formula is defined as follows, where p t For the category prediction probability, y is the image category label
loss＝∑y log(p t )
Step four: model training
In the implementation of the scheme, firstly, the acquired data set is divided into a training set, a verification set and a test set, then a model obtained based on large-scale scene data set training is used for migration learning, in the process of migration learning, operations such as zooming and normalization are performed on pictures to enhance model robustness, then a network after migration learning is used for multi-scale feature extraction, the extracted features are trained, and the test effect of the iterative model on the verification set is optimized.
Step five: model prediction
And D, forward prediction is carried out on the on-line attack picture by using the trained model in the step four, and the prediction probability of the image on each fractional segment is output.
For S03, the final decimal fraction is obtained by combining the predictive probability obtained in S02 with the score. The formula is as follows, where i is the score and p (i) is the predictive probability of the score.
And S04, comparing the score predicted by the model with the score estimated manually in multiple aspects to obtain the actual difference between the predicted picture and the score section picture, and supplementing a corresponding negative sample according to the actual difference to iterate the model.
In the actual implementation process, feedback results of product personnel are continuously collected in the S04 stage, and the steps from S01 to S03 are periodically and repeatedly executed to improve the performance of the image quality evaluation model based on the multi-scale features. The invention can be based on massive picture information on the attack line, comprises various pictures such as scenic spots, restaurants, delicacies, cities and the like, and utilizes a multi-scale feature extraction method to evaluate the quality of the pictures, thereby providing a good basis for subsequent head picture selection and picture sorting, saving operation and maintenance cost, ensuring the quality of the head pictures and the picture sorting quality, and effectively improving the flow and experience of users.
Finally, referring to fig. 4, the highest quality photo 21 in the sequence is added before the paragraph of the natural text "we last winter go to hokkaido skiing … …" in the user's current edit.
In the whole process, the user does not need to manually select the photos, and the photos which are most relevant to the text and have the best photo quality can be automatically selected according to the text edited by the user to carry out automatic picture allocation.
In a variation, the above steps (step S110 to step S140, etc.) are performed on the text of the travel attack edited by the user in real time, so that the photos can be reselected and the photos of the matching photos can be replaced in real time in different schedules of the text of the travel attack edited by the user, in this way, full utilization of the complete information of the text of the travel attack edited by the user is ensured, and the photos which are most relevant to the text and have the best photo quality are obtained for automatic matching.
The picture quality sorting method based on the photo library can realize quality evaluation of pictures, carries out multi-aspect overall evaluation on the pictures from the angle of a user, rapidly and accurately obtains the picture quality, enables the most attractive pictures to be ranked first, enables the user to obtain the best-looking picture at the first time when obtaining information, and enhances user experience.
Fig. 5 is a schematic block diagram of a photo library-based picture quality ordering system of the present invention. As shown in fig. 5, the photo library-based picture quality ordering system 5 of the present invention includes:
the keyword obtaining module 51 obtains at least one keyword with the highest association degree from the preset travel keyword set based on the text of the travel attack, and forms a keyword combination.
The photo searching module 52 searches the photo forming photo library satisfying the keyword combination.
The quality ranking module 53 ranks the quality of the pictures in the target photo library over a trained picture quality network.
The picture insertion module 54 adds the top ranked at least one picture to a preset location in the travel attack text.
The picture quality sorting system based on the photo library can realize quality evaluation of pictures, carries out multi-aspect overall evaluation on the pictures from the angle of a user, rapidly and accurately acquires the picture quality, enables the most attractive pictures to be ranked first, enables the user to acquire the best-looking picture at the first time when acquiring information, and enhances user experience.
The embodiment of the invention also provides a picture quality sorting device based on the photo library, which comprises a processor. A memory having stored therein executable instructions of a processor. Wherein the processor is configured to execute steps of a photo library-based travel attack editing method via execution of the executable instructions.
As shown above, the picture quality sorting system based on the photo library can realize quality evaluation of pictures, and can perform multi-aspect overall evaluation on the pictures from the angle of a user, so that the picture quality can be rapidly and accurately obtained, the most attractive picture can be ranked first, the user can often obtain the best-looking picture at the first time when obtaining information, and the user experience is enhanced.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" platform.
Fig. 6 is a schematic structural diagram of a picture quality sorting apparatus based on a photo library of the present invention. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 6. The electronic device 600 shown in fig. 6 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 6, the electronic device 600 is in the form of a general purpose computing device. Components of electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including memory unit 620 and processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code executable by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above-described electronic prescription flow processing method section of the present specification. For example, the processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 6201 and/or cache memory unit 6202, and may further include Read Only Memory (ROM) 6203.
The storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 600, and/or any device (e.g., router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 650. Also, electronic device 600 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage platforms, and the like.
The embodiment of the invention also provides a computer readable storage medium for storing a program, and the steps of the travel attack editing method based on the photo library are realized when the program is executed. In some possible embodiments, the aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the electronic prescription stream processing method section of this specification, when the program product is run on the terminal device.
As shown above, the picture quality sorting system based on the photo library can realize quality evaluation of pictures, and can perform multi-aspect overall evaluation on the pictures from the angle of a user, so that the picture quality can be rapidly and accurately obtained, the most attractive picture can be ranked first, the user can often obtain the best-looking picture at the first time when obtaining information, and the user experience is enhanced.
Fig. 7 is a schematic structural view of a computer-readable storage medium of the present invention. Referring to fig. 7, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In summary, the invention aims to provide a travel attack editing method, a system, equipment and a storage medium based on a photo library, and the photo library-based picture quality sorting system can realize quality evaluation of pictures, integrally evaluate the pictures in multiple aspects from a user angle, quickly and accurately acquire the picture quality, lead the most attractive picture to be at first, lead a user to acquire the best looking picture at the first time when acquiring information, and enhance user experience.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (4)
1. A travel attack editing method based on a photo library is characterized by comprising the following steps:
s110, obtaining at least one keyword with highest association degree from a preset travel keyword set based on a travel attack text, and forming a keyword combination, wherein the preset travel keyword set comprises a place keyword subset and a theme keyword subset, at least one place keyword is obtained from the place keyword subset based on the travel attack text, at least one theme keyword is obtained from the theme keyword subset, and the travel attack text is a natural text paragraph in current editing of a user;
s120, searching pictures meeting the place keywords and the theme keywords simultaneously to form a photo library, wherein the conditions of the pictures meeting the place keywords are as follows: the shooting place information of the picture hits the place keyword, or the shooting place GPS positioning information of the picture is in the GPS positioning range corresponding to the place keyword; the condition that the picture meets the theme keyword is as follows: the pictures are respectively labeled through a trained picture network, and each picture obtains at least one label; at least one of the labels of the pictures hits the theme keywords, the picture label network respectively carries out training of picture label classification according to the preset travel keyword set, at least one of the travel keywords in the preset travel keyword set is output to the pictures, the pictures meeting the place keywords form a process photo library, the process photo library is searched for pictures meeting the theme keywords to form a photo library, and the range of the searched pictures is at least one cloud photo library corresponding to the current user account or a photo library stored in a mobile terminal currently used by a user;
s130, sorting the quality of pictures in a target photo library through a trained picture quality network, and training the picture quality network, wherein the method comprises the following steps of: clustering high-frequency picture types by searching on-line pictures of different types and styles; establishing an image information mining model based on multi-scale features, and performing multi-dimensional evaluation on the searched attack picture information, wherein the dimensions comprise definition, layout, color and saturation; based on preset keywords, respectively finding out picture information under different keywords, and generating a quality evaluation data set, wherein evaluation standards comprise definition, saturation, composition and color; preprocessing the pictures in the data set, uniformly adjusting the sizes of all the images to be fixed in length and width, and normalizing the image pixels; carrying out batch processing on the adjusted fixed-size pictures and inputting the pictures into a subsequent network; for fractional segments with few partial pictures, horizontally turning over the pictures to form new pictures, and putting the new pictures into a data set; building a corresponding deep convolutional neural network according to the characteristics of the picture; the whole network consists of two parts, wherein the first part is a backbone network, the backbone network uses a dense convolution network and consists of 4 blocks, each block comprises a plurality of convolution layers and an activation layer, 121 layers of convolution layers are all adopted, and the characteristic size of each layer is the same, so that each layer can be connected with the characteristics of all the previous layers on a channel to realize characteristic reuse; performing model compression on the last layer of each block, reducing the size of the features, then averaging the features to 1*1, performing the operation on four blocks, and finally combining the four features on a channel to realize multi-scale; the second part is a full-connection network, the extracted characteristics are independently trained, the full-connection network comprises 3 full-connection layers, a plurality of activation layers and a pooling layer, a prediction layer is finally output, 8 categories are output, then probability combination is carried out on the 8 categories, and a final score is output;
wherein a cross entropy loss function is used for class prediction, the formula is defined as follows, where p t For the category prediction probability, y is the image category label
loss＝∑ylog(p t )
Dividing an acquired data set into a training set, a verification set and a test set, performing migration learning by using a model obtained based on large-scale scene data set training, scaling and normalizing pictures in the migration learning process to enhance model robustness, performing multi-scale feature extraction by using a network after migration learning, training the extracted features, and iterating the model until the test effect of the model on the verification set reaches the optimal;
forward prediction is carried out on the on-line attack pictures by using the trained model, and the prediction probability of the image on each fractional segment is output;
combining the prediction probability with the score to obtain a final score, wherein i is the score, and p (i) is the prediction probability of the score;
outputting the score as a result of the mining model; and
converting the result of the mining model into decimal places, reserving fixed digits, and sequencing according to the final score; and
and S140, adding at least one picture which is ranked at the front into a preset position in the text of the travel attack, and adding the selected picture into the text of the natural text paragraph currently edited by the user.
2. A photo library-based picture quality ordering system for implementing the photo library-based travel attack editing method of claim 1, comprising:
the keyword obtaining module is used for obtaining at least one keyword with highest association degree from a preset travel keyword set based on the text of the travel attack, so as to form a keyword combination;
the photo searching module searches the pictures meeting the keyword combination to form a photo library;
the quality sorting module sorts the quality of the pictures in the target photo library through a trained picture quality network; and
and the picture inserting module is used for adding at least one picture which is ranked at the front into a preset position in the text of the travel attack.
3. A picture quality ordering apparatus based on a photo library, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the photo gallery-based travel attack editing method of claim 1 via execution of the executable instructions.
4. A computer-readable storage medium storing a program, wherein the program when executed implements the steps of the photo gallery-based travel attack editing method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011287071.6A CN112256907B (en) | 2020-11-17 | 2020-11-17 | Travel attack editing method, system, equipment and storage medium based on photo library |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011287071.6A CN112256907B (en) | 2020-11-17 | 2020-11-17 | Travel attack editing method, system, equipment and storage medium based on photo library |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112256907A CN112256907A (en) | 2021-01-22 |
| CN112256907B true CN112256907B (en) | 2024-04-16 |
Family
ID=74265905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011287071.6A Active CN112256907B (en) | 2020-11-17 | 2020-11-17 | Travel attack editing method, system, equipment and storage medium based on photo library |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112256907B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104239535A (en) * | 2014-09-22 | 2014-12-24 | 重庆邮电大学 | Method and system for matching pictures with characters, server and terminal |
| CN106355429A (en) * | 2016-08-16 | 2017-01-25 | 北京小米移动软件有限公司 | Image material recommendation method and device |
| CN109783798A (en) * | 2018-12-12 | 2019-05-21 | 平安科技（深圳）有限公司 | Method, apparatus, terminal and the storage medium of text information addition picture |
| CN110717058A (en) * | 2019-09-23 | 2020-01-21 | Oppo广东移动通信有限公司 | Information recommendation method and device and storage medium |
-
2020
- 2020-11-17 CN CN202011287071.6A patent/CN112256907B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104239535A (en) * | 2014-09-22 | 2014-12-24 | 重庆邮电大学 | Method and system for matching pictures with characters, server and terminal |
| CN106355429A (en) * | 2016-08-16 | 2017-01-25 | 北京小米移动软件有限公司 | Image material recommendation method and device |
| CN109783798A (en) * | 2018-12-12 | 2019-05-21 | 平安科技（深圳）有限公司 | Method, apparatus, terminal and the storage medium of text information addition picture |
| CN110717058A (en) * | 2019-09-23 | 2020-01-21 | Oppo广东移动通信有限公司 | Information recommendation method and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112256907A (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10289643B2 (en) | Automatic discovery of popular landmarks | |
| CN109657054B (en) | Abstract generation method, device, server and storage medium | |
| CN111062871B (en) | Image processing method and device, computer equipment and readable storage medium | |
| CN110245259B (en) | Video labeling method and device based on knowledge graph and computer readable medium | |
| US8577882B2 (en) | Method and system for searching multilingual documents | |
| CN104794171B (en) | Mark the method and device of picture geographical location information | |
| CN111046225B (en) | Audio resource processing method, device, equipment and storage medium | |
| CN111651636A (en) | Video similar segment searching method and device | |
| CN109408672B (en) | Article generation method, article generation device, server and storage medium | |
| CN110297897B (en) | Question-answer processing method and related product | |
| CN110263218B (en) | Video description text generation method, device, equipment and medium | |
| CN108959304B (en) | Label prediction method and device | |
| CN108763481A (en) | A kind of picture geographic positioning and system based on extensive streetscape data | |
| CN108268540A (en) | A kind of video recommendation method based on video similarity, system and terminal | |
| Kaneko et al. | Visual event mining from geo-tweet photos | |
| CN113704506A (en) | Media content duplication eliminating method and related device | |
| CN115269913A (en) | Video retrieval method based on attention fragment prompt | |
| WO2017130201A1 (en) | Method and system for providing audio content | |
| CN113704507A (en) | Data processing method, computer device and readable storage medium | |
| CN113301382B (en) | Video processing method, device, medium, and program product | |
| Panda et al. | Heritage app: annotating images on mobile phones | |
| CN112256907B (en) | Travel attack editing method, system, equipment and storage medium based on photo library | |
| CN112949777B (en) | Similar image determining method and device, electronic equipment and storage medium | |
| CN114299435A (en) | Scene clustering method and device in video and related equipment | |
| Seymour et al. | Multi-label triplet embeddings for image annotation from user-generated tags |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |