CN107122393A - Electron album generation method and device - Google Patents
Electron album generation method and device Download PDFInfo
- Publication number
- CN107122393A CN107122393A CN201710138877.0A CN201710138877A CN107122393A CN 107122393 A CN107122393 A CN 107122393A CN 201710138877 A CN201710138877 A CN 201710138877A CN 107122393 A CN107122393 A CN 107122393A
- Authority
- CN
- China
- Prior art keywords
- photo
- descriptive information
- adjective
- sample
- electron album
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses a kind of electron album generation method and device.Wherein method includes：Content analysis is carried out to photo, the descriptive information of photo is generated；And Video Composition is carried out according to photo and descriptive information, to generate target electronic photograph album.Thus, in the generating process of electron album, the making of electron album also can intelligently be completed without requiring that user possesses image procossing knowledge, while manual manufacture cost is saved, photograph album content is enriched, photograph album is improved interesting, imagination space has been expanded, and has improved Consumer's Experience.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to a kind of electron album generation method and device.
Background technology
With the development of computer technology and multimedia technology, the multimedia resource that people touch becomes increasingly abundant.With
People interest is widened, and many people directly shoot video with video camera or digital camera now, on computers using playing
Device software watches video frequency program, has become a kind of very universal study, leisure, entertainment way of user.
At the same time, some simply marvelous pictures can be concerned about on film, and want to preserve, by these
Excellent picture constitutes an exquisite electron album.Electron album is to produce and generate video by one group of given photo, is led to
Reason condition can be with background music and descriptive text.In correlation technique, Most electronic photograph album Making programme is all user's base
In Video editing software, descriptive text and background music are mixed on photo to complete electron album.
But, the problem of presently, there are is：Based on Video editing software to complete electron album, this production method belongs to
Pure artifact is, it is necessary to which the user for possessing certain professional knowledge could complete descriptive text of being arranged in pairs or groups on photo, significantly
Manual manufacture cost is increased, it is not intelligent.
The content of the invention
The purpose of the present invention is intended at least solve one of above-mentioned technical problem to a certain extent.
Therefore, first purpose of the present invention is to propose a kind of electron album generation method.This method is saving artificial
While cost of manufacture, photograph album content is enriched, photograph album interest is improved, has expanded imagination space, and improve user's body
Test.
Second object of the present invention is to propose a kind of electron album generating means.
To reach above-mentioned purpose, the electron album generation method that first aspect present invention embodiment is proposed, including：To photo
Content analysis is carried out, the descriptive information of the photo is generated；And regarded according to the photo and the descriptive information
Frequency is synthesized, to generate target electronic photograph album.
The electron album generation method of the embodiment of the present invention, can generate retouching for photo by carrying out content analysis to photo
The property stated information, and Video Composition is carried out according to photo and descriptive information, to generate target electronic photograph album.It is based on photograph image
Content analyzed, and appropriate descriptive information is provided for the photo based on analysis result automatically, so, in electron album
Generating process in, also can intelligently complete the making of electron album without requiring that user possesses image procossing knowledge, section
While less manpower cost of manufacture, photograph album content is enriched, photograph album interest is improved, has expanded imagination space, and improve
Consumer's Experience.
To reach above-mentioned purpose, the electron album generating means that second aspect of the present invention embodiment is proposed, including：It is descriptive
Information generating module, for carrying out content analysis to photo, generates the descriptive information of the photo；And electron album generation
Module, for carrying out Video Composition according to the photo and the descriptive information, to generate target electronic photograph album.
The electron album generating means of the embodiment of the present invention, can carry out content by descriptive information generation module to photo
Analysis, generates the descriptive information of photo, and electron album generation module carries out Video Composition according to photo and descriptive information, with
Generate target electronic photograph album.I.e. the content based on photograph image is analyzed, and is provided automatically for the photo based on analysis result
Appropriate descriptive information, so, in the generating process of electron album, also can without requiring that user possesses image procossing knowledge
The making of electron album is intelligently completed, while manual manufacture cost is saved, photograph album content is enriched, improves photograph album
Interest, has expanded imagination space, and improve Consumer's Experience.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
The above-mentioned and/or additional aspect and advantage of the present invention will become from description of the accompanying drawings below to embodiment is combined
Substantially and be readily appreciated that, wherein：
Fig. 1 is the flow chart of electron album generation method according to an embodiment of the invention；
Fig. 2 is the flow chart of the electron album generation method according to a specific embodiment of the invention；
Fig. 3 (a), (b), (c) are the signal of the classification results of object type in photo according to embodiments of the present invention with (d)
Figure；
Fig. 3 (e) is the schematic diagram of the vector representation of word " present " according to embodiments of the present invention；
Fig. 4 is the exemplary plot of vgg16 model structures according to embodiments of the present invention；
Fig. 5 is the flow chart of the alternative descriptive information set of generation according to an embodiment of the invention；
Fig. 6 is the flow chart of generation descriptive information according to an embodiment of the invention；
Fig. 7 is the structural representation of electron album generating means according to an embodiment of the invention；
Fig. 8 is the structural representation of the electron album generating means according to a specific embodiment of the invention；
Fig. 9 is the structural representation of generation submodule according to an embodiment of the invention；
Figure 10 is the structural representation of the electron album generating means according to another specific embodiment of the invention；
Figure 11 is the structural representation of the electron album generating means according to another specific embodiment of the invention.
Embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and be not considered as limiting the invention.
Below with reference to the accompanying drawings the electron album generation method and device of the embodiment of the present invention described.
Fig. 1 is the flow chart of electron album generation method according to an embodiment of the invention.It should be noted that this hair
The electron album generation method of bright embodiment can be applied to the electron album generating means of the embodiment of the present invention.The electron album is given birth to
Terminal device can be configured in into device, the terminal device can be that (such as mobile phone, tablet personal computer, individual digital are helped mobile terminal
The hardware devices such as reason), PC etc..
As shown in figure 1, the electron album generation method can include：
S110, content analysis is carried out to photo, generates the descriptive information of photo.
It is appreciated that the content analysis to photo for convenience, can enter before content analysis is carried out to photo to photo
Row pretreatment.For example, original photo can have situations such as oversized, length-width ratio is inconsistent, it is necessary to carry out size change over processing,
In the case of keeping the ratio of width to height immovable in photo, the width and height dimensions of suitable target electronic photograph album are converted to.
After being pre-processed to photo, content analysis can be carried out to pretreated photo, and based on analysis result
Generate the descriptive information of photo.Specifically, CNN (Convolutional Neural Network, convolutional Neural net can be used
Network) method, load object in the good model comparison piece of training in advance and analyzed, obtain the object type in photo, afterwards, can
The descriptive information of the photo is selected from the alternative descriptive information set previously generated according to object type.It is specific real
Existing mode can be found in the description of subsequent embodiment.
It should be noted that the language form of descriptive information is not limited to Chinese, English, German, Russian etc. can also be；
The file format of descriptive information can be text and/or voice etc..That is, arranging in pairs or groups to the description on target electronic photograph album
Property information can be Chinese text, can also be other spoken and written languages, also can also be Chinese speech etc., specific presentation mode can
Determine, be not especially limited herein according to the actual requirements.
S120, carries out Video Composition, to generate target electronic photograph album according to photo and descriptive information.
Specifically, the Video Composition of electron album can be carried out by video coding technique according to photo and descriptive information,
Finally give target electronic photograph album.Concrete implementation process can be found in the description of subsequent embodiment.
The electron album generation method of the embodiment of the present invention, can generate retouching for photo by carrying out content analysis to photo
The property stated information, and Video Composition is carried out according to photo and descriptive information, to generate target electronic photograph album.It is based on photograph image
Content analyzed, and appropriate descriptive information is provided for the photo based on analysis result automatically, so, in electron album
Generating process in, also can intelligently complete the making of electron album without requiring that user possesses image procossing knowledge, section
While less manpower cost of manufacture, photograph album content is enriched, photograph album interest is improved, has expanded imagination space, and improve
Consumer's Experience.
In order that the realization of the present invention can clearly be understood by obtaining those skilled in the art, below in conjunction with Fig. 2
Electron album generation method to the present invention is described further.
Fig. 2 is the flow chart of the electron album generation method according to a specific embodiment of the invention.As shown in Fig. 2 should
Electron album generation method can include：
S210, photo is identified according to default disaggregated model, to obtain the object type in photo.
It is appreciated that the content analysis to photo for convenience, can enter before content analysis is carried out to photo to photo
Row pretreatment.For example, original photo can have situations such as oversized, length-width ratio is inconsistent, it is necessary to carry out size change over processing,
In the case of keeping the ratio of width to height immovable in photo, the width and height dimensions of suitable target electronic photograph album are converted to.
For example, so that descriptive information is Chinese text as an example, after being pre-processed to photo, CNN can be used
The good disaggregated model of method loading training in advance, classifies to the object in photo.It is appreciated that during classification results are photo
Object belong to the probability of each classification, threshold value min_prob (for example, can be set as 0.5) can be pre-set, each class is taken
Maximum in other probability, if the value is more than min_prob, takes the corresponding Chinese label (label) of the category, otherwise
Extract label failures, i.e. object type and obtain failure.If the object type multiple Chinese label of correspondence in photo, take each
One label of word frequency highest in label.
It is for instance possible to use the object in the good vgg16 model comparison pieces of pre-training is classified, classification results are photo
In object belong to the probability of 1000 classifications.As shown in Fig. 3 (a), prob is the probability of identification classification, predict_index
For vgg16 classification results sequence numbers, cn_label is by the corresponding Chinese label of the former classifications of vgg16 1000.As can be seen that Fig. 3
(a) most probable value is 0.628759 in, more than threshold value 0.5, then can assert that main object is hamster in the photo.
It should be noted that the present invention uses CNN as sorting algorithm, it is because current CNN classification accuracies have reached
To a certain extent, as shown in Fig. 3 (b)-Fig. 3 (d), to carry out the classification that object classification is obtained to each photo using vgg16 models
As a result.Wherein, most probable value is 0.374364 in Fig. 3 (b), is the maximum in these each class probabilities, then can assert
Main object is chihuahua in the photo；Most probable value is 0.403069 in Fig. 3 (c), in being these each class probabilities
Maximum, and multiple label (i.e. billiard table, pool table) are now corresponded to, because the occurrence number of billiard table is more than pool table
Occurrence number, so the object predominantly billiard table in the photo can be assert；Most probable value is 0.756128 in Fig. 3 (d), is
Maximum in these each class probabilities, then can assert that main object is upright piano in the photo.
As a kind of example, as shown in figure 4, being vgg16 model structures, the model is 16 layers of convolutional neural networks.Need
It is noted that this step can also use other CNN disaggregated models, many class probability results of photo only need to be obtained,
This is not especially limited.
S220, according to object type, chooses the descriptive letter of photo from the alternative descriptive information set previously generated
Breath.
It should be noted that in an embodiment of the present invention, alternative descriptive information set can be previously generated.Make
For a kind of example, as shown in figure 5, the alternative descriptive information set can be previously generated by following steps：
S510, obtains the descriptive words list in high frequency word list, wherein, descriptive words list includes multiple describe
Word sample and multiple adverbial word samples.
For example, so that descriptive information is Chinese text as an example, Chinese high frequency words can be got by internet collection
Descriptive words list in list, the descriptive words list may include multiple adjective samples and multiple adverbial word samples.Example
Such as, the adjective sample in Chinese high frequency word list can be taken, such as：It is " good new general high long basic old complete few fast low big more than size
Measure the easy freely strong actively difficult closely complicated identical many many rapid red vast simply remote abundant substantially objective happinesss of old morning of nature
It is maximum beautiful to being also clear that the black very huge necessary stable bad deep great heat normal extensively of universal advanced balance conscientiously writes
Name ... "；The adverbial word sample in Chinese high frequency word list is taken, the adverbial word sample of the positive degree of description is obtained by artificial filter,
Such as：" very very especially quite most very very too many is so exactly extremely tangible well extremely very really extremely the most special really
It is definitely how rather abnormal enough ".
S520, obtains the vector representation of each adjective sample and each adverbial word sample respectively, calculates each adjective sample
Originally the second similarity between each adverbial word sample.
Specifically, continue so that descriptive information is Chinese text as an example, Chinese corpus can be obtained by internet collection,
By Chinese Word Segmentation, by neural metwork training, the vector representation of word in corpus is obtained.For example：Word2vec can be used
Word in language material is handled, the K dimension real number vector representations of generation word.For example, " present " one word, is counted by word2vec
Calculation can obtain its vector representation for shown in Fig. 3 (e).So, from the corpus term vector set, each can be obtained respectively
The vector representation of adjective sample and each adverbial word sample, calculates between each adjective sample and each adverbial word sample
Two similarities (such as cosine cosine similarities).
S530, for each adjective sample, obtains the N number of adverbial word sample of the second similarity highest with each adjective sample
This, wherein, N is positive integer.
That is, each adjective sample acquisition and the N number of adverbial word sample of its similarity highest.For example, N is 3, you can obtain
Take 3 adverbial word samples of similarity highest with each adjective sample.
S540, each adjective sample is combined to generate alternative descriptive information with corresponding N number of adverbial word sample
Set.
That is, each adjective sample is combined with as one group of Collocation with corresponding N number of adverbial word sample
Language, such as：" extremely excellent, how glorious, enough beautiful " etc., finally, all collocation words is gathered together, alternatively retouched
The property stated information aggregate.It is appreciated that the collocation word obtained by above-mentioned acquisition modes, as meets the language material speech habits
Text is described, photo text description alternately.
It should be added that, the composition of corpus may decide that the style of generation descriptive information.For example, can come
From the then daily description of output of encyclopaedia language material, can come from social networks output has neologisms term of network style etc..
It should also be noted that, in one embodiment of the invention, can also constantly be obtained from internet and newly enter text, with
Update corpus.Thus, the continuous renewal to corpus is passed through, it is ensured that the freshness of word in corpus, so as to improve
The degree of accuracy of the descriptive information of generation and flexibility ratio.
Thus, alternative descriptive information set can be generated by above-mentioned steps S510-S540.So, in this step,
It can be chosen by neural LISP program LISP NLP technologies, according to object type from the alternative descriptive information set previously generated
The descriptive information of photo, i.e., generate descriptive information to the photo.As a kind of example, as shown in fig. 6, according to object class
Not, from the alternative descriptive information set previously generated choose photo descriptive information implement process may include as
Lower step：
S610, obtains the text term vector of object type.
Specifically, the text of object type can be handled by word2vec, to generate the text of the object type
Term vector.
S620, obtains high frequency corresponding with object type and describes word list, wherein, high frequency is described in word list with multiple
High frequency adjective.
As a kind of example, it can be obtained from the adjective sample in the descriptive words list got in advance with being somebody's turn to do
The corresponding high frequency of object type describes word list.
S630, the text term vector of calculating object type is similar to first between the adjectival term vector of each high frequency
Degree.
S640, obtains the target adjective that the first similarity is more than predetermined threshold value.
It is appreciated that predetermined threshold value min_similar can be pre-defined, for example, the predetermined threshold value can value 0.3.
S650, descriptive information is chosen according to target adjective from alternative descriptive information set.
Specifically, adjective sample corresponding with the target adjective can be found out first from alternative descriptive information set
With the matched combined of adverbial word sample, afterwards, can be according to the adjective sample the second phase corresponding with the matched combined of adverbial word sample
Determine that the target adjective is directed to the descriptive information of the photo like degree.
Alternatively, in one embodiment of the invention, according to target adjective from alternative descriptive information set
Choose before descriptive information, can also choose wherein the first phase from the first similarity is more than the target adjective of predetermined threshold value
Like degree M adjective of highest as target adjective, M is positive integer, and such as M is 3.So, according to the M target adjective
Descriptive information is chosen from alternative descriptive information set.
For example, by taking the upright piano object shown in Fig. 3 (d) as an example, as shown in Table 1 below, Label->Adj is photograph
Piece label texts and target adjective top3 similarities, Adj->Adv is the collocations in alternative descriptive information set,
Description is the photo descriptive information selected.So, the descriptive information of photo is obtained by the present invention, even if
Some recognition effects are not absolutely accurate, but to describe effect still good for text.
S230, video segment is converted to by the descriptive information of photo and photo.
For example, so that the number of descriptive information is 3 as an example, using predefined electron album target size as mark
Standard, it is random from upper left 1/3, upper right 1/3, lower-left 1/3, bottom right 1/3 to take three positions to occur as the text of descriptive information
Position.So, the descriptive information of photo and photo can be converted into video segment in the following manner：First, with Original Photo
Piece is background, replicates the frame of 20 frame number 1~20, position occurs in each frame correspondence text, using predefined font according to from small
To small font size change is arrived again greatly, first descriptive information is drawn out；Again using the 20th frame as background, replicate 20 frame numbers 21~
40 frames, mode same as above draws second descriptive information；The 3rd descriptive information is finally drawn by background of the 40th frame；
Finally, 60 frame assembly codings are the video segment that 10~20 frame per second is played by more than.
S240, is linked to generate target electronic photograph album to video segment.
Specifically, all videos fragment is chained up, i.e., the video segment generated each photo is linked at one
Rise, composition sequence photograph video, and the video after link mixes background music, and head are mixed in video, so as to complete
The making of target electronic photograph album.
The electron album generation method of the embodiment of the present invention, synthesis used multinomial technology to complete, i.e., using image at
Reason technology pre-processed to photo, to photo generate descriptive words, using video coding and decoding technology using CNN+NLP technologies
Electron album Video Composition is carried out, target electronic photograph album is finally given, in the generating process of electron album, without requiring user
Possessing image procossing knowledge also can intelligently complete the making of electron album, while manual manufacture cost is saved, enrich
Photograph album content, it is interesting to improve photograph album, has expanded imagination space, and improve Consumer's Experience.
Corresponding with the electron album generation method that above-mentioned several embodiments are provided, a kind of embodiment of the invention is also provided
A kind of electron album generating means, because electron album generating means provided in an embodiment of the present invention are carried with above-mentioned several embodiments
The electron album generation method of confession is corresponding, therefore is also applied for this implementation in the embodiment of aforementioned electronic album creating method
The electron album generating means that example is provided, are not described in detail in the present embodiment.Fig. 7 is according to an embodiment of the invention
The structural representation of electron album generating means.As shown in fig. 7, the electron album generating means can include：Descriptive information
Generation module 700 and electron album generation module 800.
Specifically, descriptive information generation module 700 is used to carry out content analysis to photo, generates the descriptive letter of photo
Breath.
It is appreciated that the content analysis to photo for convenience, can enter before content analysis is carried out to photo to photo
Row pretreatment.For example, original photo can have situations such as oversized, length-width ratio is inconsistent, it is necessary to carry out size change over processing,
In the case of keeping the ratio of width to height immovable in photo, the width and height dimensions of suitable target electronic photograph album are converted to.
After being pre-processed to photo, descriptive information generation module 700 can carry out interior to pretreated photo
Hold analysis, and based on the descriptive information of analysis result generation photo.Specifically, CNN methods can be used, loading training in advance is good
Model comparison piece in object analyzed, obtain the object type in photo, afterwards, can be according to object type from previously generating
Alternative descriptive information set in select the descriptive information of the photo.Concrete implementation mode can be found in subsequent embodiment
Description.
It should be noted that the language form of descriptive information is not limited to Chinese, English, German, Russian etc. can also be；
The file format of descriptive information can be text and/or voice etc..That is, arranging in pairs or groups to the description on target electronic photograph album
Property information can be Chinese text, can also be other spoken and written languages, also can also be Chinese speech etc., specific presentation mode can
Determine, be not especially limited herein according to the actual requirements.
Electron album generation module 800 is used to carry out Video Composition according to photo and descriptive information, to generate target electricity
Sub- photograph album.More specifically, electron album generation module 800 can be entered according to photo and descriptive information by video coding technique
The Video Composition of row electron album, finally gives target electronic photograph album.Concrete implementation process can be found in retouching for subsequent embodiment
State.
The electron album generating means of the embodiment of the present invention, can carry out content by descriptive information generation module to photo
Analysis, generates the descriptive information of photo, and electron album generation module carries out Video Composition according to photo and descriptive information, with
Generate target electronic photograph album.I.e. the content based on photograph image is analyzed, and is provided automatically for the photo based on analysis result
Appropriate descriptive information, so, in the generating process of electron album, also can without requiring that user possesses image procossing knowledge
The making of electron album is intelligently completed, while manual manufacture cost is saved, photograph album content is enriched, improves photograph album
Interest, has expanded imagination space, and improve Consumer's Experience.
Fig. 8 is the structural representation of the electron album generating means according to a specific embodiment of the invention.Such as Fig. 8 institutes
Show, the electron album generating means can include：Descriptive information generation module 700 and electron album generation module 800.Its
In, the descriptive information generation module 700 may include：Recognize submodule 710 and generation submodule 720.
Wherein, identification submodule 710 is used to photo is identified according to default disaggregated model, to obtain in photo
Object type.
Generating submodule 720 is used for according to object type, chooses and shines from the alternative descriptive information set previously generated
The descriptive information of piece.As a kind of example, as shown in figure 9, the generation submodule 720 may include：First acquisition unit 721,
Second acquisition unit 722, computing unit 723, the 3rd acquiring unit 724 and generation unit 725.
Wherein, first acquisition unit 721 is used for the text term vector for obtaining object type.
Second acquisition unit 722 is used for acquisition high frequency corresponding with object type and describes word list, wherein, high frequency adjective
There are multiple high frequency adjectives in list.
Computing unit 723 is used to calculate between the text term vector of object type and the adjectival term vector of each high frequency
First similarity.
3rd acquiring unit 724 is used to obtain the target adjective that the first similarity is more than predetermined threshold value.
Generation unit 725 is used to choose descriptive information from alternative descriptive information set according to target adjective.
It should be noted that in an embodiment of the present invention, the alternative descriptive information set can be previously generated.
As a kind of example, as shown in Figure 10, the electron album generating means may also include：Module 900 is anticipated, for pre- Mr.
Into alternative descriptive information set.Wherein, as shown in Figure 10, this is anticipated module 900 and can included：First acquisition submodule
910th, calculating sub module 920, the second acquisition submodule 930 and generation submodule 940.
Wherein, the first acquisition submodule 910 is used to obtain the descriptive words list in high frequency word list, wherein, description
Property word list include multiple adjective samples and multiple adverbial word samples.
Calculating sub module 920 is used to obtain each adjective sample and the vector representation of each adverbial word sample respectively, calculates
The second similarity between each adjective sample and each adverbial word sample.
Second acquisition submodule 930 is used to be directed to each adjective sample, obtains the second phase with each adjective sample
Like the degree N number of adverbial word sample of highest, wherein, N is positive integer.
Generation submodule 940 is standby to generate for each adjective sample and corresponding N number of adverbial word sample to be combined
Select descriptive information set.
As a kind of example, as shown in figure 11, the electron album generation module 800 can include：The He of transform subblock 810
Generate submodule 820.Wherein, transform subblock 810 is used to the descriptive information of photo and photo being converted to video segment.It is raw
It is used to link video segment to generate target electronic photograph album into submodule 820.
The electron album generating means of the embodiment of the present invention, synthesis used multinomial technology to complete, i.e., using image at
Reason technology pre-processed to photo, to photo generate descriptive words, using video coding and decoding technology using CNN+NLP technologies
Electron album Video Composition is carried out, target electronic photograph album is finally given, in the generating process of electron album, without requiring user
Possessing image procossing knowledge also can intelligently complete the making of electron album, while manual manufacture cost is saved, enrich
Photograph album content, it is interesting to improve photograph album, has expanded imagination space, and improve Consumer's Experience.
In the description of the invention, it is to be understood that term " first ", " second " are only used for describing purpose, and can not
It is interpreted as indicating or implies relative importance or the implicit quantity for indicating indicated technical characteristic.Thus, define " the
One ", at least one this feature can be expressed or be implicitly included to the feature of " second ".In the description of the invention, " multiple "
It is meant that at least two, such as two, three etc., unless otherwise specifically defined.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means to combine specific features, structure, material or the spy that the embodiment or example are described
Point is contained at least one embodiment of the present invention or example.In this manual, to the schematic representation of above-mentioned term not
Identical embodiment or example must be directed to.Moreover, specific features, structure, material or the feature of description can be with office
Combined in an appropriate manner in one or more embodiments or example.In addition, in the case of not conflicting, the skill of this area
Art personnel can be tied the not be the same as Example or the feature of example and non-be the same as Example or example described in this specification
Close and combine.
Any process described otherwise above or method description are construed as in flow chart or herein, represent to include
Module, fragment or the portion of the code of one or more executable instructions for the step of realizing specific logical function or process
Point, and the scope of the preferred embodiment of the present invention includes other realization, wherein can not be by shown or discussion suitable
Sequence, including according to involved function by it is basic simultaneously in the way of or in the opposite order, carry out perform function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Represent in flow charts or logic and/or step described otherwise above herein, for example, being considered use
In the order list for the executable instruction for realizing logic function, it may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress for combining these instruction execution systems, device or equipment and using
Put.The more specifically example (non-exhaustive list) of computer-readable medium includes following：Electricity with one or more wirings
Connecting portion (electronic installation), portable computer diskette box (magnetic device), random access memory (RAM), read-only storage
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device, and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, can even is that can be in the paper of printing described program thereon or other are suitable for computer-readable medium
Medium, because can then enter edlin, interpretation or if necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.Above-mentioned
In embodiment, the software that multiple steps or method can in memory and by suitable instruction execution system be performed with storage
Or firmware is realized.If, and in another embodiment, can be with well known in the art for example, realized with hardware
Any one of row technology or their combination are realized：With the logic gates for realizing logic function to data-signal
Discrete logic, the application specific integrated circuit with suitable combinational logic gate circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that to realize all or part of step that above-described embodiment method is carried
Rapid to can be by program to instruct the hardware of correlation to complete, described program can be stored in a kind of computer-readable storage medium
In matter, the program upon execution, including one or a combination set of the step of embodiment of the method.
In addition, each functional unit in each embodiment of the invention can be integrated in a processing module, can also
That unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould
Block can both be realized in the form of hardware, it would however also be possible to employ the form of software function module is realized.The integrated module is such as
Fruit is realized using in the form of software function module and as independent production marketing or in use, can also be stored in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only storage, disk or CD etc..Although having been shown and retouching above
Embodiments of the invention are stated, it is to be understood that above-described embodiment is exemplary, it is impossible to be interpreted as the limit to the present invention
System, one of ordinary skill in the art can be changed to above-described embodiment, change, replace and become within the scope of the invention
Type.
Claims (10)
1. a kind of electron album generation method, it is characterised in that comprise the following steps：
Content analysis is carried out to photo, the descriptive information of the photo is generated；And
Video Composition is carried out according to the photo and the descriptive information, to generate target electronic photograph album.
2. electron album generation method as claimed in claim 1, it is characterised in that described that content analysis, life are carried out to photo
Into the descriptive information of the photo, including：
The photo is identified according to default disaggregated model, to obtain the object type in the photo；
According to the object type, the descriptive letter of the photo is chosen from the alternative descriptive information set previously generated
Breath.
3. electron album generation method as claimed in claim 2, it is characterised in that described according to the object type, from pre-
The descriptive information of the photo is chosen in the alternative descriptive information set first generated, including：
Obtain the text term vector of the object type；
Obtain high frequency corresponding with the object type and describe word list, wherein, the high frequency is described in word list with multiple
High frequency adjective；
Calculate the first similarity between the text term vector of the object type and the adjectival term vector of each high frequency；
Obtain the target adjective that the first similarity is more than predetermined threshold value；
The descriptive information is chosen from the alternative descriptive information set according to the target adjective.
4. electron album generation method as claimed in claim 2 or claim 3, it is characterised in that previously generate institute by following steps
State alternative descriptive information set：
The descriptive words list in high frequency word list is obtained, wherein, the descriptive words list includes multiple adjective samples
Originally with multiple adverbial word samples；
The vector representation of each adjective sample and each adverbial word sample is obtained respectively, calculates each described adjective sample and institute
State the second similarity between each adverbial word sample；
For each adjective sample, the N number of adverbial word sample of the second similarity highest with each adjective sample is obtained, its
In, N is positive integer；
Each adjective sample is combined with corresponding N number of adverbial word sample to generate the alternative descriptive letter
Breath set.
5. electron album generation method as claimed in claim 1, it is characterised in that described according to the photo and the description
Property information carry out Video Composition, to generate target electronic photograph album, including：
The descriptive information of the photo and the photo is converted into video segment；
The video segment is linked to generate the target electronic photograph album.
6. a kind of electron album generating means, it is characterised in that including：
Descriptive information generation module, for carrying out content analysis to photo, generates the descriptive information of the photo；And
Electron album generation module, for carrying out Video Composition according to the photo and the descriptive information, to generate target
Electron album.
7. electron album generating means as claimed in claim 6, it is characterised in that the descriptive information generation module bag
Include：
Submodule is recognized, for the photo to be identified according to default disaggregated model, to obtain pair in the photo
As classification；
Submodule is generated, for according to the object type, choosing described from the alternative descriptive information set previously generated
The descriptive information of photo.
8. electron album generating means as claimed in claim 7, it is characterised in that the generation submodule includes：
First acquisition unit, the text term vector for obtaining the object type；
Second acquisition unit, word list is described for obtaining high frequency corresponding with the object type, wherein, the high frequency is described
There are multiple high frequency adjectives in word list；
Computing unit, for calculating between the text term vector of the object type and the adjectival term vector of each high frequency
One similarity；
3rd acquiring unit, the target adjective of predetermined threshold value is more than for obtaining the first similarity；
Generation unit, for choosing the descriptive letter from the alternative descriptive information set according to the target adjective
Breath.
9. electron album generating means as claimed in claim 7 or 8, it is characterised in that also include：
Module is anticipated, for previously generating the alternative descriptive information set；
Wherein, the module of anticipating includes：
First acquisition submodule, for obtaining the descriptive words list in high frequency word list, wherein, the descriptive words row
Table includes multiple adjective samples and multiple adverbial word samples；
Calculating sub module, the vector representation for obtaining each adjective sample and each adverbial word sample respectively calculates described each
The second similarity between individual adjective sample and each described adverbial word sample；
Second acquisition submodule, for for each adjective sample, obtaining similar to the second of each adjective sample
The N number of adverbial word sample of highest is spent, wherein, N is positive integer；
Submodule is generated, for each adjective sample and corresponding N number of adverbial word sample to be combined to generate
The alternative descriptive information set.
10. electron album generating means as claimed in claim 6, it is characterised in that the electron album generation module includes：
Transform subblock, for the descriptive information of the photo and the photo to be converted into video segment；
Submodule is generated, for being linked to the video segment to generate the target electronic photograph album.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710138877.0A CN107122393B (en) | 2017-03-09 | 2017-03-09 | electronic album generating method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710138877.0A CN107122393B (en) | 2017-03-09 | 2017-03-09 | electronic album generating method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107122393A true CN107122393A (en) | 2017-09-01 |
| CN107122393B CN107122393B (en) | 2019-12-10 |
Family
ID=59717949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710138877.0A Active CN107122393B (en) | 2017-03-09 | 2017-03-09 | electronic album generating method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107122393B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948730A (en) * | 2017-10-30 | 2018-04-20 | 百度在线网络技术（北京）有限公司 | Method, apparatus, equipment and storage medium based on picture generation video |
| CN108446728A (en) * | 2018-03-14 | 2018-08-24 | 深圳乐信软件技术有限公司 | User personality extracting method, device, terminal and storage medium |
| CN108495059A (en) * | 2018-03-07 | 2018-09-04 | 迷你高（北京）科技有限公司 | album creating method and system |
| CN111209423A (en) * | 2020-01-07 | 2020-05-29 | 腾讯科技（深圳）有限公司 | Image management method and device based on electronic album and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783881A (en) * | 2010-03-05 | 2010-07-21 | 公安部第三研究所 | Intelligent web camera with video structural description function |
| CN103930901A (en) * | 2011-11-17 | 2014-07-16 | 微软公司 | Automatic tag generation based on image content |
| CN104331437A (en) * | 2014-10-24 | 2015-02-04 | 百度在线网络技术（北京）有限公司 | Method and device for generating picture description information |
| CN105488156A (en) * | 2015-11-30 | 2016-04-13 | 广州一刻影像科技有限公司 | Method for automatically selecting electronic album template and generating electronic album |
| US20160253833A1 (en) * | 2015-02-26 | 2016-09-01 | Grace Lew | System and method for photo album journaling |
| CN106021364A (en) * | 2016-05-10 | 2016-10-12 | 百度在线网络技术（北京）有限公司 | Method and device for establishing picture search correlation prediction model, and picture search method and device |
| CN106202183A (en) * | 2016-06-24 | 2016-12-07 | 四川长虹电器股份有限公司 | A kind of realize automatically joining literary composition and dub in background music the method for generation phonic photograph album/e-book |
-
2017
- 2017-03-09 CN CN201710138877.0A patent/CN107122393B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783881A (en) * | 2010-03-05 | 2010-07-21 | 公安部第三研究所 | Intelligent web camera with video structural description function |
| CN103930901A (en) * | 2011-11-17 | 2014-07-16 | 微软公司 | Automatic tag generation based on image content |
| CN104331437A (en) * | 2014-10-24 | 2015-02-04 | 百度在线网络技术（北京）有限公司 | Method and device for generating picture description information |
| US20160253833A1 (en) * | 2015-02-26 | 2016-09-01 | Grace Lew | System and method for photo album journaling |
| CN105488156A (en) * | 2015-11-30 | 2016-04-13 | 广州一刻影像科技有限公司 | Method for automatically selecting electronic album template and generating electronic album |
| CN106021364A (en) * | 2016-05-10 | 2016-10-12 | 百度在线网络技术（北京）有限公司 | Method and device for establishing picture search correlation prediction model, and picture search method and device |
| CN106202183A (en) * | 2016-06-24 | 2016-12-07 | 四川长虹电器股份有限公司 | A kind of realize automatically joining literary composition and dub in background music the method for generation phonic photograph album/e-book |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948730A (en) * | 2017-10-30 | 2018-04-20 | 百度在线网络技术（北京）有限公司 | Method, apparatus, equipment and storage medium based on picture generation video |
| CN107948730B (en) * | 2017-10-30 | 2020-11-20 | 百度在线网络技术（北京）有限公司 | Method, device and equipment for generating video based on picture and storage medium |
| CN108495059A (en) * | 2018-03-07 | 2018-09-04 | 迷你高（北京）科技有限公司 | album creating method and system |
| CN108446728A (en) * | 2018-03-14 | 2018-08-24 | 深圳乐信软件技术有限公司 | User personality extracting method, device, terminal and storage medium |
| CN111209423A (en) * | 2020-01-07 | 2020-05-29 | 腾讯科技（深圳）有限公司 | Image management method and device based on electronic album and storage medium |
| CN111209423B (en) * | 2020-01-07 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Image management method and device based on electronic album and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107122393B (en) | 2019-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108986186B (en) | Method and system for converting text into video | |
| KR101855597B1 (en) | Systems and methods for video paragraph captioning using hierarchical recurrent neural networks | |
| CN107644085A (en) | The generation method and device of competitive sports news | |
| CN110364146A (en) | Audio recognition method, device, speech recognition apparatus and storage medium | |
| CN107122393A (en) | Electron album generation method and device | |
| CN110457466A (en) | Generate method, computer readable storage medium and the terminal device of interview report | |
| CN114339450B (en) | Video comment generation method, system, device and storage medium | |
| CN111626049A (en) | Title correction method and device for multimedia information, electronic equipment and storage medium | |
| CN116227474B (en) | Method and device for generating countermeasure text, storage medium and electronic equipment | |
| WO2023197749A1 (en) | Background music insertion time point determining method and apparatus, device, and storage medium | |
| CN110457424A (en) | Generate method, computer readable storage medium and the terminal device of interview report | |
| CN116975615A (en) | Task prediction method and device based on video multi-mode information | |
| CN111681678A (en) | Method, system, device and storage medium for automatically generating sound effect and matching video | |
| CN117216535A (en) | Training method, device, equipment and medium for recommended text generation model | |
| CN114677402A (en) | Poster text layout, poster generation method and related device | |
| CN114049501A (en) | Image description generation method, system, medium and device fusing cluster search | |
| CN110297906A (en) | Generate method, computer readable storage medium and the terminal device of interview report | |
| Yang et al. | Automatic comic generation with stylistic multi-page layouts and emotion-driven text balloon generation | |
| Mei et al. | Vision and language: from visual perception to content creation | |
| US20230326369A1 (en) | Method and apparatus for generating sign language video, computer device, and storage medium | |
| CN115169472A (en) | Music matching method and device for multimedia data and computer equipment | |
| CN114443916A (en) | Supply and demand matching method and system for test data | |
| CN113407766A (en) | Visual animation display method and related equipment | |
| Snoek | The authoring metaphor to machine understanding of multimedia | |
| Fendri et al. | A novel approach for soccer video summarization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |