CN107862241A - A kind of clothes fashion method for digging and visually-perceptible system based on star's identification - Google Patents

A kind of clothes fashion method for digging and visually-perceptible system based on star's identification Download PDF

Info

Publication number
CN107862241A
CN107862241A CN201710851088.1A CN201710851088A CN107862241A CN 107862241 A CN107862241 A CN 107862241A CN 201710851088 A CN201710851088 A CN 201710851088A CN 107862241 A CN107862241 A CN 107862241A
Authority
CN
China
Prior art keywords
clothes
face
image
detection
star
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710851088.1A
Other languages
Chinese (zh)
Other versions
CN107862241B (en
Inventor
张海军
姬玉柱
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Graduate School Harbin Institute of Technology
Original Assignee
Shenzhen Graduate School Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Graduate School Harbin Institute of Technology filed Critical Shenzhen Graduate School Harbin Institute of Technology
Publication of CN107862241A publication Critical patent/CN107862241A/en
Application granted granted Critical
Publication of CN107862241B publication Critical patent/CN107862241B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0269—Targeted advertisements based on user profile or attribute
    • G06Q30/0271—Personalized advertisement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions

Abstract

The present invention relates to a kind of clothes fashion method for digging based on star's identification and visually-perceptible system, methods described to include：A, human testing step；B, posture selection step, the posture quality of the human body to detecting are screened；C, face detection and star's authentication step, face detection is carried out to the human region screened, alignd using critical point detection technology to detecting face, and face characteristic is extracted using depth convolutional network, face verification is carried out with benchmark star face database；D, clothes detection steps, clothes detection is carried out to the human region verified by star；E, clothes image de-redundancy step to be retrieved, the clothes image to be retrieved of redundancy is removed using clustering algorithm；F, with the retrieval of money clothes image and recommendation step, searched for using image retrieval algorithm in clothes database with money or similar clothes and recommend user.Present invention is mainly used for video ads recommendation, improves attraction of the advertisement to user.

Description

A kind of clothes fashion method for digging and visually-perceptible system based on star's identification
Technical field
The invention belongs to video content analysis and field is perceived, more particularly to a kind of star of internet video perception of content Clothes fashion method for digging and system.
Background technology
In recent years, as the rapid popularization of conventional internet and mobile Internet, internet economy have obtained fast development. Wherein, the growth year by year of Online Video business and flow, huge business opportunity is brought for video ads business.It is reported that claim：" from Over 2014, global Online Video advertising market is grown continuously and fast, and domestic relevant market then remains up to more than 40% Growth rate, and traditional TV media advertisement has begun to face the historic turning of atrophy, with the migration of user, operator is just The budget of traditional tv, gradually it is transferred in Online Video business.”
According to investigation, video-frequency advertisement put-on method the most frequently used at present mainly uses the dispensing similar to traditional tv advertisement Mode, before being mainly included in video playback, play in playing processing completion time used for them be inserted directly into one section of advertisement.However, these advertisements It is mostly unrelated with video content.This mode more considers meet the needs of advertiser, does not consider the use of video user Experience at family.Advertising message is forced to be pushed to video user so this dispensing mode is more like.Because tediously long advertisement plays Reason, this aspect such as duration, ad content and user's request mismatch more aggravate to destroy body of the user when watching video Test so that user has to introduce the advertisements such as Adblock or Adblock Plus shielding plug-in unit, and then advertisement putting is imitated Fruit, and the commodity accordingly promoted bring negative impact.On the other hand, it is this force promote mode also with internet video The theory of service business consumer uppermost fails to agree.Therefore, how according to the real experiences and demand of viewing user, guidance quality has been carried out Products Show, become advertisement pushing business problem urgently to be resolved hurrily.
In recent years, fast development of the deep learning in visual fields such as image, videos so that a series of to be based on deep learning Application commercially emerge in multitude.Wherein, depth convolutional neural networks (CNNs) are known in object identification, object detection, face Not, the immense success obtained in the Conventional visual task such as image retrieval so that deep learning model conversion is into actual scene It is applied in order to possible.
The content of the invention
The present invention relies on existing depth learning technology, the particularly machine learning algorithm such as depth convolutional neural networks, will The advertisement of dispensing is closely chained up with video content, reduces the invasion to video user, improves the product promotion of advertiser Efficiency, and make that video ads are more accurate, recommend spectators naturally.Based on this thinking, the present invention proposes a kind of based on bright The clothes fashion method for digging and visually-perceptible system of star identification.
The present invention is achieved through the following technical solutions：
A kind of clothes fashion method for digging based on star's identification, comprises the following steps：
A, human body detection step, human body detection is carried out to original video frame image using depth convolutional neural networks, and led to Cross the positional information that detection obtains and cut out human region；
B, posture selection step, posture is carried out to the human body detected using the grader of depth convolutional neural networks training The judgement of quality, filter out the human body for the posture being in；
C, face detection and star's authentication step, face is carried out to the human body after screening using face detection technology Detection；Face is alignd using face critical point detection technology；The feature being drawn into using depth convolutional neural networks with Star's face database carries out authentication；
D, clothes detection steps, using the clothes detector of depth convolutional neural networks training to the star people after checking Body region carries out clothes detection, and the positional information obtained according to detection cuts out star Garment region, structure retrieval Candidate Set；
E, clothes image de-redundancy step to be retrieved, the depth convolutional neural networks of clothes image are detected by extracting Feature, the similar clothes detected is clustered using clustering algorithm, and the sample by choosing cluster centre is used as most Whole image to be retrieved, redundant image is removed, reduce retrieval number.
F, it is similar or with the retrieval of money clothes image and recommendation step, using retrieving the convolutional neural networks feature of image in structure The extraction of image retrieval and similar image is carried out in the clothes image data set built, and user is recommended into similar image list.
Further, the step A comprises the following steps：
A1, the detection data collection comprising a variety of objects is built by way of manually demarcating, many of object includes people Body, data message should include the positional information in the classification information and residing picture of object；
A2, projected depth convolutional neural networks, model training is carried out using existing object detection data set, in image Various objects detected；
A3, according to result of detection, the human region of high confidence level will be judged to detecting successfully by system, and trigger step B.
Further, the step B comprises the following steps：
B1, the human body image for being carried out using artificial and semi-artificial mode fine or not posture are selected, and are demarcated positive and negative sample, are built Human posture selects data set；
B2, the positive and negative sample of demarcation standard：The positive human body of the upright model of whole body is positive sample；And half body, clothes distort The larger grade human body image of amplitude will be negative sample；
B3, projected depth convolutional neural networks structure two-value grader, select data set to enter using the human posture of demarcation Row model training, and the judgement of human posture's quality is carried out to detecting human region；
B4, according to model result of determination, the human region with preferable posture will be screened out, and trigger step C.
Further, the step C comprises the following steps：
C1, the performers and clerks' list provided according to video, star's face subset of structure viewing video is as authenticated The benchmark face storehouse of journey, and face critical point detection technology is utilized, affine transformation matrix is built, carries out face alignment, and utilize The depth convolutional neural networks trained extract face characteristic；
C2, the human region with preferable posture is detected using face detection technology, the face detected will be carried out equally Critical point detection and face alignment step；
C3, same, the feature of the face detected using the extraction of identical depth convolutional neural networks, and and benchmark face Face characteristic in storehouse is compared, and calculates distance；When distance is less than certain threshold value, you can be verified as what is occurred in the video Star；
C4, according to the result, the human region with preferable posture can detect face, and face belongs to the collection of drama Star's will triggering step D.
Further, the step D comprises the following steps：
D1, the detection data collection comprising multiclass clothes is built by way of manually demarcating.Data message should include clothes Classification information and residing picture in positional information；
D2, projected depth convolutional neural networks, model training is carried out using the clothes detection data collection demarcated, to step Star's human region obtained by C carries out clothes detection；
D3, according to result of detection, the Garment region of high confidence level will be judged to detecting successfully by system, and trigger step E.
Further, the step E comprises the following steps：
E1, the clothes result of detection in step D collected；
E2, using clustering algorithm similar clothes image is gathered in cluster one by one；
E3, according to cluster result, judge that the cluster that number of samples is 1 is abnormal clusters, these clusters are most rejected at last；The sample of cluster This number is more than 1, and its cluster centre sample will be by as inquiry sample triggering step F.
Further, the step F comprises the following steps：
F1, the categorized data set for including multiclass clothes can be built by way of artificial and semi-artificial demarcation.
F2, projected depth convolutional neural networks, utilize the clothes detection data collection training depth convolutional Neural net demarcated Network image encrypting algorithm；
F3, utilize the depth image retrieval model extraction candidate data collection retrieval character trained and clothes image to be checked Feature；
F4, by calculating distance, the result retrieved is ranked up, obtains final retrieval image list, and show Retrieval result recommends user.
On the other hand, present invention also offers a kind of clothes fashion based on star's identification to excavate visually-perceptible system, bag Include：
Human body detection module, for obtaining human region and coordinate position in video frame images；
Human posture's selecting module, for the human body for judging the quality of posture residing for detected human body and having filtered out Region；
Star's authentication module, for detecting the face in human region, and the face to detecting carries out star's identity Checking；
Clothes detecting module, for obtaining clothes subregion and the positional information that star's human region is worn, and cut The Garment region for cutting out is as Candidate Set to be retrieved；
Clothes retrieves image clustering and de-redundancy module, for the clothes image cut out to be clustered, removes phase As redundancy clothes image, reduce system queries number；
Clothes picture retrieval and similar clothes recommending module, for utilizing depth convolutional neural networks feature in clothes data Clothes list similar to inquiry or with money is searched in storehouse, result is presented, and recommend user.
Further, the human body detection module includes：
Frame of video extracts submodule：For extracting the video frame images of human body to be detected；
Submodule is trained, is instructed for carrying out the model of depth convolutional neural networks of more type objects using data with existing collection Practice；
Submodule is tested, for carrying out carrying out object detection to original video frame image, and provides result of detection；
Output sub-module, for determining whether to detect human body, and trigger human posture's selecting module.
Further, human posture's selecting module includes：
Submodule is trained, for selecting the mould of data set progress depth convolutional neural networks using the human posture built Type training, wherein, whole body is upright, positive human body is positive sample；And the human body images such as half body, clothes twisting magnitude be larger will be Negative sample；
Submodule is tested, for carrying out the judgement of human posture's quality to the human region detected；
Output sub-module, for obtaining judging the human region of preferably posture, and trigger star's checking.
Further, star's checking includes：
Face detection submodule, for face location and the coordinate letter for obtaining being determined in positive sample human posture region Breath, and cut；
Face critical point detection submodule, for obtaining the key point positional information of the face detected；
Face alignment submodule, according to standard faces key point information and the face key point information detected, calculate imitative Transformation matrix is penetrated, and face alignment is carried out by affine transformation；
Face characteristic extracting sub-module, for extracting depth convolutional neural networks feature；
Face verification submodule, for verifying whether the face detected is star's face, so as to carry out authentication.
Further, the clothes detecting module includes：
Submodule is trained, for carrying out the clothes based on depth convolutional neural networks of multiclass clothes using data with existing collection The training of detection model；
Submodule is tested, for carrying out to carrying out clothes detection by the human region of checking, and provides result of detection；
Output sub-module, for exporting clothes result of detection and positional information.
Further, the clothes retrieval image clustering includes with de-redundancy module：
Feature extraction submodule, for the clothing detected using the depth convolutional neural networks model extraction trained Take feature；
Submodule is clustered, for using the feature extracted, being clustered using clustering algorithm by image is retrieved；
De-redundancy submodule, for simplifying the number of cluster in cluster result, and cluster central sample is as query image.Specifically , number of samples is rejected for 1 abnormal clusters that will be considered in cluster, and number of samples takes cluster center sample more than or equal to 2 in cluster This is query sample；
Further, the clothes picture retrieval includes to similar clothes recommending module：
Model training submodule, for carrying out based on depth convolution god for the categorized data set using existing multiclass clothes The training of image encrypting algorithm through network；
Feature extraction submodule, for extracting candidate data collection retrieval character using the depth image retrieval model trained With clothes image feature to be checked；
Image retrieval submodule, for utilizing the feature extracted, by calculating distance, candidate search collection is extracted, and it is right The result retrieved is ranked up, and obtains final retrieval image list；
Retrieval result shows submodule, for showing the phase Sihe retrieved with money clothes image list, to recommend use Family.
The beneficial effects of the invention are as follows：Clothes fashion method for digging and visually-perceptible system of the present invention based on star's identification System, can realize that advertisement is associated with Celebrity Fashion, excavate potential huge in the fashion effect and bean vermicelli effect of star's dress Business opportunity.On the one hand, star is the core of video, if it is possible to realize that video commercial product recommending is associated with star, while with The story of a play or opera of video is related, and such advertisement can leave more deep impression to the bean vermicelli of star.On the other hand, known based on star Other clothes fashion method for digging can substantially reduce the number of Recommendations by setting harsh detection and search criteria, And the orientation for having catered to hommization is recommended, the theory that this future ads is recommended.In addition, for spectators, what advertisement was recommended Hommization can increase the interest of spectators, and advertisement is combined with video scene, spectators can be made to go deep into wherein, will greatly be improved Advertisement delivery effect.By analyzing given video, the present invention recommends method, energy using the similar clothes based on star's identification The main star that enough automatic minings go out in video, detects the fashion dress of star in video, and selects closest with the object It is identical or recommend bean vermicelli with money clothes, lift the effective degree of recommendation.
Brief description of the drawings
Fig. 1 is the flow chart of the clothes fashion method for digging based on star's identification of the present invention；
Fig. 2 is the block diagram of the clothes fashion visually-perceptible system based on star's identification of the present invention；
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the present invention, not For limiting the present invention.
The design inspiration of the present invention derives from the celebrity's appeal in current Online Video.Specifically, the clothing of star is total It is ahead of fashions, the spectators of viewing and star's associated video are usually the bean vermicelli of star.Therefore, the fashion of star is worn in one Determine to attract large quantities of beans vermicelli to go to retrieve same or similar same money commodity in degree.So as to excavate star dress fashion effect and Star's bean vermicelli effect has potential huge commercial opportunities.
Accompanying drawing 1 shows the flow chart of the clothes fashion method for digging provided by the invention based on star's identification, and it is described in detail It is as follows：
Step S1：Human body detection step.This step needs to carry out by the data set of the existing more object detections demarcated The training of object detection model.Data set can select PASCAL VOC object detection Challenge data. Based on convolutional neural networks (Convolutional Neural Network, abbreviation CNN) object detection model, no matter in mould In type performance and accuracy, fast development is obtained for.It is representative wherein with region convolutional neural networks (Region CNN) Work turns into the main flow algorithm in the field.More representational work has Fast RCNN and Faster RCNN, Yi Jijie The SSD (Single Shot MultiBox Detector) of new neural network structure is closed, the work such as YOLO is also constantly being brushed The performance of new object detection.In this step, the present invention utilizes above-mentioned newest research results, by considering the expansible of system Property, train multiclass object detection model.In real system operation, only responded when detecting human body.Specific stream Journey is as follows：(1) frame of video is extracted first, can be given up head by setting start frame (for example, 1000 frames), be reduced detection time Number.Then by way of taking anchor-frame in every time interval (such as per second take 1 frame), the extraction of redundant frame is reduced；(2) will take out The frame of video taken feeds in above-mentioned object detection model and carries out human body detection；(3) people with high confidence to detecting Body region is cut.
Step S2：Posture selects step.Different from pose estimation task, posture selection step is to filter out in good appearance The human body of gesture, subsequently to carry out clothes detection.In system realization, still consider to use depth convolutional neural networks, design And train the two-value grader judged posture quality.Due to different from the task of association area, it is necessary to build oneself Posture selection data set.The mode that data set structure is combined using artificial and semi-artificial mode is built.First, by climbing Take Amazon, Taobao, street to clap model's image of the websites such as displaying, and a large amount of positive samples are obtained by human body detection model, then, The negative sample suitable by extracting the result composition of human body detection in result of detection and video image therein again.Manually The setting of the establishing criteria of screening and demarcation refers to：(1) the positive human body of the upright model of whole body is positive sample；(2) lean to one side, partly The human body images such as body, clothes twisting magnitude be larger will be demarcated as negative sample.By designing convolutional neural networks, using building Data set carry out two classifier trainings that posture quality judges, and finally give model.Using the model trained, to step Human body detection result obtained by S1 is judged.So as to which the human region with preferable posture will be retained, other postures Human body will be removed.Here, " preferable posture " or " good posture " is meant that the human posture for being judged as positive sample, accordingly , " worse posture " or " bad posture " is meant that the human posture for being judged as negative sample.
Step S3：Face detection and star's authentication step.Carry out face detection and star's authentication step it It is preceding to get out star's face database, it is necessary to realize.Star's face database can crawl from the film information such as IMDB website.Building process Need to obtain star's face using face detection technology, obtain 5 key point positions using face key point Detection Techniques afterwards Information, including right and left eyes position, nose position and two corners of the mouth positions, face is used as by artificial established standardses face and alignd The correction benchmark of (Face alignment), will detect face and standard faces are alignd, and can obtain final star Face database.In systems in practice, with American series《Life huge explosion》Exemplified by, pass through the acute cast first from IMDB websites 7 star's pictures corresponding to crawling.By face detection and aligned portions, the facial image of every star is obtained.By artificial Data cleansing, 7 width facial images are filtered out for every star.To improve the accuracy of checking, the face detected every time is entered 7x7 checking of row.Specific verification process, the depth convolutional neural networks for extracting 2048 dimensions of 49 width standard faces first are special Sign, the most benchmark face feature database, as 49x2048 matrix.Equally, 2048 dimension depth are equally extracted to the face detected Convolutional neural networks feature, and carry out 49 Cosine distances with the feature in benchmark face storehouse and calculate.Cosine range formulas It is as follows：
Assuming that detect N number of face, then final distance matrix is N*49.By given threshold, by distance matrix two Value, that is, 1 is put more than the element of threshold value, the element less than threshold value is set to 0.The two values matrix being verified.To ensure checking Accuracy, for validation matrix, every star in complete zero row representative sample and benchmark face is dissimilar therefore corresponding Detection face sample will be considered as non-star's face.The row for occurring multiple 1 is discussed, if multiple 1 appearance are same Among the section of star so that validator judges that the face is wherein star then by authentication, output result.And Occur multiple 1, and multiple 1 to appear on the checking section of multidigit star, represent the face and attempt to mislead the judgement of validator, Therefore authentication can not be passed through.Accordingly, be able to will be obtained into one by the face of authentication, and corresponding human region The filtering of step, and it is fed to next step.
Step S4：Clothes detection steps.For the human region with good posture by star's authentication, at this The detection of clothes will be carried out in step.Firstly the need of building clothes detection data collection by way of manually marking.Data acquisition Mode can be crawled by web crawlers from the electric business websites such as Amazon, Taobao.The position data of clothes in the picture Obtained by way of manually demarcating.Afterwards by reading data file, build similar to PASCALVOC object detection data sets Database format to facilitate the training of clothes detection model.Object detection model based on convolutional neural networks is in step It is discussed in S1, is not repeating here.In this step, it is necessary to model be moved in clothes detection data domain, therewith Preceding object detection step has no essential distinction.It is worth noting that, this step is not entangled with the accuracy of clothes identification, but Focus is concentrated in clothes detection, that is, clothes positional information (bounding box), and cut out accordingly in image Garment region, and build the query set of clothes retrieval.
Step S5：Clothes image de-redundancy step to be retrieved.Although we are selected by S1 human body detections step, S2 postures Step, S3 star's authentication steps, S4 clothes detection steps carry out layering to the sample to be retrieved.But due to video The variation of image in a short time is very small, and by observed result, discovery still has a large amount of identical clothes to be detected, and Retrieval module will be fed to and carry out clothes retrieval.But bulk redundancy image being present in these inquiries, this can undoubtedly give system band Carry out the waste of substantial amounts of computing resource.Therefore, in secondary step, the present invention is devised using clustering algorithm to simplify inquiry (query) number.Specifically：By realizing the depth convolutional neural networks image encrypting algorithm that trains, reciprocal the is extracted The feature of two full linking layers, the feature representation as image.These features are fed into density peaks clustering algorithm afterwards Clustered in (Density Peaks Clustering Algorithm, abbreviation DPCA).By setting hyper parameter, arest neighbors Sample percentage (percentage), and regularization parameter ρ and δ threshold range can obtain the individual of rational clustering cluster Number.Finally, it is 1 abnormal clusters by Rejection of samples number, and retains the central sample of cluster of the number of samples more than or equal to 2, from And construct final query set.
Step S6：It is similar or with money clothes image retrieval and recommendation step.The key issue of clothes retrieval is to retrieve speed Degree and retrieval precision.In recent years, depth convolutional neural networks field of image search development, for large-scale image retrieve in reality Application in the system of border provides credible scheme.In this step, we still need to build more massive clothes image number According to collection, including more clothes images and more clothes classifications.By investigation, newest achievement in research, depth two-value are used Hash network carries out the study of image feature representation.Specifically, entered by the clothes data of the extensive multiple types of structure The training of row depth two-value Hash network.Feeding model will be inquired about afterwards obtains the Hash coding of image and the feature of full linking layer Expression.Data are got out by extracting retrieval data set and inquiring about this two category feature of data set.By calculating inquiry and retrieval The Hamming distances (Hamming Distance) of data set obtain candidate search outcome pool, are then examined by calculating inquiry with candidate Euclidean distance in rope outcome pool between the feature representation of the full linking layer of sample, and can be obtained by ascending sort final Retrieval result list (closest comes before list).By indexing the sample in list, in abstract image database Original image and image network address (URL), and finally show retrieval result.
The clothes fashion based on star's identification that accompanying drawing 2 show the present invention excavates visually-perceptible system, including：Human body is visited Module is surveyed, for obtaining human region and coordinate position in original video frame picture；Human posture's selecting module, for sentencing The quality of posture residing for fixed detected human body and the human region filtered out；Face detection and star's authentication module, are used In detecting the face in human region, and the face to detecting carries out the checking of star's identity；Clothes detecting module, for The clothes subregion and positional information worn to star's human region, and the Garment region cut out is as time to be retrieved Selected works；Clothes retrieves image clustering and de-redundancy module, for the clothes image cut out to be clustered, removes similar Redundancy clothes image, reduce system queries number；Module is presented with result in clothes picture retrieval, for utilizing depth convolutional Neural Network characterization searches for the clothes list to match with inquiry in clothes searching database, and result is presented.
The main contributions of the present invention have at following 2 points：(1) present invention proposes a kind of clothes fashion based on star's identification Method for digging and visually-perceptible system, and design specific implementation flow.By relying on field of machine vision, depth convolutional Neural The immense success that network obtains on the visual tasks such as object identification, object detection, recognition of face, image retrieval, realizes star Wear, in the fashion effect and bean vermicelli effect of excavation star dress potential huge commercial opportunities associated with electric business product clothes with money.(2) The present invention is identified as dominating with star, by human body detection, human posture's selection, face detection and star's authentication, Yi Jili The key technologies such as query image de-redundancy are combined with clustering algorithm, clothes detection and the retrieval of similar clothes are carried out successively Harshly screen.On the one hand reduce the waste of computing resource, on the other hand reduce the frequency of video push advertisement Recommendations Rate, improve the experience during viewing video of user.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention All any modification, equivalent and improvement made within refreshing and principle etc., should be included in the scope of the protection.

Claims (14)

  1. A kind of 1. clothes fashion method for digging based on star's identification, it is characterised in that：It the described method comprises the following steps：
    A, human body detection step：Human body detection is carried out to original video frame image using depth convolutional neural networks, and passes through spy The positional information measured cuts out human region；
    B, posture selection step, posture quality is carried out to the human body detected using the grader of depth convolutional neural networks training Judgement, filter out the human body of the posture being in；
    C, face detection and star's authentication step：The spy of face is carried out to the human body after screening using face detection technology Survey；Face is alignd using face critical point detection technology；Using the feature that depth convolutional neural networks are drawn into it is bright Star face database carries out authentication；
    D, clothes detection steps：Using the clothes detector of depth convolutional neural networks training to star's human body area after checking Domain carries out clothes detection, and the positional information obtained according to detection cuts out star Garment region, structure retrieval Candidate Set；
    E, clothes image de-redundancy step to be retrieved：By the spy for extracting the depth convolutional neural networks for detecting clothes image Sign, is clustered, and be used as finally by choosing the sample of cluster centre using clustering algorithm to the similar clothes detected Image to be retrieved, remove redundant image, reduce retrieval number；
    F, it is similar or with money clothes image retrieval and recommendation step：Using retrieving the convolutional neural networks feature of image in structure The extraction of image retrieval and similar image is carried out in clothes image data set, and user is recommended into similar image list.
  2. 2. according to the method for claim 1, it is characterised in that：The step A comprises the following steps：
    A1, the detection data collection comprising a variety of objects is built by way of manually demarcating, wherein a variety of objects include people Body, data message should include the positional information in the classification information and residing picture of object；
    A2, projected depth convolutional neural networks, model training is carried out using existing object detection data set, to each in image Kind object is detected；
    A3, according to result of detection, the human region of high confidence level will be judged to detecting successfully by system, and trigger step B.
  3. 3. according to the method for claim 1, it is characterised in that：The step B comprises the following steps：
    B1, the human body image for being carried out using artificial and semi-artificial mode fine or not posture are selected, and are demarcated positive and negative sample, are built human body Posture selects data set；
    B2, the positive and negative sample of demarcation standard：Whole body is upright, positive human body is positive sample；And half body, clothes twisting magnitude are larger To be negative sample Deng human body image；
    B3, projected depth convolutional neural networks structure two-value grader, select data set to carry out mould using the human posture of demarcation Type training, and the judgement of human posture's quality is carried out to detecting human region；
    B4, according to model result of determination, the human region with preferable posture will be screened out, and trigger step C.
  4. 4. according to the method for claim 1, it is characterised in that：The step C comprises the following steps：
    C1, the performers and clerks' list provided according to video, star's face subset of structure viewing video is as verification process Benchmark face storehouse, and face critical point detection technology is utilized, affine transformation matrix is built, carries out face alignment, and utilize training Good depth convolutional neural networks extract face characteristic；
    C2, the human region with preferable posture is detected using face detection technology, the face detected will equally carry out key Point detection and face alignment step；
    It is C3, same, extract the feature of the face detected using identical depth convolutional neural networks, and with benchmark face storehouse Face characteristic be compared, calculate distance；When distance is less than certain threshold value, you can be verified as occurring in the video bright Star；
    C4, according to the result, the human region with preferable posture can detect face, and face belongs to collection of drama star Will triggering step D.
  5. 5. according to the method for claim 1, it is characterised in that：The step D comprises the following steps：
    D1, the detection data collection comprising multiclass clothes is built by way of manually demarcating.Data message should include the class of clothes Positional information in other information and residing picture；
    D2, projected depth convolutional neural networks, model training is carried out using the clothes detection data collection demarcated, in step C Resulting star's human region carries out clothes detection；
    D3, according to result of detection, the Garment region of high confidence level will be judged to detecting successfully by system, and trigger step E.
  6. 6. according to the method for claim 1, it is characterised in that：The step E comprises the following steps：
    E1, the clothes result of detection in step D collected；
    E2, using clustering algorithm similar clothes image is gathered in cluster one by one；
    E3, according to cluster result, judge that the cluster that number of samples is 1 is abnormal clusters, these clusters are most rejected at last；The sample of cluster Number is more than 1, and its cluster centre sample will be by as inquiry sample triggering step F.
  7. 7. according to the method for claim 1, it is characterised in that：The step F comprises the following steps：
    F1, the categorized data set for including multiclass clothes is built by way of artificial and semi-artificial demarcation；
    F2, projected depth convolutional neural networks, utilize the clothes detection data collection training depth convolutional neural networks figure demarcated As retrieval model；
    F3, candidate data collection retrieval character and clothes image to be checked spy are extracted using the depth image retrieval model trained Sign；
    F4, by calculating distance, the result retrieved is ranked up, obtains final retrieval image list, and show retrieval As a result user is recommended.
  8. 8. a kind of clothes fashion based on star's identification excavates visually-perceptible system, it is characterised in that：The system includes：
    Human body detection module, for obtaining human region and coordinate position in video frame images；
    Human posture's selecting module, for the human body area for judging the quality of posture residing for detected human body and having filtered out Domain；
    Star's authentication module, for detecting the face in human region, and the face to detecting carries out the checking of star's identity；
    Clothes detecting module, for obtaining clothes subregion and the positional information that star's human region is worn, and cut out The Garment region come is as Candidate Set to be retrieved；
    Clothes retrieves image clustering and de-redundancy module, for the clothes image cut out to be clustered, removes similar Redundancy clothes image, reduce system queries number；
    Clothes picture retrieval and similar clothes recommending module, for utilizing depth convolutional neural networks feature in clothes database Search clothes list similar to inquiry or with money, is presented result, and recommend user.
  9. 9. system according to claim 8, it is characterised in that：The human body detection module includes：
    Frame of video extracts submodule：For extracting the video frame images of human body to be detected；
    Submodule is trained, the model training of the depth convolutional neural networks for carrying out more type objects using data with existing collection；
    Submodule is tested, for carrying out carrying out object detection to original video frame image, and provides result of detection；
    Output sub-module, for determining whether to detect human body, and trigger human posture's selecting module.
  10. 10. system according to claim 8, it is characterised in that：Human posture's selecting module includes：
    Submodule is trained, is instructed for selecting data set to carry out the model of depth convolutional neural networks using the human posture built Practice, wherein whole body is upright, positive human body is positive sample；And the human body images such as half body, clothes twisting magnitude be larger will be negative sample Example；
    Submodule is tested, for carrying out the judgement of human posture's quality to the human region detected；
    Output sub-module, for obtaining judging the human region of preferably posture, and trigger star's checking.
  11. 11. system according to claim 8, it is characterised in that：Star's checking includes：
    Face detection submodule, for the face location and coordinate information for obtaining being determined in positive sample human posture region, and Cutting；
    Face critical point detection submodule, for obtaining the key point positional information of the face detected；
    Face alignment submodule, according to standard faces key point information and the face key point information detected, calculates affine change Matrix is changed, and face alignment is carried out by affine transformation；
    Face characteristic extracting sub-module, for extracting depth convolutional neural networks feature；
    Face verification submodule, for verifying whether the face detected is star's face, so as to carry out authentication.
  12. 12. system according to claim 8, it is characterised in that the clothes detecting module includes：
    Submodule is trained, the clothes based on depth convolutional neural networks for being carried out multiclass clothes using data with existing collection is detected The training of model；
    Submodule is tested, for carrying out to carrying out clothes detection by the human region of checking, and provides result of detection；
    Output sub-module, for exporting clothes result of detection and positional information.
  13. 13. system according to claim 8, it is characterised in that the clothes retrieval image clustering and de-redundancy module bag Include：
    Feature extraction submodule, the clothes for being detected using the depth convolutional neural networks model extraction trained are special Sign；
    Submodule is clustered, for using the feature extracted, being clustered using clustering algorithm by image is retrieved；
    De-redundancy submodule, for simplifying the number of cluster in cluster result, and cluster central sample is used as query image sample； Specifically, number of samples is rejected for 1 abnormal clusters that will be considered in cluster, number of samples taking in cluster more than or equal to 2 in cluster Heart sample is query image sample.
  14. 14. system according to claim 8, it is characterised in that the clothes picture retrieval and similar clothes recommending module Including：
    Model training submodule, for the categorized data set using existing multiclass clothes carry out be based on depth convolutional Neural net The training of the image encrypting algorithm of network；
    Feature extraction submodule, for extracting candidate data collection retrieval character using the depth image retrieval model trained and treating Inquire about clothes image feature；
    Image retrieval submodule, for utilizing the feature extracted, by calculating distance, candidate search collection is extracted, and to retrieval To result be ranked up, obtain final retrieval image list；
    Retrieval result recommends submodule, for showing the phase Sihe retrieved with money clothes image list, to recommend user.
CN201710851088.1A 2017-06-06 2017-09-20 Clothes fashion mining method based on star recognition and visual perception system Active CN107862241B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2017104392388 2017-06-06
CN201710439238 2017-06-06

Publications (2)

Publication Number Publication Date
CN107862241A true CN107862241A (en) 2018-03-30
CN107862241B CN107862241B (en) 2021-07-20

Family

ID=61699648

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710851088.1A Active CN107862241B (en) 2017-06-06 2017-09-20 Clothes fashion mining method based on star recognition and visual perception system

Country Status (1)

Country Link
CN (1) CN107862241B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108852268A (en) * 2018-04-23 2018-11-23 浙江大学 A kind of digestive endoscopy image abnormal characteristic real-time mark system and method
CN109063784A (en) * 2018-08-23 2018-12-21 深圳码隆科技有限公司 A kind of character costume image data screening technique and its device
CN109558821A (en) * 2018-11-21 2019-04-02 哈尔滨工业大学（深圳） The clothes article number calculating method of particular persons in a kind of video
CN109684909A (en) * 2018-10-11 2019-04-26 武汉工程大学 A kind of unmanned plane target key point real-time location method, system and storage medium
CN110188701A (en) * 2019-05-31 2019-08-30 上海媒智科技有限公司 Dress ornament recognition methods, system and terminal based on the prediction of human body key node
CN110348529A (en) * 2019-07-16 2019-10-18 韶关市启之信息技术有限公司 A kind of intelligent clothes Trend of fashion prediction technique and system
CN110413823A (en) * 2019-06-19 2019-11-05 腾讯科技（深圳）有限公司 Garment image method for pushing and relevant apparatus

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881642A (en) * 2015-05-22 2015-09-02 海信集团有限公司 Method and device for content pushing, and equipment
CN105184280A (en) * 2015-10-10 2015-12-23 东方网力科技股份有限公司 Human body identity identification method and apparatus
CN105913275A (en) * 2016-03-25 2016-08-31 哈尔滨工业大学深圳研究生院 Clothes advertisement putting method and system based on video leading role identification

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881642A (en) * 2015-05-22 2015-09-02 海信集团有限公司 Method and device for content pushing, and equipment
CN105184280A (en) * 2015-10-10 2015-12-23 东方网力科技股份有限公司 Human body identity identification method and apparatus
CN105913275A (en) * 2016-03-25 2016-08-31 哈尔滨工业大学深圳研究生院 Clothes advertisement putting method and system based on video leading role identification

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
安玉松: "基于视频主角识别的服装广告投放方法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
张海军 等: "用小波分析进行基于遗传算法的2DMBV图像分割", 《自动化技术与应用》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108852268A (en) * 2018-04-23 2018-11-23 浙江大学 A kind of digestive endoscopy image abnormal characteristic real-time mark system and method
CN109063784A (en) * 2018-08-23 2018-12-21 深圳码隆科技有限公司 A kind of character costume image data screening technique and its device
WO2020037940A1 (en) * 2018-08-23 2020-02-27 深圳码隆科技有限公司 Method and device for screening image data of character clothing
CN109684909A (en) * 2018-10-11 2019-04-26 武汉工程大学 A kind of unmanned plane target key point real-time location method, system and storage medium
CN109684909B (en) * 2018-10-11 2023-06-09 武汉工程大学 Real-time positioning method, system and storage medium for target essential points of unmanned aerial vehicle
CN109558821A (en) * 2018-11-21 2019-04-02 哈尔滨工业大学（深圳） The clothes article number calculating method of particular persons in a kind of video
CN109558821B (en) * 2018-11-21 2021-10-22 哈尔滨工业大学（深圳） Method for calculating number of clothes of specific character in video
CN110188701A (en) * 2019-05-31 2019-08-30 上海媒智科技有限公司 Dress ornament recognition methods, system and terminal based on the prediction of human body key node
CN110413823A (en) * 2019-06-19 2019-11-05 腾讯科技（深圳）有限公司 Garment image method for pushing and relevant apparatus
CN110348529A (en) * 2019-07-16 2019-10-18 韶关市启之信息技术有限公司 A kind of intelligent clothes Trend of fashion prediction technique and system
CN110348529B (en) * 2019-07-16 2021-10-22 上海惟也新文化科技有限公司 Intelligent clothes fashion style prediction method and system

Also Published As

Publication number Publication date
CN107862241B (en) 2021-07-20

Similar Documents

Publication Publication Date Title
CN107862241A (en) A kind of clothes fashion method for digging and visually-perceptible system based on star's identification
Diem et al. cBAD: ICDAR2017 competition on baseline detection
Matzen et al. Streetstyle: Exploring world-wide clothing styles from millions of photos
Yao et al. Highlight detection with pairwise deep ranking for first-person video summarization
Porzi et al. Predicting and understanding urban perception with convolutional neural networks
WO2020155939A1 (en) Image recognition method and device, storage medium and processor
CN103530540B (en) User identity attribute detection method based on man-machine interaction behavior characteristics
US9471849B2 (en) System and method for suspect search
CN102201061B (en) Intelligent safety monitoring system and method based on multilevel filtering face recognition
CN101162470B (en) Video frequency advertisement recognition method based on layered matching
Liciotti et al. Person re-identification dataset with rgb-d camera in a top-view configuration
US20160239711A1 (en) Visual Data Mining
CN103714181B (en) A kind of hierarchical particular persons search method
CN109558821A (en) The clothes article number calculating method of particular persons in a kind of video
CN106845357A (en) A kind of video human face detection and recognition methods based on multichannel network
CN105913275A (en) Clothes advertisement putting method and system based on video leading role identification
CN105808610A (en) Internet picture filtering method and device
CN103324677B (en) Hierarchical fast image global positioning system (GPS) position estimation method
CN107615298A (en) Face identification method and system
CN102117413A (en) Method for automatically filtering defective image based on multilayer feature
CN109919060A (en) A kind of identity card content identifying system and method based on characteristic matching
Chang et al. Fashion world map: Understanding cities through streetwear fashion
CN104156413A (en) Trademark density based personalized trademark matching recognition method
CN109903053A (en) A kind of anti-fraud method carrying out Activity recognition based on sensing data
CN109272440A (en) A kind of reduced graph generating method and system for combining text and picture material

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant