CN107066578B - 3D picture intelligent recommendation method based on deep learning and transfer learning - Google Patents
3D picture intelligent recommendation method based on deep learning and transfer learning Download PDFInfo
- Publication number
- CN107066578B CN107066578B CN201710239554.0A CN201710239554A CN107066578B CN 107066578 B CN107066578 B CN 107066578B CN 201710239554 A CN201710239554 A CN 201710239554A CN 107066578 B CN107066578 B CN 107066578B
- Authority
- CN
- China
- Prior art keywords
- picture
- scene
- learning
- data set
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000013135 deep learning Methods 0.000 title claims abstract description 18
- 238000013526 transfer learning Methods 0.000 title claims abstract description 16
- 238000013461 design Methods 0.000 claims abstract description 26
- 230000008447 perception Effects 0.000 claims abstract description 7
- 238000013508 migration Methods 0.000 claims abstract description 5
- 230000005012 migration Effects 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 abstract 1
- 238000010422 painting Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
Abstract
The invention discloses a 3D picture intelligent recommendation method based on deep learning and transfer learning, which comprises the steps of firstly, obtaining a universal scene image classifier based on deep learning by utilizing a large open sample scene image data set; then, migration learning is carried out through a 3D scene picture data set collected by a user, and a universal image classifier is converted into a 3D scene space classifier; then, an information fingerprint library of the 3D picture design scheme recommendation gallery is constructed through a Hash perception algorithm; and finally, matching and screening the pictures of the scene shot by the user and the 3D picture design scheme library to obtain a matching candidate subset, calculating the information fingerprint Hamming distance between each picture in the subset and the picture of the user, and intelligently recommending the 3D picture with the minimum distance to the user. The method is based on deep learning and transfer learning, realizes the design of the 3D picture in a specific environment and a specific space structure, and shortens the design period of the 3D picture.
Description
Technical Field
The invention relates to the field of machine learning and image processing, in particular to a 3D picture intelligent recommendation method based on deep learning and transfer learning.
Background
In recent years, naked-eye 3D pictures are more and more concerned and sought after with special artistic expression, super-strong visual impact and extremely interesting interactivity, cover multiple fields such as decoration, advertisement, exhibition, home furnishing and the like, and have wide development prospects. The 3D picture is a special artistic form using the principles of anti-perspective and optical illusion, and needs to be created skillfully using the fusion of environment and spatial structure. Therefore, 3D picture design according to specific environment and space structure is time-consuming and labor-consuming, and certain requirements are also made on experience and level of painters. In the traditional mode, a painter needs to skillfully use various perspective relations in the painting and have very strong sense of space to design a good 3D painting. Some new painters cannot independently design the 3D painting due to inexperience, and the new painters become a vacancy in the 3D painting industry.
Disclosure of Invention
The invention provides a 3D picture intelligent recommendation method based on deep learning and transfer learning, which overcomes the defects of the 3D picture intelligent recommendation method based on deep learning and transfer learning in the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows: A3D picture intelligent recommendation method based on deep learning and transfer learning comprises the following steps:
s1: constructing an image classifier RCLF based on the public image dataset; the public image data set BS is an MIT computational science Places205 public data set, and an image classifier RCLF for identifying each scene in the Places205 public data set is obtained after a convolution model increment-ResNet is selected to train on the Places205 public data set;
s2: 3D scene space migration learning based on the image classifier RCLF obtained in step S1;
s21, keeping the parameters of other layers of the image classifier RCLF except the softmax layer unchanged, increasing the parameter learning rate of the softmax layer by 2 times, and decreasing the learning rate of other parameters of the full-connection layer by half;
s22, training a full connection layer by using a 3D scene data set SS collected by a user to capture 3D scene space information, and finally obtaining a 4-classification 3D scene space classifier DCLF;
s3: constructing an information fingerprint of a 3D picture design scheme recommendation gallery;
s31, collecting a large number of images of the 3D picture design scheme as a result data set to be recommended;
s32, constructing an information fingerprint FPS for each collected 3D picture image;
s33, solving the Hash perception fingerprint of each image in the step S32 by using a Hash perception algorithm;
s4: outputting a 3D picture design scheme based on a real scene;
s41, the user shoots a picture Ps of the real scene as a matching service request;
s42, transmitting the picture in the step S41 to a classifier DCLF for identification to obtain an intelligent identification result scene of the user scene;
s43, calculating the corresponding information fingerprint fp of the user scene picture Ps by adopting the method of the step S3;
s44, searching a sample subset CSS belonging to scene with scene category being scene in a 3D picture design scheme recommendation gallery;
s45, calculating a Hamming distance between the information fingerprint cp of each picture in the sample subset CSS and the information fingerprint fp of the user scene picture Ps, wherein the 3D drawing design scheme picture with the minimum Hamming distance in the sample subset CSS is the recommendation result Ds.
Further, the Places205 dataset is over two hundred and fifty thousand pictures of scenes collected by MIT computer science and artificial intelligence laboratories, for a total of 205 scene categories.
Further, the 3D painting scene data set SS includes 3D paintings of different colors, wall surfaces, ground surfaces, wall and ground surfaces, and recessed corner surfaces.
Further, the information fingerprint FPS construction process is to set the pixel value of each channel in the picture central area a to zero.
Compared with the prior art, the invention has the following beneficial effects: the method realizes intelligent design of the 3D picture based on deep learning and transfer learning, and avoids the problems of long design period and difficult design caused by the problems of personal inspiration of painters, drawing experience and the like in any environment space required by users; and the method saves training time on one hand, further shortens the period of 3D picture design, and greatly expands the application field of deep learning on the other hand.
The invention is further explained in detail with the accompanying drawings and the embodiments; however, the 3D picture intelligent recommendation method based on deep learning and transfer learning of the present invention is not limited to the embodiment.
Drawings
FIG. 1 is a block diagram of a transfer learning process of the present invention;
FIG. 2 is a block diagram of the recommendation process of the present invention.
Detailed Description
In an embodiment, please refer to fig. 1 and fig. 2, a 3D picture intelligent recommendation method based on deep learning and transfer learning of the present invention includes the following steps:
s1: constructing an image classifier based on a public image data set, and naming the image classifier as RCLF; the public image data set is named as BS, the MIT computing science Places205 public data set, and after a convolution model increment-ResNet is selected to be trained on the Places205 public data set, an image classifier RCLF for identifying each scene in the Places205 public data set is obtained; the recognizer RCLF can classify 205 scenes in the Places205 public data set according to factors such as color, structure, environment and the like;
s2: 3D scene space migration learning based on the image classifier RCLF obtained in step S1;
s21, keeping the parameters of other layers of the image classifier RCLF except the softmax layer unchanged, increasing the parameter learning rate of the softmax layer by 2 times, and decreasing the learning rate of other parameters of a full connection layer by half, wherein the full connection layer is a connection medium connecting the last layer, namely the softmax layer, with the penultimate layer;
s22, training a full connection layer by using a 3D scene data set SS collected by a user to capture 3D scene space information, and finally obtaining a 4-classification 3D scene space classifier named as DCLF; the spatial classifier DCLF further classifies the 205 scenes classified by the image classifier RCLF in step S1 into 4 classes;
s3: constructing an information fingerprint of a 3D picture design scheme recommendation gallery;
s31, collecting a large number of images of the 3D picture design scheme as a result data set to be recommended;
s32, constructing an information fingerprint library for each collected 3D picture image, and naming the fingerprint library as an FPS;
s33, solving the Hash perception fingerprint of each image in the step S32 by using a Hash perception algorithm;
s4: outputting a 3D picture design scheme based on a real scene;
s41, a user shoots a picture of a real scene, and the picture is marked as Ps and serves as a matching service request;
s42, transmitting the picture Ps in the step S41 to a space classifier DCLF for identification to obtain an intelligent identification result of the user scene, wherein the result is expressed by scene, and the scene comprises the characteristics of the user scene such as color, composition and the like;
s43, calculating the corresponding information fingerprint of the user scene picture Ps by adopting the method of the step S3, and recording as fp;
s44, searching pictures which belong to scene categories and are closest to feature categories in scene in a 3D picture design scheme recommendation gallery to serve as a sample subset, and recording the sample subset as CSS;
s45, recording the information fingerprint of each picture in the sample subset CSS as cp and the information fingerprint fp of the user scene picture Ps to obtain the Hamming distance, and recording the 3D drawing design scheme picture with the minimum Hamming distance in the sample subset CSS as a recommendation result as Ds.
In this embodiment, the Places205 data set is more than two hundred and fifty thousand scene pictures collected by MIT computer science and artificial intelligence laboratories, and there are 205 scene categories in total; the 3D picture scene data set SS comprises 3D pictures of different colors, wall surfaces, ground surfaces, wall surfaces and concave wall corner surfaces; the FPS construction process of the information fingerprint database comprises the step of setting the pixel value of each channel in the central area a of the picture to zero, wherein the central area a of the picture is 50% of the length and width of the original picture.
The above embodiments are only used to further illustrate the intelligent 3D image recommendation method based on deep learning and migration learning, but the present invention is not limited to the embodiments, and any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical spirit of the present invention fall within the scope of the technical solution of the present invention.
Claims (4)
1. A3D picture intelligent recommendation method based on deep learning and transfer learning is characterized by comprising the following steps:
s1: constructing an image classifier RCLF based on the public image dataset; the public image data set is an MIT computational science Places205 public data set, and an image classifier RCLF for identifying each scene in the Places205 public data set is obtained after a convolution model increment-ResNet is selected to train on the Places205 public data set;
s2: 3D scene space migration learning based on the image classifier RCLF obtained in step S1;
s21, keeping the parameters of other layers of the image classifier RCLF except for the softmax layer and the full connection layer unchanged, increasing the parameter learning rate of the softmax layer by 2 times, and decreasing the learning rate of other parameters of the full connection layer by half;
s22, training a full connection layer by using a 3D scene data set SS collected by a user to capture 3D scene space information, and finally obtaining a 4-classification 3D scene space classifier DCLF;
s3: constructing an information fingerprint of a 3D picture design scheme recommendation gallery;
s31, collecting 3D picture design scheme images as a result data set to be recommended;
s32, constructing an information fingerprint FPS for each collected 3D picture image;
s33, solving the Hash perception fingerprint of each image in the step S32 by using a Hash perception algorithm;
s4: outputting a 3D picture design scheme based on a real scene;
s41, the user shoots a picture Ps of the real scene as a matching service request;
s42, transmitting the picture in the step S41 to a classifier DCLF for identification to obtain an intelligent identification result scene of the user scene;
s43, calculating the corresponding information fingerprint fp of the user scene picture Ps by adopting the method of the step S3;
s44, searching a sample subset CSS belonging to scene with scene category being scene in a 3D picture design scheme recommendation gallery;
s45, calculating a Hamming distance between the information fingerprint cp of each picture in the sample subset CSS and the information fingerprint fp of the user scene picture Ps, wherein the 3D drawing design scheme picture with the minimum Hamming distance in the sample subset CSS is the recommendation result Ds.
2. The 3D picture intelligent recommendation method based on deep learning and transfer learning according to claim 1, characterized in that: the Places205 public dataset is over two hundred and fifty thousand pictures of scenes collected by MIT computer science and artificial intelligence laboratories, for a total of 205 scene categories.
3. The 3D picture intelligent recommendation method based on deep learning and transfer learning according to claim 1, characterized in that: the 3D picture scene data set SS comprises 3D pictures of different colors, wall surfaces, ground surfaces, wall surfaces and concave wall corner surfaces.
4. The 3D picture intelligent recommendation method based on deep learning and transfer learning according to claim 1, characterized in that: the information fingerprint FPS construction process is to set the pixel value of each channel in the picture central area a to zero, and the picture central area a is 50% of the length and width of the original picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710239554.0A CN107066578B (en) | 2017-04-13 | 2017-04-13 | 3D picture intelligent recommendation method based on deep learning and transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710239554.0A CN107066578B (en) | 2017-04-13 | 2017-04-13 | 3D picture intelligent recommendation method based on deep learning and transfer learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107066578A CN107066578A (en) | 2017-08-18 |
| CN107066578B true CN107066578B (en) | 2020-06-26 |
Family
ID=59599586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710239554.0A Expired - Fee Related CN107066578B (en) | 2017-04-13 | 2017-04-13 | 3D picture intelligent recommendation method based on deep learning and transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107066578B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107526843A (en) * | 2017-09-22 | 2017-12-29 | 四川长虹电器股份有限公司 | A kind of KBS |
| CN111368120B (en) * | 2020-05-28 | 2020-09-15 | 广东博智林机器人有限公司 | Target fingerprint database construction method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105512289A (en) * | 2015-12-07 | 2016-04-20 | 郑州金惠计算机系统工程有限公司 | Image retrieval method based on deep learning and Hash |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150324690A1 (en) * | 2014-05-08 | 2015-11-12 | Microsoft Corporation | Deep Learning Training System |
-
2017
- 2017-04-13 CN CN201710239554.0A patent/CN107066578B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105512289A (en) * | 2015-12-07 | 2016-04-20 | 郑州金惠计算机系统工程有限公司 | Image retrieval method based on deep learning and Hash |
Non-Patent Citations (2)
| Title |
|---|
| Representations and techniques for 3D object recognition and scene interpretation;Derek Hoiem 等;《Synthesis Lectures on Artificial Intelligence and Machine Learning》;20111230;第5卷(第5期);第1-169页 * |
| 面向智能交互的图像识别技术综述与展望;蒋树强 等;《计算机研究与发展》;20151116;第53卷(第1期);第113-122页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107066578A (en) | 2017-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105472434B (en) | It is implanted into method and system of the content into video display | |
| CN105359190B (en) | According to single image estimating depth | |
| Kelly et al. | FrankenGAN: guided detail synthesis for building mass-models using style-synchonized GANs | |
| US11928957B2 (en) | Audiovisual secondary haptic signal reconstruction method based on cloud-edge collaboration | |
| CN106920243A (en) | The ceramic material part method for sequence image segmentation of improved full convolutional neural networks | |
| CN111027421A (en) | Graph-based direct-push type semi-supervised pedestrian re-identification method | |
| JP2024502495A (en) | Generating augmented reality pre-rendering using template images | |
| CN113395578A (en) | Method, device and equipment for extracting video theme text and storage medium | |
| CN107066578B (en) | 3D picture intelligent recommendation method based on deep learning and transfer learning | |
| CN108446605B (en) | Double interbehavior recognition methods under complex background | |
| Shen et al. | The influence of artificial intelligence on art design in the digital age | |
| US20240038274A1 (en) | 3d media elements in 2d video | |
| Pan et al. | Multi‐source information art painting fusion interactive 3d dynamic scene virtual reality technology application research | |
| Ben‐Zvi et al. | Line‐drawing video stylization | |
| Peng | Machines' perception of space | |
| Wang et al. | Wuju opera cultural creative products and research on visual image under VR technology | |
| CN113039561A (en) | Aligning sequences by generating encoded representations of data items | |
| Kim et al. | Data-driven single image depth estimation using weighted median statistics | |
| Sellami et al. | Video semantic segmentation using deep multi-view representation learning | |
| Stekovic et al. | S4-net: Geometry-consistent semi-supervised semantic segmentation | |
| Tan et al. | RGBD-FG: A large-scale RGB-D dataset for fine-grained categorization | |
| CN105488785B (en) | A kind of depth drawing generating method of view-based access control model dictionary | |
| CN105335990A (en) | Human portrait material image generation method and apparatus | |
| He et al. | Application of High-Resolution Face Recognition and EDF Image Reconstruction in English Classroom Teaching | |
| CN110837770B (en) | Video behavior self-adaptive segmentation method and device based on multiple Gaussian models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200626 |