CN104850633B - A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component - Google Patents
A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component Download PDFInfo
- Publication number
- CN104850633B CN104850633B CN201510267873.3A CN201510267873A CN104850633B CN 104850633 B CN104850633 B CN 104850633B CN 201510267873 A CN201510267873 A CN 201510267873A CN 104850633 B CN104850633 B CN 104850633B
- Authority
- CN
- China
- Prior art keywords
- component
- sampled point
- image
- carrying
- carried out
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Abstract
The embodiment of the invention discloses a kind of three-dimensional model searching systems and method based on the segmentation of cartographical sketching component, wherein the system includes：Preprocessing module obtains gray-scale map for carrying out denoising to Freehandhand-drawing inquiry sketch, and binary conversion treatment is carried out to the gray-scale map, border extension processing, vacancy filling are handled, obtain treated image；Component mark module for carrying out equal interval sampling to treated image, and adds component tag to sampled point；Sampled point characteristic extracting module, each feature vectors for extracting sampled point；Component divides module, for being split model training according to each feature vectors of the sampled point after addition component tag；Similarity calculation is ranked up, and ranking results are returned to client with overall score sorting module for carrying out component local similarity calculating according to overall score.Implement the embodiment of the present invention, can so that the three-dimensional model search based on cartographical sketching is more precisely effective.
Description
Technical field
The present invention relates to computer image processing technology field more particularly to a kind of three divided based on cartographical sketching component
Dimension module searching system and method.
Background technology
In recent years, along with CAD, computer-aided manufacturing, virtual reality, three-dimensional animation and three-dimensional game
The fast development in the fields such as play, the threedimensional model quantity sharp increase on internet.However, threedimensional model is different from traditional figure
The multimedia messages such as piece, audio or video, itself contain many detailed information and are difficult to be come out with literal expression.
However, current method for searching three-dimension model is upper still not fully up to expectations in application.On the one hand, when user needs certain
When threedimensional model resource, often there is no ready-made model files on hand；On the other hand, quick with touch screen and electronic pen
Universal, the mode that user can easily pass cartographical sketching sketches the contours of the profile of model.The cartographical sketching of threedimensional model can regard
For the contour line for being from some visual angle projection view.Cartographical sketching can be simple outer contour, can also include Internal periphery
The detailed information of line.Since user's art activities of cartographical sketching are different, input equipment is different, describes the level of detail of model certainly
So also it is not quite similar.And the cartographical sketching of threedimensional model generally comprises overlapping, separation or inc component outline line, it is existing
Some correlative studys are typically based on the manual segmentation carried out to cartographical sketching or label, although these information specified by hand help
Cartographical sketching is analyzed in computer, but when it usually requires that user's cartographical sketching, follows certain rule constraint, this
The degrees of freedom of user's Freehandhand-drawing are limited to a certain extent, and requirement is proposed to the foundation of painting of user in other words.
Classify according to retrieval mode, the retrieval of current threedimensional model is broadly divided into two major classes, be respectively it is traditional based on
Retrieval (Text-based Retrieval) method and content-based retrieval (Content-based Retrieval) of text
Method.
(1) text based method for searching three-dimension model
Text based method for searching three-dimension model is to be presently the most universal retrieval mode based on keyword.This
It needs artificially to add to describe its keyword, such as the 3D model libraries of SketchUp to the threedimensional model in database
The 3D protein retrieval systems (3D of (3D Warehouse), the special model library of TurboSquid and TaiWan, China university
Protein Retrieval System) etc., some large-scale commercial model index platforms can be found on the net now, it
Be this kind of three-dimensional model search mode based on keyword mostly.
(2) method for searching three-dimension model based on content
Content-based retrieval method is the research hotspot of three-dimensional model search.It is the three-dimensional based on content as shown in Figure 1
The basic framework of model index, frame entirety are divided into offline part and online part.For offline part, each 3D models need
It is indicated with shape description symbols.In order to promote recall precision, index usually is established to each aspect of model descriptor in database.It is right
In online part, the input for carrying out query express can be divided mainly into two ways：One kind be to provide one it is similar with object module
Threedimensional model；Another kind is the sketch of Freehandhand-drawing object module.After calculating feature descriptor, by user search input data
Descriptor carries out similarity-rough set with aspect of model descriptor in database, and the sequence then successively decreased according to similarity size will be tied
Fruit returns, and is visually presented with to user.
Disadvantage existing in the prior art：
(1) text based method for searching three-dimension model
Traditional mode based on text key word can not be useful in the scene of three-dimensional model search well, master
Reason is wanted to have at 3 points：First, threedimensional model has complicated topological structure, shape feature, and type is various, itself contains very
More detailed information are difficult to be expressed clearly with several keywords.Second, add tagging for text key word to threedimensional model
Journey needs have been manually done, and threedimensional model quantity rapid growth on internet, the mode manually added are relatively complicated, workload
It is very big.Third, since different people is different to the understanding of miscellaneous threedimensional model, description its keyword expected also has
Larger difference, the label for being easy to cause search key and object module is inconsistent, and by hand plus keyword label mode by
It is limited to markup language type, is also not easy to carry out internationalization popularization.Be based on these reasons, only with simple keyword into
Row retrieval, success rate can be very low, the result that cannot be wanted when many.For example, user want retrieve certain given configuration and
The car of pattern is difficult to search accurate, satisfied result then relying solely on keyword.
(2) method for searching three-dimension model based on content
For the model index based on threedimensional model example, the disadvantage is that user is when initiating to retrieve, it is generally difficult to find
One most suitable model instance is used as input, if because if user has most suitable object module at hand, then
Also it need not just retrieved.
For the three-dimensional model search based on cartographical sketching, its shortcoming is that it usually not considers the knot of sketch entirety
Structure, it is based only on region locally to consider；Its another disadvantage is exactly more sensitive to the style of user's sketch, if
Contour line drafting stylistic differences of the user in part are larger, then the result difference that it is extracted will amplify, this will certainly be influenced
Final retrieval result.
Invention content
It is an object of the invention to overcome the deficiencies in the prior art, and the present invention provides one kind based on cartographical sketching component point
The three-dimensional model searching system and method cut can so that the three-dimensional model search based on cartographical sketching is more precisely effective.
To solve the above-mentioned problems, the present invention proposes a kind of three-dimensional model search system divided based on cartographical sketching component
System, the system comprises：
Preprocessing module inquires sketch for receiving Freehandhand-drawing, and carrying out denoising to Freehandhand-drawing inquiry sketch obtains ash
Figure is spent, and binary conversion treatment, border extension processing, vacancy filling processing are carried out to the gray-scale map, treated for acquisition
Image；
Component mark module obtains sampled point, and to described for carrying out equal interval sampling to treated the image
Sampled point adds component tag；
Sampled point characteristic extracting module, each feature vectors for extracting the sampled point；
Component divides module, for being split model according to each feature vectors of the sampled point after addition component tag
Training；
Similarity calculation and overall score sorting module, for carrying out component local shape factor and portion based on parted pattern
Part local similarity calculates, and carries out the extraction of view global characteristics and view overall situation similarity meter to treated the image
It calculates, is ranked up according to overall score, and ranking results are returned into client.
Preferably, the preprocessing module includes：
Sketch denoising unit obtains gray-scale map for carrying out denoising to Freehandhand-drawing inquiry sketch；
Binary conversion treatment unit, for carrying out binary conversion treatment to the gray-scale map；
Border extension processing unit, for being handled into line blank filling the image surrounding after binary conversion treatment；
Vacancy fills processing unit, for carrying out vacancy filling processing to blank filling treated image.
Preferably, the component mark module includes：
Contour line extraction unit, for carrying out contour line extraction to treated the image；
Sampling unit obtains sampled point for carrying out equal interval sampling to the image after Extracting contour；
Component marking unit, for adding component tag to the sampled point.
Preferably, the sampled point characteristic extracting module includes：
Unitary feature extraction unit, for carrying out unitary feature extraction to the sampled point after addition component tag；
Binary feature extraction unit, for carrying out binary feature extraction to the sampled point after addition component tag.
Preferably, the component segmentation module includes：
Parted pattern training unit, for being split according to each feature vectors of the sampled point after addition component tag
Model training；
Component cutting unit, for carrying out component segmentation to the sampled point after addition component tag according to parted pattern.
Correspondingly, the present invention also provides a kind of method for searching three-dimension model based on the segmentation of cartographical sketching component, the sides
Method includes：
It receives Freehandhand-drawing and inquires sketch, carrying out denoising to Freehandhand-drawing inquiry sketch obtains gray-scale map, and to the ash
Degree figure carries out binary conversion treatment, border extension processing, vacancy filling processing, obtains treated image；
Equal interval sampling is carried out to treated the image, obtains sampled point, and component mark is added to the sampled point
Label；
Extract each feature vectors of the sampled point；
It is split model training according to each feature vectors of the sampled point after addition component tag；
Based on parted pattern component local shape factor and component local similarity is carried out to calculate, and to the processing after
Image carry out the extraction of view global characteristics and view overall situation similarity calculation, be ranked up according to overall score, and sequence is tied
Fruit returns to client.
Preferably, it is described to the Freehandhand-drawing inquiry sketch carry out denoising obtain gray-scale map, and to the gray-scale map into
Row binary conversion treatment, border extension processing, vacancy filling processing, the step of obtaining treated image, including：
Denoising is carried out to Freehandhand-drawing inquiry sketch and obtains gray-scale map；
Binary conversion treatment is carried out to the gray-scale map；
Image surrounding after binary conversion treatment is handled into line blank filling；
Vacancy filling processing is carried out to blank filling treated image.
Preferably, described that equal interval sampling is carried out to treated the image, sampled point is obtained, and to the sampled point
The step of adding component tag, including：
Contour line extraction is carried out to treated the image；
Equal interval sampling is carried out to the image after Extracting contour, obtains sampled point；
Component tag is added to the sampled point.
Preferably, the step of each feature vectors of the extraction sampled point, including：
Unitary feature extraction is carried out to the sampled point after addition component tag；
Binary feature extraction is carried out to the sampled point after addition component tag.
Preferably, each feature vectors of the sampled point after the component tag according to addition are split model training
Step, including：
It is split model training according to each feature vectors of the sampled point after addition component tag；
Component segmentation is carried out to the sampled point after addition component tag according to parted pattern.
In embodiments of the present invention, the topology information between the geological information of comprehensive utilization cartographical sketching component, component
And the global information of whole picture view, and the mechanism provided with three-view diagram dynamic weight index, amplify important visual angle in overall score
It influences, so that the three-dimensional model search based on cartographical sketching is more precisely effective；In addition, can be individually used for sketch understanding,
In the application scenarios of the manual draws component segmentation such as sketch classification.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the basic framework schematic diagram of the three-dimensional model search based on content in the prior art；
Fig. 2 is that the structure composition for the three-dimensional model searching system of the embodiment of the present invention divided based on cartographical sketching component is shown
It is intended to；
Fig. 3 is the inter-process mistake for the three-dimensional model searching system of the embodiment of the present invention divided based on cartographical sketching component
Journey schematic diagram；
Fig. 4 is to add tagged effect diagram in the embodiment of the present invention；
Fig. 5 is the flow signal for the method for searching three-dimension model of the embodiment of the present invention divided based on cartographical sketching component
Figure.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without creative efforts
Embodiment shall fall within the protection scope of the present invention.
Fig. 2 is that the structure composition for the three-dimensional model searching system of the embodiment of the present invention divided based on cartographical sketching component is shown
It is intended to, as shown in Fig. 2, the system includes：
Preprocessing module 1 inquires sketch for receiving Freehandhand-drawing, and carrying out denoising to Freehandhand-drawing inquiry sketch obtains gray scale
Figure, and binary conversion treatment is carried out to gray-scale map, border extension processing, vacancy filling are handled, obtain treated image；
Component mark module 2 obtains sampled point for carrying out equal interval sampling to treated the image of preprocessing module 1,
And component tag is added to sampled point；
Sampled point characteristic extracting module 3, the various features of the sampled point obtained for extracting parts mark module 2 to
Amount；
Component divides module 4, for being split mould according to each feature vectors of the sampled point after addition component tag
Type training；
Similarity calculation and overall score sorting module 5, for based on parted pattern carry out component local shape factor and
Component local similarity calculates, and image carries out the extraction of view global characteristics and view overall situation similarity calculation to treated,
It is ranked up according to overall score, and ranking results is returned into client.
Fig. 3 is the inter-process mistake of the three-dimensional model searching system based on the segmentation of cartographical sketching component of the embodiment of the present invention
Journey is described in detail the system of the embodiment of the present invention with reference to Fig. 2, Fig. 3.
The cartographical sketching that user inputs in retrieval is different from the projection view of threedimensional model.Due to user by mouse,
Touch screen or stylus skeletonizing, cartographical sketching is often inaccurate, and contour line usually contains noise, it is therefore desirable to
Family cartographical sketching carries out necessary pretreatment, could carry out next processing routine.
Wherein, preprocessing module 1 includes：
Sketch denoising unit obtains gray-scale map for carrying out denoising to Freehandhand-drawing inquiry sketch；
Binary conversion treatment unit, for carrying out binary conversion treatment to gray-scale map；
Border extension processing unit, for being handled into line blank filling the image surrounding after binary conversion treatment；
Vacancy fills processing unit, for carrying out vacancy filling processing to blank filling treated image.
Sketch denoising：Following feature extraction can be had an impact since cartographical sketching end is not closed, the present invention
It is pre-processed in embodiment, beginning and end coordinate when record user draws per unicursal, if the endpoint of certain unicursal
Euclidean distance between coordinate and the extreme coordinates of other strokes is less than the threshold value of 30 pixels, then just connecting between the two points
Line.
Bounding box processing：It is subsequent in order to effectively, consistently carry out since the drafting part of different sketches is not of uniform size
Feature extraction and machine learning need to carry out bounding box processing to view, obtain the partial zones for having sketch lines in original image
Domain.
Keep proportional zoom processing：In order to meet graphical rule invariance, allow as much as possible all figures blank boundary all
It minimizes, carries out keeping proportional zoom processing.Use 180 divided by bounding box longest edge length as scaling ratio because
Son, to carry out image scaling.
Binary conversion treatment：Cartographical sketching becomes after scaling for gray-scale map, needs to carry out binary conversion treatment, in order to model
Global characteristics extraction in retrieval flow.
Border extension processing：In order to enable sketch drafting region is placed in the middle, need to carry out border extension processing, by image surrounding
It fills in the blanks so that image size is unified for 200*200 pixels.
Vacancy filling is handled：Since the contour line of sketch includes outer contour and inner outline, for model index
Scene, usual outer contour distinguish different classes of model enough, so to make cartographical sketching with projection view similar
Details is compared in the case of showing degree, needs first to be filled it pretreatment.
Further, component mark module 2 includes：
Contour line extraction unit, for image to carry out contour line extraction to treated；
Sampling unit obtains sampled point for carrying out equal interval sampling to the image after Extracting contour；
Component marking unit, for adding component tag to sampled point.
(1) major class divides
Model classification in database is numerous, and topological structure is multifarious, therefore carries out major class division before component marks
It is necessary.The standard that major class divides is two aspects：On the one hand, it should make the model of similar topology structure as much as possible
It is included into the same major class, carries out the training study of component segmentation together；On the other hand, it should make all model partitions in database
The major class number gone out is as few as possible, to reduce the total degree that online part cartographical sketching is divided by each major class.
(2) equally spaced sampling point design and component tag are based on
In view of the feature extraction subsequently to each sampled point, it is desirable that sampled point size must assure that following three aspects：
Sampled point is intensive enough, in order to avoid different components are fallen into the same sampled point, causes the inaccuracy of component tag；Sampled point is not
Preferably excessively intensive, in order to avoid total operation cost of extraction feature is excessively high, causing can not real-time response in retrieval；The size of sampled point
Determination will consider sampled point size of a variety of unitary features in extraction so that between the sampled point length of side for extracting different characteristic
In the relationship of integral multiple, to ensure that each sampled point can obtain rational, effective feature combination.
It based on these above-mentioned reasons, is attempted by many experiments, finally by the way of equal interval sampling, to every width profile
Line is along contour line apart from every 10 pixel extractions, 1 sampled point.Component tag is added to sampled point, such as head, body, four limbs, tail
Bar etc., component segmentation is done for the view to big class model, with the local feature vectors of each view samples point in current major class
It is trained.
(3) the automatic pre-segmentation based on discrete curve evolvement model and framework information
In the research for carrying out component segmentation based on view, an intuitive idea is namely based on the geometry of view outline line
Feature is analyzed, so that it is determined that the cut-off rule between adjacent component, and then carry out the segmentation of component.In order to improve addition label
Efficiency, in embodiments of the present invention, using the automatic partitioning scheme of view component based on discrete curve evolvement model and skeleton,
Automatically carry out the pre-segmentation of component.
By the difference set of the Extreme points set and skeleton end point set of the simplified polygon of discrete curve evolvement model, and on
In the set each element be point of contact, using axis as other all point of contacts of the inscribed circle in the center of circle, to constitute pre-segmentation point
Collection.It is found through experiments that, which is all the set that the potential cut-point of most models is constituted, therefore uses the set conduct
Contour line is segmented foundation, adds and marks according to contour line sampled point sequential segment, eliminates each component circle and polygon is selected to surround
The process of box, to effectively promote labeling effciency.
(4) design of component label small tool
Component tag is added to sample point data for convenience, uses the mode of interactive addition and modification label.Such as
It is to add tagged schematic diagram shown in Fig. 4, in such a way that mouse is clicked and delimit component polygonal embracing cartridge, you can easily
Sampled point where component is marked.
The label for not adding major class label and model class in output file mainly considers in later stage convenient constantly tune
Whole feature extraction mode and parameter, as long as and the processing mode of bounding box and sampled point is constant, the component tag of sampled point is not
It is influenced by adjustment feature extracting method.Therefore the degree of coupling for reducing label and characteristic extraction part as far as possible, this two parts
It peels away, in accordance with unified file designation rule, is respectively written into two files under same catalogue, facilitates post-processing.Add
The work of label is fully completed in interactive small tool, is not interfere with each other with automatic characteristic extraction procedure.
Further, sampled point characteristic extracting module 3 includes：
Unitary feature extraction unit, for carrying out unitary feature extraction to the sampled point after addition component tag；
Binary feature extraction unit, for carrying out binary feature extraction to the sampled point after addition component tag.
(1) the unitary feature of single sampled point itself
Unitary feature is used to characterize the feature inside each sampled point.Unitary feature employed in the embodiment of the present invention is all
It is that each feature vectors of each sampled point are calculated based on sampled point.It will illustrate eigen extraction process respectively below
Details.
2D shape characteristics of diameters：To the calculating of each sampled point according to the angle of its line between neighbouring sample point, meter
The tangential direction for calculating the point sends out ray in image mask, it is another to meet at image then along the direction vertical with the tangent line
The marginal point of side calculates the length in shaped interior ray portion.Similarly, it calculates with vertical line both sides into 30 °, 60 ° angles
The ray that direction is drawn meets at the marginal point of the image other side, and also the same computational length value finally seeks the flat of these distance values
Mean value 1 is tieed up totally as the 2D shape characteristics of diameters at the sampled point.
Distance feature of the sampled point to image center：Using sampled point to the Euclidean distance between image center as one
A part for first feature.
Average Euclidean distance feature：The average euclidean distance metric based on sampled point, for characterize each sampled point away from
Separate degree from other sampled points.Such as in the view of a width insect, the average Euclidean distance of insect leg is usually than other
Component is farther.The average Euclidean distance of each sampled point is acquired by the distance matrix of SC sampled points, if having in each sampled point
Multiple sampled points then use sample point Euclidean distance average value.Euclidean distance is by calculating each sample point to other each samples
The Euclidean distance average value of point obtains.Calculate simultaneously mean value square and the 10th, the 20th, the 30th until the 90th quantile
Data, then by the maximum value during this 11 statistical measures are unified divided by all sampled point Euclidean distances of present image, to
It is normalized, constitutes 11 dimensional vectors.
Shape context histogram feature：Shape context algorithm equally spaced takes sampled point on object edge line, meter
Calculate Euclidean distance and angle of each sampled point relative to other each sampled points.
Place connected component proportion feature：Since some manual draws and view are made of multiple contour lines, each profile
Line segment usually characterizes semantically independent component.1 dimensional vector is used in the embodiment of the present invention, for characterizing current sampling point
The connected component at place accounts for the ratio of entire image.First, each section of wheel is recorded respectively in the stage of Extracting contour and down-sampling
The sampling number of profile, then by judging that the sampling number of the contour line where current sampling point accounts for the ratio of total sampling number
Example, to obtain a ratio characteristic unrelated with sampling step length, takes turns to characterize the contour line where current sampling point always
Proportion in profile.To insects it was found that, with smaller connected component's proportion feature sampled point be typically leg,
Feeler etc. is in the more long and narrow component in object periphery.
(2) binary feature between adjacent sampled point
Binary feature is the tag compliance weighed between each sampled point and adjacent sampled point.Therefore, two are being calculated
Before first feature, need first to obtain each sampled point abutment points information on contour line.It is ordered into due to sampling point sequence, so
Whether syntople can be judged more than sampling step length threshold value by the Euclidean distance to consecutive points in sampling point sequence, if greatly
In threshold value then be abutment points, on the contrary it is then be abutment points.In addition, it is contemplated that some models have a plurality of profile line segment, so also
It need to judge whether abutted between the starting sample point of every section of contour line and termination sampled point.Binary feature needs enough differentiations
Degree, that is, the binary feature of the binary feature and non-component intersection sampled point that require component intersection sampled point should have larger difference
It is different.The absolute value of the difference of 2D shape diameters and the absolute value of the difference of tangential direction, as binary feature.Component intersection samples
The numerical value of each component value of binary feature vector of point is larger；Without each point of the binary feature of the sampled point near component point of interface
Numerical quantity is smaller, therefore has significant discrimination to the intersection of component.
Further, component segmentation module 4 includes：
Parted pattern training unit, for being split according to each feature vectors of the sampled point after addition component tag
Model training；
Component cutting unit, for carrying out component segmentation to the sampled point after addition component tag according to parted pattern.
In specific implementation, after adding component tag, using condition random field (CRF) model, it is split the instruction of model
Practice.
(1) conditional random field models
The objective energy function of CRF models is made of unitary item and binary item.Wherein unitary item weighs the unitary of sampled point
The consistency of feature and its label；And binary Xiang Ze is weighed the label compatibility between sampled point and abutment points by binary feature.
Here component segmentation and labeling method based on CRF models are used, each major class internal model view is respectively used to
Component segmentation training study.The optimum label for calculating all sampled points requires to minimize object function, such as formula (1) institute
Show：
Object function includes two large divisions, wherein E1For unitary energy term, E2For dual-energy item.
Unitary energy term E1：To assess a grader.Grader is defeated using the feature vector x of sampled point as input
Go out under the conditions of this feature, and the probability distribution P of sampled point label (c | x, θ1).Engineering is carried out using JointBoost graders
It practises.As formula (2) show the calculation of unitary energy term：
E1(c；x,θ1)=- logP (c | x, θ1) (2)
In formula (2), x is unitary feature, and the unitary energy term of each label c is equal in this feature vector condition
Under, the negative logarithm of sampled point label probability distribution.
Dual-energy item E2：Different labels are labeled as between each sampled point and neighbouring sample point on contour line for characterizing
Probability, definition is as shown in formula (3)：
E2(c,c'；y,θ2)=W (c, c') [- κ logP (c ≠ c'| y, ξ)+μ] (3)
In formula (3), y is binary feature.P (c ≠ c'| y, ξ) characterizes the different possibility size of label, it is one
The function of a geometric properties.Label punishes that matrix W (c, c') indicates the degree of compatibility between label c and c'.It is symmetrical matrix,
In matrix each element is both initialized to 9999, and iteratively learning process obtains the penalty value between each pair of label.
(2) iteration of conditional random field models parameter optimizes learning method
In the unitary for extracting sampled point, after binary feature, learn the ginseng of CRF models by the way of iteration optimization
Number, is randomly divided into 5 equal portions by the training sampled point set marked, wherein 4 parts are used as training set, 1 part as verification collection.It is first
First, JointBoost graders are trained with sample set, learns the partial parameters of unitary item and binary item.Then, with verification collection with
The method optimizing segmentation result of iteration, to learn remaining parameter in CRF model binary items.
The training that the component that three projection views of each model are partitioned into is carried out to grader respectively is also three in retrieval
A view calculates separately the votes fallen in respective classes.It is trained in this way than view component feature is all put together
Precision higher, and reduce the calculation amount of online part, to improve recall precision.
Sampled point component flag sequence can be obtained using component segmentation.The sequential recording component tag of each sampled point
And its coordinate position in the picture.
Since sampled point marks the result is that along contour line ordered arrangement, the embodiment of the present invention is designed based on this feature
Component diagram generating algorithm sequentially reads in each sampled point label and its image coordinate point, records upper one of each component tag
Sample point coordinate position, and the Euclidean distance between adjacent same label sampled point is calculated based on this, if distance is less than contour line
Equidistant sampling step length, then in this point-to-point transmission connecting line segment.It is accordingly created if the new component tag for reading in non-generating unit figure
The component diagram matrix.All parts sampling number is also recorded while reading in sampled point label.It is made iteratively the process, directly
It has been read to sampled point flag sequence.Finally, bounding box and extended boundary are calculated separately to the image of each component so that component occupies
In, to generate the component diagram that size is 100*100 pixels.The weight of each component is that the component sampling number accounts for total sampled point
Several ratios.
The Zernike moment characteristics descriptors of use, constitute the global characteristics of view.Zernike squares meet rotational invariance,
There is preferable discrimination to different shape profile.There are two parameter, i.e. n and m, n to indicate the rank of Zernike squares for Zernike squares,
M indicates the repeat number of Zernike squares.A plural number Zernike square value can be obtained in the combination of every group of n and m, using Zernike squares
Component of the amplitude as global characteristics vector.The more results of component that this feature descriptor is selected are more accurate, but component is got over
It is calculate time-consuming be consequently increased more.Take into account retrieval accuracy and calculate take two aspect, select 10 dimension Zernike squares, n and
The combination of m values is as shown in table 1.
N the and m values of 1 Zernike square global characteristics of table combine
| N values | 3 | 5 | 7 | 9 | 11 | 4 | 6 | 8 | 10 | 12 |
| M values | 3 | 3 | 3 | 3 | 3 | 4 | 4 | 4 | 4 | 4 |
It is lateral to image, longitudinal to be divided into 32 parts, successively using 200/32 pixel as step-length movable block.In the process of movable block
In, if encountering the situation without any contour line in block, the statistical data to the block is directly abandoned, to avoid due to image
The case where characteristic component is largely 0 caused by local blank.
In offline part, Gabor characteristic is clustered with K-means clustering algorithms, 128 clusters are formed, by each cluster
Center preserves, and constitutes dictionary.Finally, the number of the nearest each Gabor characteristic vector in each cluster center of statistical distance.
The direct syntople of component tag provides the topological structure between component, and topological features are for component
Torsion have very strong robustness.For example, the geometric properties difference of the leg of the horse of the leg and standing of horse run
It is larger, however, whether being the state of running, leg is all adjacent with body part.It is established between component based on this dot characteristics
The graph model of topological structure connects side between adjacent component, is realized with adjacency list using component as the node of figure.If hand
The cartographic sketching component is different from the adjacent part of projection view corresponding component, then punishes the difference of two figure topology levels,
Reduce similarity score.
Consider topology information between bottom geological information and component, all portions are accounted for according to the component sampling number
The weighted value for the ratio-dependent component that the ratio of the total sampling number of part and the component occur in all model libraries.Formula
(4) it show the calculation formula of component weight：
Formula (5) show the calculation formula of the similarity score at the visual angles view between sketch and model：
The component topological structure of sketch may not be met with the topological structure of projection view, here using rational punishment
, to consider topological structure difference.It will be compared between corresponding component in topological structure adjacency list, tied if there is topology
The inconsistent situation of structure then punishes similarity score according to the weight of the projection view component.
The information content that the projection view of threedimensional model different visual angles is provided is different.Such as, it is virtually impossible to from people's
Component information is obtained in top view, can not also find out that it belongs to the model projection of people, and what the front view of people then clearly showed
The semanteme component such as head, body, four limbs.The component count that only view publishing goes out objectively reflects the information that projection view can be provided
Amount.Based on this feature, as shown in formula (6) calculating separately the component count being partitioned into front view, side view and vertical view accounts for
Three-view diagram is partitioned into the ratio of total parts count, respectively as front view weight wi,front, side view weight wi,sideIt is weighed with vertical view
Weight wi,top, the calculation formula of each model overall similarity scoring in current major class is shown with formula (7)：
Finally, according to overall score ascending sort, preceding 200 models are returned into user, corresponding thumbnail is presented in paging
In a browser, that is, primary retrieval process is completed.
Correspondingly, the embodiment of the present invention also provides a kind of method for searching three-dimension model divided based on cartographical sketching component,
As shown in figure 5, this method includes：
S501 receives Freehandhand-drawing and inquires sketch, and carrying out denoising to Freehandhand-drawing inquiry sketch obtains gray-scale map, and to gray-scale map
Binary conversion treatment, border extension processing, vacancy filling processing are carried out, treated image is obtained；
S502, to treated, image carries out equal interval sampling, obtains sampled point, and add component tag to sampled point；
S503 extracts each feature vectors of sampled point；
S504 is split model training according to each feature vectors of the sampled point after addition component tag；
S505 carries out component local shape factor based on parted pattern and component local similarity calculates, and to processing
Image afterwards carries out the extraction of view global characteristics and view overall situation similarity calculation, is ranked up according to overall score, and will sequence
As a result client is returned to.
Wherein, S501 further comprises：
Denoising is carried out to Freehandhand-drawing inquiry sketch and obtains gray-scale map；
Binary conversion treatment is carried out to gray-scale map；
Image surrounding after binary conversion treatment is handled into line blank filling；
Vacancy filling processing is carried out to blank filling treated image.
S502 further comprises：
To treated, image carries out contour line extraction；
Equal interval sampling is carried out to the image after Extracting contour, obtains sampled point；
Component tag is added to sampled point.
S503 further comprises：
Unitary feature extraction is carried out to the sampled point after addition component tag；
Binary feature extraction is carried out to the sampled point after addition component tag.
S504 further comprises：
It is split model training according to each feature vectors of the sampled point after addition component tag；
Component segmentation is carried out to the sampled point after addition component tag according to parted pattern.
Specifically, the realization process of the method for the present invention embodiment can be found in the phase of the operation principle of system related functions module
Description is closed, which is not described herein again.
In embodiments of the present invention, the topology information between the geological information of comprehensive utilization cartographical sketching component, component
And the global information of whole picture view, and the mechanism provided with three-view diagram dynamic weight index, amplify important visual angle in overall score
It influences, so that the three-dimensional model search based on cartographical sketching is more precisely effective；In addition, can be individually used for sketch understanding,
In the application scenarios of the manual draws component segmentation such as sketch classification.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium may include：Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random
Access Memory), disk or CD etc..
In addition, being provided for the embodiments of the invention the three-dimensional model searching system divided based on cartographical sketching component above
And method is described in detail, principle and implementation of the present invention are described for specific case used herein,
The explanation of above example is only intended to facilitate the understanding of the method and its core concept of the invention；Meanwhile for the one of this field
As technical staff, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, to sum up institute
It states, the content of the present specification should not be construed as limiting the invention.
Claims (6)
1. it is a kind of based on cartographical sketching component segmentation three-dimensional model searching system, including preprocessing module, component mark module,
Sampled point characteristic extracting module, component segmentation module and similarity calculation and overall score sorting module, which is characterized in that
The preprocessing module inquires sketch for receiving Freehandhand-drawing, and carrying out denoising to Freehandhand-drawing inquiry sketch obtains ash
Figure is spent, and binary conversion treatment, border extension processing, vacancy filling processing are carried out to the gray-scale map, treated for acquisition
Image；The preprocessing module includes：
Sketch denoising unit obtains gray-scale map for carrying out denoising to Freehandhand-drawing inquiry sketch；With
Binary conversion treatment unit, for carrying out binary conversion treatment to the gray-scale map；With
Border extension processing unit, for being handled into line blank filling the image surrounding after binary conversion treatment；With
Vacancy fills processing unit, for carrying out vacancy filling processing to blank filling treated image；
The component mark module obtains sampled point, and to described for carrying out equal interval sampling to treated the image
Sampled point adds component tag；
The sampled point characteristic extracting module, each feature vectors for extracting the sampled point；
The component divides module：
Parted pattern training unit, for being split model according to each feature vectors of the sampled point after addition component tag
Training；With
Component cutting unit, for carrying out component segmentation to the sampled point after addition component tag according to parted pattern；
The similarity calculation and overall score sorting module, for carrying out component local shape factor and portion based on parted pattern
Part local similarity calculates, and carries out the extraction of view global characteristics and view overall situation similarity meter to treated the image
It calculates, is ranked up according to overall score, and ranking results are returned into client.
2. the three-dimensional model searching system based on the segmentation of cartographical sketching component as described in claim 1, which is characterized in that institute
Stating component mark module includes：
Contour line extraction unit, for carrying out contour line extraction to treated the image；
Sampling unit obtains sampled point for carrying out equal interval sampling to the image after Extracting contour；
Component marking unit, for adding component tag to the sampled point.
3. the three-dimensional model searching system based on the segmentation of cartographical sketching component as described in claim 1, which is characterized in that institute
Stating sampled point characteristic extracting module includes：
Unitary feature extraction unit, for carrying out unitary feature extraction to the sampled point after addition component tag；
Binary feature extraction unit, for carrying out binary feature extraction to the sampled point after addition component tag.
4. a kind of method for searching three-dimension model based on the segmentation of cartographical sketching component, which is characterized in that include the following steps：
It receives Freehandhand-drawing and inquires sketch, carrying out denoising to Freehandhand-drawing inquiry sketch obtains gray-scale map, and to the gray-scale map
Binary conversion treatment, border extension processing, vacancy filling processing are carried out, treated image is obtained；
Equal interval sampling is carried out to treated the image, obtains sampled point, and component tag is added to the sampled point；
Unitary feature extraction is carried out to the sampled point after addition component tag；
Binary feature extraction is carried out to the sampled point after addition component tag；
It is split model training according to each feature vectors of the sampled point after addition component tag；
Component segmentation is carried out to the sampled point after addition component tag according to parted pattern；
Component local shape factor and component local similarity are carried out based on parted pattern to calculate, and to treated the figure
As carrying out the extraction of view global characteristics and view overall situation similarity calculation, it is ranked up according to overall score, and ranking results are returned
Back to client.
5. the method for searching three-dimension model based on the segmentation of cartographical sketching component as described in claim 4, which is characterized in that institute
It states and denoising acquisition gray-scale map is carried out to Freehandhand-drawing inquiry sketch, and binary conversion treatment, boundary are carried out to the gray-scale map
Extension process, vacancy filling processing, the step of obtaining treated image, including：
Denoising is carried out to Freehandhand-drawing inquiry sketch and obtains gray-scale map；
Binary conversion treatment is carried out to the gray-scale map；
Image surrounding after binary conversion treatment is handled into line blank filling；
Vacancy filling processing is carried out to blank filling treated image.
6. the method for searching three-dimension model based on the segmentation of cartographical sketching component as described in claim 4, which is characterized in that institute
It states and equal interval sampling is carried out to treated the image, obtain sampled point, and add the step of component tag to the sampled point
Suddenly, including：
Contour line extraction is carried out to treated the image；
Equal interval sampling is carried out to the image after Extracting contour, obtains sampled point；
Component tag is added to the sampled point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510267873.3A CN104850633B (en) | 2015-05-22 | 2015-05-22 | A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510267873.3A CN104850633B (en) | 2015-05-22 | 2015-05-22 | A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104850633A CN104850633A (en) | 2015-08-19 |
| CN104850633B true CN104850633B (en) | 2018-10-12 |
Family
ID=53850277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510267873.3A Active CN104850633B (en) | 2015-05-22 | 2015-05-22 | A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104850633B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122396A (en) * | 2017-03-13 | 2017-09-01 | 西北大学 | Three-dimensional model searching algorithm based on depth convolutional neural networks |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243137B (en) * | 2015-09-30 | 2018-12-11 | 华南理工大学 | A kind of three-dimensional model search viewpoint selection method based on sketch |
| CN106126581B (en) * | 2016-06-20 | 2019-07-05 | 复旦大学 | Cartographical sketching image search method based on deep learning |
| CN106530397B (en) * | 2016-10-13 | 2019-06-11 | 成都希盟泰克科技发展有限公司 | A kind of geology face three-dimensional rebuilding method based on sparse section geology contour line |
| CN106874350B (en) * | 2016-12-27 | 2020-09-25 | 合肥阿巴赛信息科技有限公司 | Diamond ring retrieval method and system based on sketch and distance field |
| CN107220277A (en) * | 2017-04-14 | 2017-09-29 | 西北大学 | Image retrieval algorithm based on cartographical sketching |
| JP6659641B2 (en) * | 2017-09-13 | 2020-03-04 | ファナック株式会社 | 3D model creation device |
| CN108109205B (en) * | 2017-12-08 | 2019-12-20 | 山东师范大学 | Incomplete model retrieval and reconstruction method based on hole filling |
| US10521700B2 (en) | 2017-12-14 | 2019-12-31 | Honda Motor Co., Ltd. | Methods and systems for converting a line drawing to a rendered image |
| CN108170823B (en) * | 2018-01-04 | 2021-10-15 | 江西师范大学 | Hand-drawn interactive three-dimensional model retrieval method based on high-level semantic attribute understanding |
| CN110019914B (en) * | 2018-07-18 | 2023-06-30 | 王斌 | Three-dimensional model database retrieval method supporting three-dimensional scene interaction |
| CN109325529B (en) * | 2018-09-06 | 2021-07-30 | 安徽大学 | Sketch identification method and application of sketch identification method in commodity retrieval |
| CN109308324A (en) * | 2018-09-08 | 2019-02-05 | 中山大学 | A kind of image search method and system based on hand drawing style recommendation |
| CN109741454A (en) * | 2018-11-20 | 2019-05-10 | 浙江大学 | A kind of sketch formula three-dimensional cartoon model formative method based on component combination and deformation |
| CN110544300B (en) * | 2019-09-05 | 2021-06-29 | 佛山市玖章智能科技有限公司 | Method for automatically generating three-dimensional model based on two-dimensional hand-drawn image characteristics |
| CN110598634B (en) * | 2019-09-12 | 2020-08-07 | 山东文多网络科技有限公司 | Machine room sketch identification method and device based on graph example library |
| CN110889854B (en) * | 2019-10-16 | 2023-12-05 | 深圳信息职业技术学院 | Sketch part segmentation method, system, device and storage medium based on multi-scale deep learning |
| CN112764538A (en) * | 2021-01-13 | 2021-05-07 | 杭州师范大学 | Gesture interaction based space capacity improving method in VR environment |
| CN112966138B (en) * | 2021-02-22 | 2022-04-12 | 济南大学 | Two-dimensional shape retrieval method and system based on contour feature point matching |
| CN113093904A (en) * | 2021-03-26 | 2021-07-09 | 河北建筑工程学院 | Method, system, medium and device for obtaining image based on somatosensory device |
| CN114913330B (en) * | 2022-07-18 | 2022-12-06 | 中科视语(北京)科技有限公司 | Point cloud component segmentation method and device, electronic equipment and storage medium |
| CN116597163A (en) * | 2023-05-18 | 2023-08-15 | 广东省旭晟半导体股份有限公司 | Infrared optical lens and method for manufacturing the same |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942795A (en) * | 2014-04-22 | 2014-07-23 | 浙江大学 | Structural synthesis method of image object |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7729531B2 (en) * | 2006-09-19 | 2010-06-01 | Microsoft Corporation | Identifying repeated-structure elements in images |
-
2015
- 2015-05-22 CN CN201510267873.3A patent/CN104850633B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942795A (en) * | 2014-04-22 | 2014-07-23 | 浙江大学 | Structural synthesis method of image object |
Non-Patent Citations (2)
| Title |
|---|
| 一种基于部件空间分布的三维模型检索方法;万丽莉等;《软件学报》;20071130;第18卷(第11期);第2902-2913页 * |
| 基于手绘草图的三维模型检索系统;张必欢;《中国优秀硕士学位论文全文库》;20111215(第S2期);第12、15、20、27、49-50页 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122396A (en) * | 2017-03-13 | 2017-09-01 | 西北大学 | Three-dimensional model searching algorithm based on depth convolutional neural networks |
| CN107122396B (en) * | 2017-03-13 | 2019-10-29 | 西北大学 | Method for searching three-dimension model based on depth convolutional neural networks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104850633A (en) | 2015-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104850633B (en) | A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component | |
| KR101768521B1 (en) | Method and system providing informational data of object included in image | |
| CN111143479B (en) | Knowledge graph relation extraction and REST service visualization fusion method based on DBSCAN clustering algorithm | |
| WO2022116537A1 (en) | News recommendation method and apparatus, and electronic device and storage medium | |
| CN111177591B (en) | Knowledge graph-based Web data optimization method for visual requirements | |
| CN104881458B (en) | A kind of mask method and device of Web page subject | |
| Wilkinson et al. | Neural Ctrl-F: segmentation-free query-by-string word spotting in handwritten manuscript collections | |
| CN111190900B (en) | JSON data visualization optimization method in cloud computing mode | |
| CN101477529B (en) | Three-dimensional object retrieval method and apparatus | |
| US20150178321A1 (en) | Image-based 3d model search and retrieval | |
| JP6888484B2 (en) | A search program, a search method, and an information processing device on which the search program operates. | |
| CN111460201B (en) | Cross-modal retrieval method for modal consistency based on generative countermeasure network | |
| CN110210431B (en) | Point cloud semantic labeling and optimization-based point cloud classification method | |
| US20160132498A1 (en) | Color Sketch Image Searching | |
| US20150199567A1 (en) | Document classification assisting apparatus, method and program | |
| CN105740378B (en) | Digital pathology full-section image retrieval method | |
| CN108763348A (en) | A kind of classification improved method of extension short text word feature vector | |
| JP4937395B2 (en) | Feature vector generation apparatus, feature vector generation method and program | |
| CN107958067A (en) | It is a kind of based on without mark Automatic Feature Extraction extensive electric business picture retrieval system | |
| CN102073713A (en) | Optimal bag-of-words (BoW) model based three-dimensional model searching method | |
| CN109857957B (en) | Method for establishing label library, electronic equipment and computer storage medium | |
| CN106844482B (en) | Search engine-based retrieval information matching method and device | |
| CN113239818B (en) | Table cross-modal information extraction method based on segmentation and graph convolution neural network | |
| CN110147841A (en) | The fine grit classification method for being detected and being divided based on Weakly supervised and unsupervised component | |
| CN109308324A (en) | A kind of image search method and system based on hand drawing style recommendation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |