CN105205169A - Distributed image index and retrieval method - Google Patents
Distributed image index and retrieval method Download PDFInfo
- Publication number
- CN105205169A CN105205169A CN201510657308.8A CN201510657308A CN105205169A CN 105205169 A CN105205169 A CN 105205169A CN 201510657308 A CN201510657308 A CN 201510657308A CN 105205169 A CN105205169 A CN 105205169A
- Authority
- CN
- China
- Prior art keywords
- image
- node
- retrieval
- index
- distributed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/134—Distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
Abstract
The invention discloses a distributed image index and retrieval method. The method includes the steps that a Hadoop distributed system cluster is built and comprises an HDFS distributed file system, a YARN resource manager, a ZooKeeper distributed application program coordination server, a Spark cluster computing environment and an HBase database; a Spark cluster based on the YARN resource manager is deployed and configured; an image index stream processing task is started; an image retrieval stream processing task is started; the image retrieval stream processing task receives images input from the outside, and an index is built; the image retrieval stream processing task receives images to be retrieved, and retrieved similar images are output. According to the method, the Spark Streaming technology based on memory computing is adopted, distributed quick computing of image processing tasks and feature extraction tasks can be achieved, and instantaneity is high.
Description
Technical field
The invention belongs to large-scale image process and searching field, particularly a kind of distributed image index based on SparkStreaming and search method.
Background technology
Present stage, unit image procossing and retrieval technique, unit computing power is limited.Along with the growth of image data amount, unit process can cause very large time delay.In addition, traditional retrieving similar images algorithm, computation complexity is high, and result of calculation cannot be multiplexing, and each retrieval all spends long time to carry out Similarity Measure.
Summary of the invention
Goal of the invention: technical matters to be solved by this invention is that to overcome the computing power of unit image procossing poor, time ductility long, the limited and slow-footed problem of image retrieval of memory capacity.
Technical scheme: the present invention proposes a kind of based on SparkStreaming distributed image index and search method.The method comprises the following steps:
Step 1, builds Hadoop distributed system cluster, comprises HDFS distributed file system, YARN explorer, the coordination service of ZooKeeper distributed application program, Spark cluster and HBase database;
Step 2, deployment configuration is based on the Spark cluster of explorer YARN;
Step 3, starts image index stream Processing tasks;
Step 4, starts image retrieval stream Processing tasks;
Step 5, image index stream Processing tasks receives outside input image sequence and sets up index;
Step 6, image retrieval stream Processing tasks receives image to be retrieved, exports the similar image retrieved.
Wherein, in step 1, described YARN explorer is used for task scheduling, be responsible for PC cluster resource management and for each task matching resource (comprising memory source, processor resource etc.), and in the cluster one malfunctions time be that task redistributes computational resource (specifically see step 3 with step 4), described HBase database is used for storage figure picture.
Step 2 comprises: at Hadoop distributed system cluster deploy Spark cluster, utilize YARN explorer to dispatch the Spark task container of Spark cluster, the working node in Hadoop distributed system cluster comprises index node, retrieval node and output node;
Wherein, index node is responsible for the image zooming-out feature to input, and is saved in the HBase database table of specifying, for later retrieval;
Retrieval node, is responsible for calculating itself and the similarity of specifying institute's storage figure picture in HBase database table, go forward side by side line ordering and output to the image to be retrieved of input;
The result that output node is responsible for all retrieval nodes return carries out merge sort, and from HBase database, reading images is encoded, and then generates original image, then the image retrieved is returned to user.
Step 3 comprises: in the coordination service of ZooKeeper distributed application program, set up N number of concordance list T
i, i=1 ..., N, application Spark flow treatment technology in Spark cluster, set up N number of index node S
iand distribute an idle concordance list for each index node, when an index node lost efficacy, such as extremely exit or network failure, index node corresponding in the coordination service of ZooKeeper distributed application program and concordance list distribution node are deleted automatically, flow the automatic newly-built index node for the treatment of technology by Spark, and re-establish manipulative indexing node and concordance list distribution node in the coordination service of ZooKeeper distributed application program.
Step 4 comprises: Spark flows treatment technology in Spark cluster, sets up N number of retrieval node R
ii=1, N, and be the idle concordance list of each retrieval peer distribution one, when a retrieval node failure, such as extremely exit or network failure, retrieval node corresponding in the coordination service of ZooKeeper distributed application program and concordance list distribution node are deleted automatically, flow the automatic newly-built retrieval node for the treatment of technology by Spark, and in the coordination service of ZooKeeper distributed application program, re-establish corresponding retrieval node and concordance list distribution node.
Step 5 comprises: to the image sequence of user's input, first carries out BASE64 coding to each image I and obtains coding result B, and the character string after coding is distributed to as input the index node S started
i, and from the coordination service of ZooKeeper distributed application program, obtain concordance list T corresponding to this index node
i, then utilize MD5 to encode to the key assignments K of the Image Coding computed image of input, coding result B1 decoded simultaneously, and to decoded image zooming-out visual feature vector X, then [K, X, B] is saved in concordance list T.
Step 6 comprises: to the image to be retrieved of user's input, first carries out BASE64 coding to this image and obtains coding result B2, and the character string after coding is distributed to as input the retrieval node R started
i, i=1 ..., N, and from the coordination service of ZooKeeper distributed application program, obtain concordance list T corresponding to this index node
u, u ∈ [1, N], then decodes to coding result B2, and to decoded image zooming-out visual feature vector X, then calculates itself and concordance list T
uthe color distribution similarity σ (use color distribution similarity in the embodiment of the present invention, also can use other any similarity calculating methods) of the characteristics of image of middle preservation, line ordering of going forward side by side, by sequence after before M group result W
i={ <K, σ, T
u>
d| d=1 ..., M} is sent to output node, and wherein d is the sequence number of sequence, and output node is to the result for retrieval W received
icarry out merge sort, M group result before retaining, and according to the concordance list T of key assignments K and correspondence thereof
ureading images coding result B, decoding synthetic image file also returns image path.
Beneficial effect: present invention employs the SparkStreaming technology calculated based on internal memory, can realize the distributed quick calculating of image processing tasks and feature extraction tasks, has the real-time of height.Secondly, Spark technology is utilized to make whole system design have good extendability and very high handling capacity.Finally, the present invention utilizes ZooKeeper to safeguard multiple concordance list and index node, to retrieve the relations of distribution of node, makes full use of the availability of distributed system, and when single node lost efficacy, cluster can continue to carry out index and retrieval to the image of input.The SparkStreaming technology introduced has good property extending transversely and fault-tolerant ability.SparkStreaming can operate on 100+ node, can provide powerful computing power and reach level delay second.After setting up image index, level search second of large nuber of images can be realized.
The SparkStreaming technology that the present invention introduces has good property extending transversely and fault-tolerant ability.SparkStreaming can operate on 100+ node, can provide powerful computing power and reach level delay second.After once setting up image index, level search second of large nuber of images can be realized.
Accompanying drawing explanation
Fig. 1 is distributed image retrieval based on SparkStreaming and directory system process flow diagram.
Fig. 2 is ZooKeeper node dendrogram.
Fig. 3 is index node Booting sequence figure.
Fig. 4 is retrieval node Booting sequence figure.
Fig. 5 is index node workflow diagram.
Fig. 6 is retrieval node and output node workflow diagram journey figure.
Embodiment
Below in conjunction with accompanying drawing, the present invention is described in further detail.
Present stage, unit image procossing and retrieval technique, unit computing power is limited.Along with the growth of image data amount, unit process can cause very large time delay.In addition, traditional retrieving similar images algorithm, computation complexity is high, and result of calculation cannot be multiplexing, and each retrieval all spends long time to carry out Similarity Measure.The SparkStreaming technology that the present invention introduces has good property extending transversely and fault-tolerant ability.SparkStreaming can operate on 100+ node, can provide powerful computing power and reach level delay second.After once setting up image index, level search second of large nuber of images can be realized.The present invention includes following treatment step.
(1) integrating storage and the computing power of multiple stage computing machine by building Hadoop cluster, utilizing YARN to carry out task scheduling, utilizing HBase to be used as database purchase image.
(2) at Hadoop cluster deploy Spark, YARN is utilized to dispatch Spark task container.As shown in Figure 1, the working node in the present invention comprises index node, retrieval node and output node.Wherein index node is responsible for the image zooming-out feature to input, and is saved in the HBase table of specifying, for later retrieval; Retrieval node is then responsible for calculating itself and the similarity of specifying institute's storage figure picture in HBase table, go forward side by side line ordering and output to the image to be retrieved of input; Output node is then responsible for carrying out merge sort to the result that all retrieval nodes return, and from HBase, reading images is encoded, and then generates original image, then the image retrieved is returned to user.
(3) start image index stream Processing tasks, in ZooKeeper, set up N number of concordance list T
i, (i=1 ..., N), SparkStreaming sets up N number of index node S in the cluster
i, (i=1 ..., N), and distribute an idle concordance list for each index node.When certain index node lost efficacy (extremely exiting or network failure), index node corresponding in ZooKeeper and concordance list distribution node are deleted automatically, by the automatic newly-built index node of SparkStreaming, and in ZooKeeper, re-establish manipulative indexing node and concordance list distribution node.
As shown in Figure 2, store concordance list, index node and index assignment situation in ZooKeeper, utilize ZooKeeper to ensure the consistance of distributed system, ZooKeeper is first newly-built concordance list subtree when starting, and adds N number of concordance list T wherein
i, (i=1 ..., N) and node.As shown in Figure 3, each index node S
iduring startup, first locked by ZooKeeper, then in index node subtree, add oneself, search unappropriated concordance list T
u, and by <S
i, T
u> is saved in index assignment subtree.Unlock after completing aforesaid operations.The node added in index node and index assignment subtree is transient node, and when the index node of correspondence lost efficacy, transient node can be deleted automatically.Spark can detect node failure and redistributes and start index node simultaneously, and this index node oneself also can obtain concordance list according to again adding at ZooKeeper shown in Fig. 3, thus ensure that system can be recovered rapidly when node failure.
(4) start image retrieval stream Processing tasks, SparkStreaming sets up N number of retrieval node R in the cluster
i, (i=1 ..., N), and be the idle concordance list of each retrieval peer distribution one.When certain retrieval node failure (extremely exiting or network failure), retrieval node corresponding in ZooKeeper and concordance list distribution node are deleted automatically, by the automatic newly-built retrieval node of SparkStreaming, and in ZooKeeper, re-establish corresponding retrieval node and concordance list distribution node, continue index task.
As shown in Figure 2, store concordance list, retrieval node and retrieval allocation situation in ZooKeeper, utilize ZooKeeper to ensure the consistance of distributed system.As shown in Figure 4, each retrieval node R
iduring startup, first locked by ZooKeeper, then in retrieval node subtree, add oneself, search unappropriated concordance list T
u, and by <R
i, T
u> is saved in retrieval and distributes in subtree.Unlock after completing aforesaid operations.Distribute at retrieval node and retrieval the node added in subtree and be transient node, when the retrieval node failure of correspondence, transient node can be deleted automatically.Spark can detect node failure and redistributes and start retrieval node simultaneously, this retrieval node oneself also can obtain concordance list according to again adding at ZooKeeper shown in Fig. 4, thus ensure that system can be recovered rapidly when node failure, continue retrieval tasks.
(5) as shown in Figure 5, to the image sequence that user specifies, first BASE64 coding is carried out to each image I and obtain coding result B, and the character string after coding is distributed to as input the index node S started, and from ZooKeeper, obtain concordance list T corresponding to this index node.Then utilize MD5 to the key assignments K of the Image Coding computed image of input, coding result B is decoded simultaneously, and to decoded image zooming-out visual feature vector X.First rgb space image is divided into 8x8=64 block.From each piecemeal, select a kind of color as the remarkable color of this piecemeal.Can use the remarkable color extraction algorithm of any one, usual the present invention uses the typical color of average color as this piecemeal of each piecemeal, because the method is simple and it is enough accurate under normal conditions.And each piecemeal remarkable color of one pixel is represented, generate the little image of 8x8 pixel size.Again image is transformed into YCbCr space from rgb space.Dct transform (discrete cosine transform) is done respectively to the little image Y (brightness) of YCbCr space 8x8, Cb (blue intensity skew), Cr (red-color concentration skew) passage, 3x64 DCT coefficient (DCTY can be obtained, DCTCb, DCTCr), luminance DCT coefficients, blue intensity skew DCT coefficient and red-color concentration skew DCT coefficient is represented respectively.Two-dimensional dct transform as shown in the formula:
Wherein, G, H are respectively width and the height of image, A
ghfor the value at image slices vegetarian refreshments (g, h) place, B
pqfor dct transform result B is in the value at coordinate (p, q) place, α
pand α
qfor intermediate variable.
Then the scanning of zigzag mode is carried out respectively to DCT coefficient (DCTY, DCTCb, DCTCr), obtain the result (DY, DCb, DCr) after zigzag scanning, be characteristics of image X.Finally [K, X, B] is saved in concordance list T.
(6) as shown in Figure 5, to the image to be retrieved that user specifies, first BASE64 coding is carried out to this image and obtain coding result B, and the character string after coding is distributed to as input the retrieval node R started
i, and from ZooKeeper, obtain concordance list T corresponding to this index node
u.Then B is decoded, and to decoded image zooming-out visual feature vector X, then calculate itself and concordance list T
uthe similarity σ of the characteristics of image of middle preservation, if the descriptor of two images is respectively (DY, DCb, DCr) and (DY', DCb ', DCr'), then the distance computing formula of two descriptors:
Wherein, w
yi, w
bi, w
rifor weight coefficient, represent Y respectively, i-th weight of Cb, Cr tri-passages, span is [0,1].If two width images are identical, then σ=0; If two width image similarities, then σ is close to 0.Then sort according to σ, by sequence after before concordance list M group result W
i={ <K, σ, T
u>
d| d=1 ..., M} is sent to output node, as shown in Figure 6, and the result for retrieval W that output node receives
i, i=1 ..., N carries out merge sort, M group result before retaining, and according to the concordance list T of key assignments K and correspondence thereof
ureading images coding B, decoding synthetic image file also returns image path.
The invention provides a kind of distributed image index and search method; the method and access of this technical scheme of specific implementation is a lot; the above is only the preferred embodiment of the present invention; should be understood that; for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.The all available prior art of each ingredient not clear and definite in the present embodiment is realized.
Claims (7)
1. distributed image index and a search method, is characterized in that, comprises the following steps:
Step 1, builds Hadoop distributed system cluster, comprises HDFS distributed file system, YARN explorer, the coordination service of ZooKeeper distributed application program, Spark cluster and HBase database;
Step 2, deployment configuration is based on the Spark cluster of explorer YARN;
Step 3, starts image index stream Processing tasks;
Step 4, starts image retrieval stream Processing tasks;
Step 5, image index stream Processing tasks receives outside input image sequence and sets up index;
Step 6, image retrieval stream Processing tasks receives image to be retrieved, exports the similar image retrieved.
2. a kind of distributed image index according to claim 1 and search method, it is characterized in that, in step 1, described YARN explorer is used for task scheduling, be responsible for the management of PC cluster resource and be each task matching resource, described HBase database is for storage figure picture.
3. a kind of distributed image index according to claim 2 and search method, it is characterized in that, step 2 comprises: at Hadoop distributed system cluster deploy Spark cluster, utilize YARN explorer to dispatch the Spark task container of Spark cluster, the working node in Hadoop distributed system cluster comprises index node, retrieval node and output node;
Wherein, index node is responsible for the image zooming-out feature to input, and is saved in the HBase database table of specifying, for later retrieval;
Retrieval node, is responsible for calculating itself and the similarity of specifying institute's storage figure picture in HBase database table, go forward side by side line ordering and output to the image to be retrieved of input;
The result that output node is responsible for all retrieval nodes return carries out merge sort, and from HBase database, reading images is encoded, and then generates original image, then the image retrieved is returned to user.
4. a kind of distributed image index according to claim 3 and search method, it is characterized in that, step 3 comprises: in the coordination service of ZooKeeper distributed application program, set up N number of concordance list T
i, i=1 ..., N, application Spark flow treatment technology in Spark cluster, set up N number of index node S
iand distribute an idle concordance list for each index node, when an index node lost efficacy, index node corresponding in the coordination service of ZooKeeper distributed application program and concordance list distribution node are deleted automatically, flow the automatic newly-built index node for the treatment of technology by Spark, and re-establish manipulative indexing node and concordance list distribution node in the coordination service of ZooKeeper distributed application program.
5. a kind of distributed image index according to claim 4 and search method, it is characterized in that, step 4 comprises: Spark flows treatment technology in Spark cluster, sets up N number of retrieval node R
ii=1, N, and be the idle concordance list of each retrieval peer distribution one, when a retrieval node failure, retrieval node corresponding in the coordination service of ZooKeeper distributed application program and concordance list distribution node are deleted automatically, flow the automatic newly-built retrieval node for the treatment of technology by Spark, and in the coordination service of ZooKeeper distributed application program, re-establish corresponding retrieval node and concordance list distribution node.
6. a kind of distributed image index according to claim 5 and search method, it is characterized in that, step 5 comprises: to the image sequence of user's input, first carries out BASE64 coding to each image I and obtains coding result B and the character string after coding is distributed to as input the index node S started
i, and from the coordination service of ZooKeeper distributed application program, obtain concordance list T corresponding to this index node
i, then utilize MD5 to encode to the key assignments K of the Image Coding computed image of input, coding result B decoded simultaneously, and to decoded image zooming-out visual feature vector X, then [K, X, B] is saved in concordance list T.
7. a kind of distributed image index according to claim 6 and search method, it is characterized in that, step 6 comprises: to the image to be retrieved of user's input, first BASE64 coding is carried out to this image and obtain coding result B, and the character string after coding is distributed to as input the retrieval node R started
i, and from the coordination service of ZooKeeper distributed application program, obtain concordance list T corresponding to this index node
u, u ∈ [1, N], then decodes to coding result B, and to decoded image zooming-out visual feature vector X, then calculates itself and concordance list T
uthe similarity σ of the characteristics of image of middle preservation, line ordering of going forward side by side, by sequence after before M group result W
i={ <K, σ, T
u>
d| d=1 ..., M} is sent to output node, and wherein d is the sequence number of sequence, and output node is to the result for retrieval W received
icarry out merge sort, M group result before retaining, and according to the concordance list T of key assignments K and correspondence thereof
ureading images coding result B, decoding synthetic image file also returns image path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510657308.8A CN105205169B (en) | 2015-10-12 | 2015-10-12 | A kind of distributed image index and search method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510657308.8A CN105205169B (en) | 2015-10-12 | 2015-10-12 | A kind of distributed image index and search method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105205169A true CN105205169A (en) | 2015-12-30 |
| CN105205169B CN105205169B (en) | 2018-06-15 |
Family
ID=54952852
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510657308.8A Active CN105205169B (en) | 2015-10-12 | 2015-10-12 | A kind of distributed image index and search method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105205169B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155798A (en) * | 2016-08-02 | 2016-11-23 | 大连文森特软件科技有限公司 | The online image conversion programing system calculated based on moving distributing |
| CN106372127A (en) * | 2016-08-24 | 2017-02-01 | 云南大学 | Spark-based diversity graph sorting method for large-scale graph data |
| CN106611046A (en) * | 2016-12-16 | 2017-05-03 | 武汉中地数码科技有限公司 | Big data technology-based space data storage processing middleware framework |
| CN106777167A (en) * | 2016-12-21 | 2017-05-31 | 中国科学院上海高等研究院 | Magnanimity Face Image Retrieval System and search method based on Spark frameworks |
| CN106844654A (en) * | 2017-01-23 | 2017-06-13 | 公安部第三研究所 | Towards the massive video distributed search method of police service practical |
| CN107766147A (en) * | 2016-08-23 | 2018-03-06 | 上海宝信软件股份有限公司 | Distributed data analysis task scheduling system |
| CN107797874A (en) * | 2017-10-12 | 2018-03-13 | 南京中新赛克科技有限责任公司 | A kind of resource management-control method based on embedded jetty and spark on yarn frameworks |
| CN107818147A (en) * | 2017-10-19 | 2018-03-20 | 大连大学 | Distributed temporal index system based on Voronoi diagram |
| CN108121763A (en) * | 2017-11-25 | 2018-06-05 | 无锡十月中宸科技有限公司 | One mode identifies directory system and its indexing means |
| CN108804602A (en) * | 2018-05-25 | 2018-11-13 | 武汉大学 | A kind of distributed spatial data storage computational methods based on SPARK |
| CN108874472A (en) * | 2018-06-08 | 2018-11-23 | 福建天泉教育科技有限公司 | A kind of the optimization display methods and system of user's head portrait |
| CN109165307A (en) * | 2018-09-19 | 2019-01-08 | 腾讯科技（深圳）有限公司 | A kind of characteristic key method, apparatus and storage medium |
| CN109189969A (en) * | 2018-10-22 | 2019-01-11 | 镇江悦乐网络科技有限公司 | A kind of three-dimensional CG animation search method based on image sequence |
| CN109598348A (en) * | 2017-09-28 | 2019-04-09 | 北京猎户星空科技有限公司 | A kind of image pattern obtains, model training method and system |
| CN109918184A (en) * | 2019-03-01 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Picture processing system, method and relevant apparatus and equipment |
| CN110209853A (en) * | 2019-06-14 | 2019-09-06 | 重庆紫光华山智安科技有限公司 | Image searching method, device and the equipment of vehicle |
| CN112131424A (en) * | 2020-09-22 | 2020-12-25 | 深圳市天维大数据技术有限公司 | Distributed image analysis method and system |
| CN112307829A (en) * | 2019-07-31 | 2021-02-02 | 北京博雅慧视智能技术研究院有限公司 | Space-time matrix presentation method for digital retina mass target retrieval |
| CN113485934A (en) * | 2021-07-23 | 2021-10-08 | 掌阅科技股份有限公司 | Distributed index data acquisition method, electronic device and storage medium |
| CN112307829B (en) * | 2019-07-31 | 2024-05-03 | 北京博雅慧视智能技术研究院有限公司 | Digital retina mass target retrieval space-time matrix presentation method |
-
2015
- 2015-10-12 CN CN201510657308.8A patent/CN105205169B/en active Active
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155798A (en) * | 2016-08-02 | 2016-11-23 | 大连文森特软件科技有限公司 | The online image conversion programing system calculated based on moving distributing |
| CN107766147A (en) * | 2016-08-23 | 2018-03-06 | 上海宝信软件股份有限公司 | Distributed data analysis task scheduling system |
| CN106372127A (en) * | 2016-08-24 | 2017-02-01 | 云南大学 | Spark-based diversity graph sorting method for large-scale graph data |
| CN106372127B (en) * | 2016-08-24 | 2019-05-03 | 云南大学 | The diversity figure sort method of large-scale graph data based on Spark |
| CN106611046A (en) * | 2016-12-16 | 2017-05-03 | 武汉中地数码科技有限公司 | Big data technology-based space data storage processing middleware framework |
| CN106777167A (en) * | 2016-12-21 | 2017-05-31 | 中国科学院上海高等研究院 | Magnanimity Face Image Retrieval System and search method based on Spark frameworks |
| CN106777167B (en) * | 2016-12-21 | 2020-05-12 | 中国科学院上海高等研究院 | Massive human face image retrieval system and retrieval method based on Spark framework |
| CN106844654A (en) * | 2017-01-23 | 2017-06-13 | 公安部第三研究所 | Towards the massive video distributed search method of police service practical |
| CN109598348A (en) * | 2017-09-28 | 2019-04-09 | 北京猎户星空科技有限公司 | A kind of image pattern obtains, model training method and system |
| CN107797874A (en) * | 2017-10-12 | 2018-03-13 | 南京中新赛克科技有限责任公司 | A kind of resource management-control method based on embedded jetty and spark on yarn frameworks |
| CN107797874B (en) * | 2017-10-12 | 2021-04-27 | 南京中新赛克科技有限责任公司 | Resource management and control method based on embedded jetty and spark on grow framework |
| CN107818147A (en) * | 2017-10-19 | 2018-03-20 | 大连大学 | Distributed temporal index system based on Voronoi diagram |
| CN108121763A (en) * | 2017-11-25 | 2018-06-05 | 无锡十月中宸科技有限公司 | One mode identifies directory system and its indexing means |
| CN108804602A (en) * | 2018-05-25 | 2018-11-13 | 武汉大学 | A kind of distributed spatial data storage computational methods based on SPARK |
| CN108874472A (en) * | 2018-06-08 | 2018-11-23 | 福建天泉教育科技有限公司 | A kind of the optimization display methods and system of user's head portrait |
| CN108874472B (en) * | 2018-06-08 | 2022-04-12 | 福建天泉教育科技有限公司 | Method and system for optimally displaying user head portraits |
| CN109165307A (en) * | 2018-09-19 | 2019-01-08 | 腾讯科技（深圳）有限公司 | A kind of characteristic key method, apparatus and storage medium |
| CN109189969A (en) * | 2018-10-22 | 2019-01-11 | 镇江悦乐网络科技有限公司 | A kind of three-dimensional CG animation search method based on image sequence |
| CN109918184A (en) * | 2019-03-01 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Picture processing system, method and relevant apparatus and equipment |
| CN109918184B (en) * | 2019-03-01 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Picture processing system, method and related device and equipment |
| CN110209853A (en) * | 2019-06-14 | 2019-09-06 | 重庆紫光华山智安科技有限公司 | Image searching method, device and the equipment of vehicle |
| CN112307829A (en) * | 2019-07-31 | 2021-02-02 | 北京博雅慧视智能技术研究院有限公司 | Space-time matrix presentation method for digital retina mass target retrieval |
| CN112307829B (en) * | 2019-07-31 | 2024-05-03 | 北京博雅慧视智能技术研究院有限公司 | Digital retina mass target retrieval space-time matrix presentation method |
| CN112131424A (en) * | 2020-09-22 | 2020-12-25 | 深圳市天维大数据技术有限公司 | Distributed image analysis method and system |
| CN113485934A (en) * | 2021-07-23 | 2021-10-08 | 掌阅科技股份有限公司 | Distributed index data acquisition method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105205169B (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105205169A (en) | Distributed image index and retrieval method | |
| US10282366B2 (en) | Multi-dimensional decomposition computing method and system | |
| CN102722880B (en) | Image main color identification method and apparatus thereof, image matching method and server | |
| CN107480163A (en) | The efficient ciphertext image search method of secret protection is supported under a kind of cloud environment | |
| CN106254458B (en) | A kind of image processing method based on cloud robot vision, platform and system | |
| CN106897295B (en) | Hadoop-based power transmission line monitoring video distributed retrieval method | |
| CN111178408A (en) | Health monitoring model construction method and system based on federal random forest learning | |
| CN106933833A (en) | A kind of positional information method for quickly querying based on Spatial Data Index Technology | |
| CN104933143B (en) | Obtain the method and device of recommended | |
| Qin et al. | Privacy-preserving outsourcing of image global feature detection | |
| Bothe et al. | Skyline query processing over encrypted data: An attribute-order-preserving-free approach | |
| CN107423942A (en) | A kind of method and device of work flow | |
| CN104303176A (en) | Query processing | |
| KR101780534B1 (en) | Method and system for extracting image feature based on map-reduce for searching image | |
| Salmi et al. | Content based image retrieval based on cell color coherence vector (Cell-CCV) | |
| CN107172430B (en) | The coding method of character block and device | |
| CN104182546B (en) | The data query method and device of database | |
| Wang et al. | QPIN: a quantum-inspired preference interactive network for E-commerce recommendation | |
| Qi et al. | An efficient deep learning hashing neural network for mobile visual search | |
| CN108319659A (en) | A kind of social discovery method based on encrypted image fast search | |
| CN114359484A (en) | Model directory tree reorganization method and device, computer equipment and storage medium | |
| CN107203633A (en) | Tables of data pushes away several processing methods, device and electronic equipment | |
| CN113869859A (en) | Event approval person replacing method and device, terminal device and storage medium | |
| CN112804446A (en) | Big data processing method and device based on cloud platform big data | |
| CN109359213A (en) | A kind of security protection video intelligent retrieval system and its search method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |