CN111553224A - Large remote sensing image block distribution method - Google Patents
Large remote sensing image block distribution method Download PDFInfo
- Publication number
- CN111553224A CN111553224A CN202010317361.4A CN202010317361A CN111553224A CN 111553224 A CN111553224 A CN 111553224A CN 202010317361 A CN202010317361 A CN 202010317361A CN 111553224 A CN111553224 A CN 111553224A
- Authority
- CN
- China
- Prior art keywords
- image
- block
- width
- starting
- height
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000000903 blocking effect Effects 0.000 claims abstract description 34
- 238000000638 solvent extraction Methods 0.000 claims abstract 4
- 238000012545 processing Methods 0.000 abstract description 16
- 238000012360 testing method Methods 0.000 description 10
- 238000001514 detection method Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a large remote sensing image block distribution method, and belongs to the field of remote sensing image processing. Which comprises the following steps: reading metadata information of the large remote sensing image; inputting image blocking parameters; performing logic partitioning to obtain a plurality of partitioning areas and starting and stopping row-column numbers of each partitioning area; reading the whole remote sensing image data to a memory; appointing a port number, constructing a TCP server, and waiting for the connection of a client; for each connected client, according to the blocking sequence, taking out the data area corresponding to the current block from the memory, and sending the current block data to the client; and after all the data blocks are completely sent, closing the TCP server. The method only carries out logic blocking on the data in the memory and does not store the blocking data to the hard disk, thereby greatly improving the blocking efficiency of the data.
Description
Technical Field
The invention belongs to the technical field of remote sensing image processing, and particularly relates to a large remote sensing image block distribution method.
Background
In recent years, with the rapid development of earth observation technology with satellite remote sensing as a core, the resolution of images obtained by remote sensing earth observation is higher and higher, and sub-meter earth observation is realized at present. The data volume of the high-resolution remote sensing image is usually large, and a single remote sensing image usually reaches dozens of GB, so that great challenges are brought to data processing.
In the prior art, for the rapid processing of large remote sensing images, the images are often partitioned first and then processed in a multi-machine parallel mode. Therefore, the remote sensing image optimization blocking strategy and the parallel processing architecture have an important influence on the processing efficiency of the large remote sensing image. At present, the existing block mode and parallel processing architecture of the large remote sensing image have the problem of low efficiency, and have further improved space.
Disclosure of Invention
The invention aims to provide a large remote sensing image blocking and distributing method which is simple and easy to implement and can greatly improve the blocking and distributing efficiency of the large remote sensing image, so that the parallel processing efficiency of the large remote sensing image is integrally improved.
In order to achieve the purpose, the invention adopts the technical scheme that:
a large remote sensing image blocking distribution method comprises the following steps:
step 1, reading large remote sensing image pixel data information, including width, height, wave band number, numerical type and projection parameter information of an image;
step 2, inputting image blocking parameters including blocking width, blocking height and overlapping area width;
step 3, calculating the blocking mode of the image according to the image pixel data information and the image blocking parameters to obtain the serial number and the starting and stopping row and column numbers of each block;
step 4, reading the whole remote sensing image data to a memory;
step 5, appointing a port number, constructing a TCP server, and waiting for the connection of a client;
step 6, aiming at each connected client, matching a block serial number, taking out a corresponding data area from the memory according to the starting and stopping row and column numbers of the block of the serial number, and sending the data of the block to the client;
and 7, closing the TCP server after all the data blocks are completely sent.
Further, the specific manner of step 3 is as follows:
step 3a, if the width w of the image is not more than the block width p and the height h of the image is not more than the block height q, the number of the blocks is 1, the initial rows and the initial rows are all 0, the ending rows and the ending rows are respectively the width and the height of the image, and then the step 3e is carried out;
step 3b, if the width w of the image is larger than the block width p and the height h of the image is not larger than the block height q, the number of the blocks is
Wherein floor (·) represents rounding down, l is the overlap region width; starting and stopping rows of the kth block are respectively l (k-1) +1, l (k-1) + p, starting and stopping columns are respectively 1 and q, and then the step 3e is carried out;
step 3c, if the width w of the image is not more than the block width p and the height h of the image is more than the block height q, the number of blocks is
Starting and stopping rows of the kth block are respectively 1 and p, starting and stopping columns are respectively l (k-1) +1, l (k-1) + q, and then the step 3e is carried out;
step 3d, if the width w of the image is larger than the block width p and the height h of the image is larger than the block height q, the number of blocks is
The starting and stopping lines of the kth block are respectively l (x-1) +1, l (x-1) + p, the starting and stopping lines are respectively l (y-1) +1, l (y-1) + q, wherein,
mod (-) denotes a remainder, then go to step 3 e;
step 3 e: and recording the serial numbers and the starting and stopping row and column numbers of all the blocks.
Further, the serial number and the starting/stopping row/column number of each block obtained in the step 3 are only stored in the memory.
Compared with the prior art, the invention has the following beneficial effects:
(1) the invention provides a method for sending the block data to the client through the TCP protocol, which is a brand new architecture form in the field of image parallel processing, has innovativeness and greatly improves the data acquisition efficiency of the client.
(2) Furthermore, the invention only carries out the logic operation of image blocking in the memory, the blocking result is only stored in the memory, and the blocking data is not stored in the hard disk, thus greatly improving the blocking efficiency of the data.
Drawings
FIG. 1 is a flow chart of a method in an embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
A large remote sensing image blocking distribution method comprises the following steps:
step 1, reading large remote sensing image pixel data information, including width, height, wave band number, numerical type and projection parameter information of an image;
step 2, inputting image blocking parameters including blocking width, blocking height and overlapping area width;
step 3, calculating the blocking mode of the image according to the image pixel data information and the image blocking parameters to obtain the serial number and the starting and stopping row and column numbers of each block;
step 4, reading the whole remote sensing image data to a memory;
step 5, appointing a port number, constructing a TCP server, and waiting for the connection of a client;
step 6, aiming at each connected client, matching a block serial number, taking out a corresponding data area from the memory according to the starting and stopping row and column numbers of the block of the serial number, and sending the data of the block to the client;
and 7, closing the TCP server after all the data blocks are completely sent.
Further, the specific manner of step 3 is as follows:
step 3a, if the width w of the image is not more than the block width p and the height h of the image is not more than the block height q, the number of the blocks is 1, the initial rows and the initial rows are all 0, the ending rows and the ending rows are respectively the width and the height of the image, and then the step 3e is carried out;
step 3b, if the width w of the image is larger than the block width p and the height h of the image is not larger than the block height q, the number of the blocks is
Wherein floor (·) represents rounding down, l is the overlap region width; starting and stopping rows of the kth block are respectively l (k-1) +1, l (k-1) + p, starting and stopping columns are respectively 1 and q, and then the step 3e is carried out;
step 3c, if the width w of the image is not more than the block width p and the height h of the image is more than the block height q, the number of blocks is
Starting and stopping rows of the kth block are respectively 1 and p, starting and stopping columns are respectively l (k-1) +1, l (k-1) + q, and then the step 3e is carried out;
step 3d, if the width w of the image is larger than the block width p and the height h of the image is larger than the block height q, the number of blocks is
The starting and stopping lines of the kth block are respectively l (x-1) +1, l (x-1) + p, and the starting line and the stopping line are respectivelyThe stop columns are respectively l (y-1) +1, l (y-1) + q, wherein,
mod (-) denotes a remainder, then go to step 3 e;
step 3 e: and recording the serial numbers and the starting and stopping row and column numbers of all the blocks.
Further, the serial number and the starting/stopping row/column number of each block obtained in the step 3 are only stored in the memory.
The method comprises the steps of firstly carrying out logic operation on data blocks in a memory, storing block results in the memory, and then directly sending the block data to a processing client through a TCP (transmission control protocol) protocol for a plurality of clients to respectively process the block data in a parallel mode.
The parallel processing architecture based on the TCP protocol is an innovation in the field of image processing, and can greatly improve the distribution and transmission efficiency of block data. In addition, the method only calculates the block in the memory and stores the block result, thereby improving the block speed.
The effect of the present method can be further illustrated by the following tests:
1. test conditions.
The blocking service end is configured to be an Intel Core i7-3770 CPU 3.4Ghz, 128GB memory;
the processing client is configured to 2 Intel Core i7-3770 CPU 3.4Ghz, 32GB memory;
the software environments are all Windows 764 bit professional versions;
and the processing clients are connected by adopting a gigabit router.
2. Test methods.
The processing program of the processing client can be any remote sensing image processing program, such as target detection, ground feature classification, feature extraction and the like. In the test, a client deploys ship target detection software, and efficiency tests of target detection are respectively carried out on input remote sensing images. In order to verify the effect of the method, a blocking method of storing the blocked data in a hard disk is also adopted for comparison.
The target detection mode is a conventional technology. Training a target detection network by using a training sample and sample marking information to obtain parameters of each level of the detection network; constructing a target geometric feature constraint model according to the geometric features of the target; converting the target geometric feature constraint model into pixel constraint by using the resolution information of the image in the horizontal direction and the vertical direction; searching a target candidate region of the remote sensing image to be detected, screening the candidate region according to the pixel constraint of the geometric characteristics, and removing the candidate region which does not accord with the target pixel constraint; and performing feature extraction and identification on the screened target candidate region by using a trained model to realize target detection.
3. Test contents and results.
In the test, remote sensing image data after high-resolution two-number mosaic is selected for target detection, the length and width of the data are 126000 pixels multiplied by 127620 pixels respectively, the resolution is 1m, and a single image is 60 GB.
In the test, the blocking parameter is 3000 × 3000 pixels, the detected target is a ship, and the target length size range is set to 30m to 300m based on empirical knowledge, so the overlap region is 150 pixels. In this test, a total of 1936 blocks were produced.
The 2 processing clients respectively start 16 target detection processes, and the total number is 32 processes.
4. And (5) testing results.
Image blocking took 100.37 minutes using the contrast method, and blocking took 7.08 minutes using the method.
Test results show that the method can improve the calculation efficiency by about 14 times for the block calculation of the large-scale image.
Claims (3)
1. A large remote sensing image block distribution method is characterized by comprising the following steps:
step 1, reading large remote sensing image pixel data information, including width, height, wave band number, numerical type and projection parameter information of an image;
step 2, inputting image blocking parameters including blocking width, blocking height and overlapping area width;
step 3, calculating the blocking mode of the image according to the image pixel data information and the image blocking parameters to obtain the serial number and the starting and stopping row and column numbers of each block;
step 4, reading the whole remote sensing image data to a memory;
step 5, appointing a port number, constructing a TCP server, and waiting for the connection of a client;
step 6, aiming at each connected client, matching a block serial number, taking out a corresponding data area from the memory according to the starting and stopping row and column numbers of the block of the serial number, and sending the data of the block to the client;
and 7, closing the TCP server after all the data blocks are completely sent.
2. The method for partitioning and distributing large remote sensing images according to claim 1, wherein the specific manner of the step 3 is as follows:
step 3a, if the width w of the image is not more than the block width p and the height h of the image is not more than the block height q, the number of the blocks is 1, the initial rows and the initial rows are all 0, the ending rows and the ending rows are respectively the width and the height of the image, and then the step 3e is carried out;
step 3b, if the width w of the image is larger than the block width p and the height h of the image is not larger than the block height q, the number of the blocks is
Wherein floor (·) represents rounding down, l is the overlap region width; starting and stopping rows of the kth block are respectively l (k-1) +1, l (k-1) + p, starting and stopping columns are respectively 1 and q, and then the step 3e is carried out;
step 3c, if the width w of the image is not more than the block width p and the height h of the image is more than the block height q, the number of blocks is
Starting and stopping rows of the kth block are respectively 1 and p, starting and stopping columns are respectively l (k-1) +1, l (k-1) + q, and then the step 3e is carried out;
step 3d, ifThe width w of the image is greater than the block width p, and the height h of the image is greater than the block height q, then the number of blocks is
The starting and stopping lines of the kth block are respectively l (x-1) +1, l (x-1) + p, the starting and stopping lines are respectively l (y-1) +1, l (y-1) + q, wherein,
mod (-) denotes a remainder, then go to step 3 e;
step 3 e: and recording the serial numbers and the starting and stopping row and column numbers of all the blocks.
3. The method for distributing the blocks of the large remote sensing image according to claim 1, wherein the serial number and the starting and stopping row and column numbers of each block obtained in the step 3 are only stored in a memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010317361.4A CN111553224A (en) | 2020-04-21 | 2020-04-21 | Large remote sensing image block distribution method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010317361.4A CN111553224A (en) | 2020-04-21 | 2020-04-21 | Large remote sensing image block distribution method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111553224A true CN111553224A (en) | 2020-08-18 |
Family
ID=72005789
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010317361.4A Pending CN111553224A (en) | 2020-04-21 | 2020-04-21 | Large remote sensing image block distribution method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111553224A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115580738A (en) * | 2022-02-23 | 2023-01-06 | 北京拙河科技有限公司 | High-resolution video display method, device and system based on-demand transmission |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101350098A (en) * | 2008-08-19 | 2009-01-21 | 黄建军 | Method, system and equipment for processing image |
| CN102778675A (en) * | 2012-04-28 | 2012-11-14 | 中国测绘科学研究院 | Atmospheric correction method and atmospheric correction module for satellite remote-sensing image |
| CN102841816A (en) * | 2012-07-16 | 2012-12-26 | 北京市遥感信息研究所 | Method for parallel processing of remote sensing imagery data |
| CN103019671A (en) * | 2012-10-08 | 2013-04-03 | 中国科学院对地观测与数字地球科学中心 | Data-intensive remote sensing image processing-oriented generic programmed framework programming method |
| CN104268096A (en) * | 2014-09-26 | 2015-01-07 | 西南科技大学 | Rapid large-scale point-cloud data reading method based on memory pre-distribution and multi-point writing technology |
| CN105786942A (en) * | 2015-11-27 | 2016-07-20 | 武汉大学 | Geographic information storage system based on cloud platform |
| CN107767324A (en) * | 2017-10-24 | 2018-03-06 | 中国科学院电子学研究所苏州研究院 | A kind of large-scale remote sensing images fast cache method |
| CN107770256A (en) * | 2017-09-29 | 2018-03-06 | 安徽师范大学 | A kind of tele-control system and method for service end and client |
| CN109313663A (en) * | 2018-01-15 | 2019-02-05 | 深圳鲲云信息科技有限公司 | Artificial intelligence calculates Auxiliary Processing Unit, method, storage medium and terminal |
-
2020
- 2020-04-21 CN CN202010317361.4A patent/CN111553224A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101350098A (en) * | 2008-08-19 | 2009-01-21 | 黄建军 | Method, system and equipment for processing image |
| CN102778675A (en) * | 2012-04-28 | 2012-11-14 | 中国测绘科学研究院 | Atmospheric correction method and atmospheric correction module for satellite remote-sensing image |
| CN102841816A (en) * | 2012-07-16 | 2012-12-26 | 北京市遥感信息研究所 | Method for parallel processing of remote sensing imagery data |
| CN103019671A (en) * | 2012-10-08 | 2013-04-03 | 中国科学院对地观测与数字地球科学中心 | Data-intensive remote sensing image processing-oriented generic programmed framework programming method |
| CN104268096A (en) * | 2014-09-26 | 2015-01-07 | 西南科技大学 | Rapid large-scale point-cloud data reading method based on memory pre-distribution and multi-point writing technology |
| CN105786942A (en) * | 2015-11-27 | 2016-07-20 | 武汉大学 | Geographic information storage system based on cloud platform |
| CN107770256A (en) * | 2017-09-29 | 2018-03-06 | 安徽师范大学 | A kind of tele-control system and method for service end and client |
| CN107767324A (en) * | 2017-10-24 | 2018-03-06 | 中国科学院电子学研究所苏州研究院 | A kind of large-scale remote sensing images fast cache method |
| CN109313663A (en) * | 2018-01-15 | 2019-02-05 | 深圳鲲云信息科技有限公司 | Artificial intelligence calculates Auxiliary Processing Unit, method, storage medium and terminal |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115580738A (en) * | 2022-02-23 | 2023-01-06 | 北京拙河科技有限公司 | High-resolution video display method, device and system based on-demand transmission |
| CN115580738B (en) * | 2022-02-23 | 2023-09-19 | 北京拙河科技有限公司 | High-resolution video display method, device and system for on-demand transmission |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN111626947B (en) | Map vectorization sample enhancement method and system based on generation of countermeasure network | |
| CN110084241B (en) | Automatic ammeter reading method based on image recognition | |
| CN107808141A (en) | A kind of electric transmission line isolator explosion recognition methods based on deep learning | |
| CN112884791B (en) | Method for constructing large-scale remote sensing image semantic segmentation model training sample set | |
| CN110211046B (en) | Remote sensing image fusion method, system and terminal based on generation countermeasure network | |
| CN113989681B (en) | Remote sensing image change detection method and device, electronic equipment and storage medium | |
| CN103970901B (en) | A kind of geography information graph data integration method | |
| CN112597495B (en) | Malicious code detection method, system, equipment and storage medium | |
| CN111553224A (en) | Large remote sensing image block distribution method | |
| CN111738310B (en) | Material classification method, device, electronic equipment and storage medium | |
| CN104484869A (en) | Image matching method and system for ordinal measure features | |
| CN113723428A (en) | Image feature matching method, device and system and PCB visual detection equipment | |
| CN111435086B (en) | Navigation method and device based on splicing map | |
| CN112819823A (en) | Furniture board-oriented circular hole detection method, system and device | |
| CN111435537A (en) | Model training method and device and pose optimization method and device based on splicing map | |
| CN112508789A (en) | Residual error-based patch image enhancement identification method and device | |
| CN115760962A (en) | Object two-dimensional positioning method based on double thresholds | |
| CN115546780B (en) | License plate recognition method, model and device | |
| CN110967741A (en) | Method and system for detecting in-phase axis based on graphics | |
| CN112287740B (en) | Target detection method and device for power transmission line based on YOLOv3-tiny, and unmanned aerial vehicle | |
| CN117648296B (en) | Graphic data reading device and using method | |
| CN117746079B (en) | Clustering prediction method, system, storage medium and equipment for hyperspectral image | |
| CN113902980B (en) | Remote sensing target detection method based on content perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200818 |
|
| RJ01 | Rejection of invention patent application after publication |