CN117809202A - Bimodal target detection method and bimodal target detection system - Google Patents

Bimodal target detection method and bimodal target detection system Download PDF

Info

Publication number
CN117809202A
CN117809202A CN202410216848.1A CN202410216848A CN117809202A CN 117809202 A CN117809202 A CN 117809202A CN 202410216848 A CN202410216848 A CN 202410216848A CN 117809202 A CN117809202 A CN 117809202A
Authority
CN
China
Prior art keywords
image
image data
illumination
visible light
obtaining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410216848.1A
Other languages
Chinese (zh)
Other versions
CN117809202B (en
Inventor
李显巨
李尧
韩旭
常毅冉
冯健
丁慧君
张潇恺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Geosciences
Original Assignee
China University of Geosciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Geosciences filed Critical China University of Geosciences
Priority to CN202410216848.1A priority Critical patent/CN117809202B/en
Publication of CN117809202A publication Critical patent/CN117809202A/en
Application granted granted Critical
Publication of CN117809202B publication Critical patent/CN117809202B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a bimodal target detection method and a bimodal target detection system, and relates to the technical field of target detection, wherein the method comprises the following steps: obtaining bimodal target detection data according to the initial remote sensing image data, wherein the bimodal target detection data comprise visible light image data and non-visible light image data; obtaining depth characteristics of a visible light part according to the visible light image data; obtaining an illumination distribution image of the invisible light part according to the invisible light image data; according to the illumination distribution image, obtaining direct illumination characteristics and indirect illumination characteristics of each pixel point in the illumination distribution image; and performing target detection according to the depth characteristic, the direct illumination characteristic and the indirect illumination characteristic of the visible light part. According to the invention, the non-visible light image data is processed, the direct illumination characteristic and the indirect illumination characteristic of the non-visible light part in the initial remote sensing image data are extracted, the illumination characteristic in the visible light image data is enhanced, and the detection precision of the remote sensing target in the remote sensing image is improved.

Description

Bimodal target detection method and bimodal target detection system
Technical Field
The invention relates to the technical field of target detection, in particular to a bimodal target detection method and a bimodal target detection system.
Background
The target detection is to locate the remote sensing target in the remote sensing image, further complete identification and classification, and at present, the remote sensing image is generally directly input into a target detection model to realize identification and location of the remote sensing target in the remote sensing image.
In the prior art, in target detection of a remote sensing image, the remote sensing image is different from a natural image, the data volume of the remote sensing image is large, the background is complex, and the influence of a picture shooting angle, a terrain gradient and illumination reflection exists, so that the similar targets possibly present different characteristics under different scenes, noise is generated in a data set, and convergence of a depth model is influenced. Such as the effect of illumination factors on the depth model, where too high or too low an illumination intensity can make the feature difficult to identify. In addition, due to the influence of the sun irradiation angle, some ground objects are shielded by other ground objects to generate shadows in the remote sensing image, and the shadows react on the remote sensing image to generate a black spot, so that the target detection model cannot learn the characteristics of the complete ground objects, the problems of false detection and inaccurate detection are caused, and the detection precision and performance of the target detection model are seriously influenced.
Disclosure of Invention
The invention solves the technical problem of how to improve the detection precision of the remote sensing target in the remote sensing image.
The invention provides a bimodal target detection method, which comprises the following steps:
obtaining bimodal target detection data according to the initial remote sensing image data, wherein the bimodal target detection data comprises visible light image data and non-visible light image data;
obtaining depth characteristics of a visible light part in the initial remote sensing image data according to the visible light image data;
obtaining an illumination distribution image of a non-visible light part in the initial remote sensing image data according to the non-visible light image data through a Phong model and an IEMBP model;
obtaining direct illumination characteristics of each pixel point in the illumination distribution image according to the illumination distribution image;
constructing indirect illumination characteristics of each pixel point in the illumination distribution image according to the direct illumination characteristics;
and performing target detection according to the depth characteristic of the visible light part, the direct illumination characteristic and the indirect illumination characteristic.
Optionally, the obtaining bimodal target detection data according to the initial remote sensing image data includes:
classifying the initial remote sensing image data according to a spectrum band;
taking an RGB wave band image in the initial remote sensing image data as the visible light image data;
and taking a near infrared band image, a radar image and a digital elevation image in the initial remote sensing image data as the non-visible light image data.
Optionally, the obtaining the depth feature of the visible light part in the initial remote sensing image data according to the visible light image data includes:
inputting the RGB band image into a depth convolution network;
and obtaining the depth characteristic of the visible light part in the initial remote sensing image data through the output of the depth convolution network.
Optionally, the obtaining, according to the non-visible light image data, an illumination distribution image of the non-visible light part in the initial remote sensing image data through a Phong model and an IEMBP model includes:
dividing the illumination intensity of each pixel point of the invisible light image data into specular reflection light intensity, scattering light intensity and environment light intensity through the Phong model and the IEMBP model;
obtaining the specular reflection light intensity according to the digital elevation image;
obtaining scattered light intensity according to the radar image;
and obtaining an illumination distribution image according to the specular reflection light intensity and the scattering light intensity of each pixel point of the invisible light image data.
Optionally, the obtaining the specular reflection light intensity according to the digital elevation image includes:
obtaining the three-dimensional coordinates of each pixel point in the digital elevation image according to the two-dimensional coordinates of each pixel point in the preset coordinate system and the pixel value of each pixel point in the digital elevation image;
obtaining a three-dimensional curved surface of the digital elevation image according to the three-dimensional coordinates of each pixel point in the digital elevation image;
and carrying out dot product on the normal vector of each point in the three-dimensional curved surface and a tangential plane perpendicular to the normal vector and a light source of the near infrared band image to obtain the specular reflection light intensity of each pixel point in the digital elevation image.
Optionally, the obtaining the scattered light intensity according to the radar image includes:
adjusting the numerical value of each pixel point in the radar image through a residual error network, and taking the adjusted numerical value as a scattering coefficient of each pixel point in the radar image;
and carrying out dot product according to the scattering coefficient and the light source of the near infrared band image to obtain the scattering light intensity of each pixel point in the radar image.
Optionally, the obtaining the direct illumination characteristic of each pixel point in the illumination distribution image according to the illumination distribution image includes:
obtaining the environment light intensity through self-adaptive adjustment of preset learnable super parameters;
and adding the specular reflection light intensity, the scattering light intensity and the environment light intensity in the illumination distribution image to obtain the direct illumination characteristic of each pixel point in the illumination distribution image.
Optionally, the constructing an indirect illumination feature of each pixel point in the illumination distribution image according to the direct illumination feature includes:
inputting the direct illumination characteristics into a graph network to obtain a weight matrix and an adjacent matrix of the direct illumination characteristics of each pixel point in the illumination distribution image;
and obtaining the indirect illumination characteristic of each pixel point in the illumination distribution image according to the weight matrix, the adjacent matrix and the activation function.
Optionally, the detecting the target according to the depth feature of the visible light part, the direct illumination feature and the indirect illumination feature includes:
performing dot product according to the direct illumination characteristic and the indirect illumination characteristic to obtain the integral illumination characteristic of the non-visible light part in the initial remote sensing image data;
obtaining object features in the initial remote sensing image data according to the product of the integral illumination features and the depth features;
and detecting the target according to the object characteristics.
The invention also provides a bimodal target detection system comprising a computer readable storage medium storing a computer program and a processor, the computer program implementing the bimodal target detection method as described above when read and run by the processor.
According to the bimodal target detection method and system, initial remote sensing image data are divided into visible light image data and non-visible light image data, the non-visible light image data are processed by utilizing a Phong model and an IEMBP model, direct illumination characteristics and indirect illumination characteristics of a non-visible light part in the initial remote sensing image data are extracted, illumination characteristics in the visible light image data are enhanced by utilizing the IEMBP model, noise caused by uneven illumination and shadow in the initial remote sensing image data is reduced, the problems of inaccurate target detection boundary and false detection are reduced, the influence of illumination conditions in an image on target detection is reduced, finally the direct illumination characteristics and the indirect illumination characteristics are fused with depth characteristics of a visible light part to be used for target detection, the image data of the visible light part in the initial remote sensing image data are enhanced by the direct illumination characteristics and the indirect illumination characteristics, and the detection accuracy of remote sensing targets in the remote sensing image is improved.
Drawings
FIG. 1 is a flow chart of a bimodal target detection method according to an embodiment of the invention;
FIG. 2 is a schematic block diagram of a bimodal target detection method according to another embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the present invention provides a bimodal target detection method, including:
s1: and obtaining bimodal target detection data according to the initial remote sensing image data, wherein the bimodal target detection data comprises visible light image data and non-visible light image data.
Specifically, initial remote sensing image data can be obtained from unmanned aerial vehicles, satellites or other rocker devices, the initial remote sensing image data comprises multispectral detection data, classification is carried out according to the multispectral image data, and bimodal target detection data are obtained, wherein the bimodal target detection data comprise visible light image data and non-visible light image data, the visible light data are true color images, and the images comprise more shadows and areas with insufficient illumination.
S2: and obtaining the depth characteristics of the visible light part in the initial remote sensing image data according to the visible light image data.
Specifically, the depth features of the visible light part are acquired according to the visible light image data, the features can be extracted by using a pre-trained convolutional neural network model according to the visible light image data, and the depth features capture advanced semantic information in the image, such as edges, textures, shapes and the like, so that the tasks of identifying, classifying, detecting targets and the like of the image can be facilitated, and more accurate results can be obtained in the field of image analysis.
S3: and obtaining an illumination distribution image of the non-visible light part in the initial remote sensing image data according to the non-visible light image data through a Phong model and an IEMBP model.
Specifically, an illumination distribution image is obtained according to non-visible light image data, a Phong model and an IEMBP model are utilized, wherein the Phong model is a classical illumination model and is used for calculating the influence of illumination on the surface of an object, the influence comprises scattered light, specular reflection light and ambient light, and the Phong model can be utilized for simulating the illumination effect. On the basis, the illumination depth features in the image are further enhanced and extracted by applying an IEMBP model (Illumination Enhanced Module Based on Phong shader-an illumination enhancement model based on a Phong shader, so that an illumination distribution image of a non-visible light part in the initial remote sensing image data is obtained.
S4: and obtaining the direct illumination characteristic of each pixel point in the illumination distribution image according to the illumination distribution image.
Specifically, the direct illumination of each pixel point in the illumination distribution image includes specular reflection light, scattered light and ambient light, and a corresponding direct illumination characteristic is calculated according to each pixel point in the illumination distribution image.
S5: and constructing indirect illumination characteristics of each pixel point in the illumination distribution image according to the direct illumination characteristics.
Specifically, according to the direct illumination characteristic, an indirect illumination characteristic of each pixel point in the illumination distribution image is constructed. Considering the indirect illumination in illumination, that is, the light of one point can be refracted or scattered to another point, therefore, modeling is required to be performed on the indirect illumination of each pixel point according to the direct illumination characteristic of each pixel point, so as to obtain the indirect illumination characteristic of each pixel point.
S6: and performing target detection according to the depth characteristic of the visible light part, the direct illumination characteristic and the indirect illumination characteristic.
Specifically, in the initial remote sensing image data, due to the relationship of the sun direct angle, shadows exist in a photographed scene, and the shadows block other objects, so that the texture features of the objects displayed on the image are blurred or removed. The image data of the visible light part in the initial remote sensing image data is enhanced through the direct illumination characteristic and the indirect illumination characteristic, the influence of shadows caused by the direct solar angle is reduced, and the target detection is carried out on the basis.
According to the bimodal target detection method, initial remote sensing image data are divided into visible light image data and non-visible light image data, the non-visible light image data are processed by utilizing a Phong model and an IEMBP model, direct illumination characteristics and indirect illumination characteristics of a non-visible light part in the initial remote sensing image data are extracted, illumination characteristics in the visible light image data are enhanced by utilizing the IEMBP model, noise caused by uneven illumination and shadow in the initial remote sensing image data is reduced, the problems of inaccurate target detection boundary and false detection are reduced, the influence of illumination conditions in an image on target detection is reduced, finally the direct illumination characteristics and the indirect illumination characteristics are fused with depth characteristics of a visible light part to be used for target detection, the image data of the visible light part in the initial remote sensing image data are enhanced by the direct illumination characteristics and the indirect illumination characteristics, and the detection accuracy of the remote sensing target in the remote sensing image is improved.
In the embodiment of the present invention, the obtaining bimodal target detection data according to the initial remote sensing image data includes:
classifying the initial remote sensing image data according to a spectrum band;
taking an RGB wave band image in the initial remote sensing image data as the visible light image data;
and taking a near infrared band image, a radar image and a digital elevation image in the initial remote sensing image data as the non-visible light image data.
In this embodiment, in the given initial remote sensing image data, for the construction of the bimodal target detection data, the image data may be classified and extracted according to the difference of the spectral bands. As shown in fig. 2, red, green and blue band images in the initial remote sensing image data are taken as visible light image data, that is, RGB in fig. 2. The Near Infrared band image, the radar image, and the digital elevation image are used as non-visible light image data, and correspond to NIR (Near Infrared), SAR (Synthetic Aperture Radar-synthetic aperture radar), and DEM (Digital Elevation Model-digital elevation model) in FIG. 2, respectively. For visible light image data (RGB band images) and non-visible light image data (including near infrared, radar, digital elevation images), a corresponding object detection algorithm may be used to extract bimodal object detection data.
According to the bimodal target detection method, the initial remote sensing image data is classified according to the spectrum wave bands, the data is extracted according to the characteristics of visible light and non-visible light, and bimodal target detection data can be constructed. Information of different wave bands can be fully utilized when the target detection task is carried out, and performance and robustness of the target detection model under a complex environment can be improved.
In the embodiment of the present invention, the obtaining the depth feature of the visible light portion in the initial remote sensing image data according to the visible light image data includes:
inputting the RGB band image into a depth convolution network;
and obtaining the depth characteristic of the visible light part in the initial remote sensing image data through the output of the depth convolution network.
In this embodiment, as shown in fig. 2, an image of RGB bands is input into a depth convolution network to extract depth features from the image. After the RGB image is processed by the deep convolution network, the high-level semantic features of the image, including edges, textures, shapes and the like, can be obtained, and the analysis and understanding of the image are facilitated. The deep convolution network consists of a plurality of convolution layers and pooling layers, and low-level features of the image gradually evolve into high-level abstract features through multiple convolution and pooling operations, so that the task related to the image is solved.
According to the bimodal target detection method, the RGB wave band image is input into the depth convolution network, and the depth characteristic in the image can be obtained by reading and processing the output of the depth convolution network. These features have important roles in image recognition, classification and segmentation tasks, helping to improve the performance of image analysis.
In the embodiment of the present invention, the obtaining, by using the Phong model and the IEMBP model, the illumination distribution image of the non-visible light portion in the initial remote sensing image data according to the non-visible light image data includes:
dividing the illumination intensity of each pixel point of the invisible light image data into specular reflection light intensity, scattering light intensity and environment light intensity through the Phong model and the IEMBP model;
obtaining the specular reflection light intensity according to the digital elevation image;
obtaining scattered light intensity according to the radar image;
and obtaining an illumination distribution image according to the specular reflection light intensity and the scattering light intensity of each pixel point of the invisible light image data.
In this example, the illumination intensity is divided into three components by the Phong model: scattered light, ambient light, and specularly reflected light. The ambient light refers to uniform illumination formed by light rays emitted by the light source after multiple reflections in the space; scattered light refers to the diffusely reflected portion of the object surface to the light source; specular reflection refers to specular light produced by light striking a smooth surface. The specular reflection intensity is obtained from a digital elevation image that provides three-dimensional information of the earth's surface, which in the preferred embodiment of the invention, in conjunction with fig. 2, may simulate the bounce (specular reflection) of the sun upon irradiation of the earth, such information being important for identifying surface features such as rocks, bodies of water, etc., in accordance with the DEM. SAR is used to detect the geometry of the earth and the electromagnetic properties of certain materials. The scattered light intensity is determined by analyzing the intensity of the scattering effect of the radar wave due to the surface features in the radar image. The intensity of specular and scattered light can be estimated for each pixel in the image separately by the Phong model and the IEMBP model. And then combining the two illumination intensities to obtain an illumination distribution image comprehensively considering the terrain influence, the object surface characteristics and the radar scattering characteristics.
According to the bimodal target detection method, the response of the physical characteristics of the earth surface to illumination is extracted and analyzed from the invisible light image through the Phong model and the IEMBP model.
In an embodiment of the present invention, the obtaining the specular reflection light intensity according to the digital elevation image includes:
obtaining the three-dimensional coordinates of each pixel point in the digital elevation image according to the two-dimensional coordinates of each pixel point in the preset coordinate system and the pixel value of each pixel point in the digital elevation image;
obtaining a three-dimensional curved surface of the digital elevation image according to the three-dimensional coordinates of each pixel point in the digital elevation image;
and carrying out dot product on the normal vector of each point in the three-dimensional curved surface and a tangential plane perpendicular to the normal vector and a light source of the near infrared band image to obtain the specular reflection light intensity of each pixel point in the digital elevation image.
In the present embodiment, in order to calculate specular reflection light, first, two-dimensional coordinates of each pixel point in the DEM are acquired, for example:wherein->Representing the width of the image and h representing the height of the image. Changing each two-dimensional point into a three-dimensional point +.>Wherein->The pixel values representing the points, i.e. three-dimensional point data, i.e. the 3D points in fig. 2, can be obtained. Thus, the whole image is changed from a two-dimensional plane to a curved surface in a three-dimensional space. After the curved surface in the three-dimensional space is obtained, the normal vector of each point in the curved surface can be calculated. The normal vector of each point is perpendicular to the tangent plane of that point. The illumination intensity of the specular reflection light of the point can be calculated through dot product with the light source. Wherein, in conjunction with fig. 2, the light source of the near infrared band image (NIR) is a simulated light source through the near infrared band image, and then the difference between the simulated light source and the real illumination is corrected through a residual block.
According to the bimodal target detection method, the three-dimensional coordinates of each pixel point in the digital elevation image can be obtained through the two-dimensional coordinates and the pixel value of the pixel point, and then the three-dimensional curved surface of the digital elevation image is obtained. And then, carrying out dot product by utilizing the normal vector of each point in the curved surface and a tangential plane perpendicular to the normal vector and combining a light source, so that the specular reflection light intensity of each pixel point can be obtained. By analyzing the information of the digital elevation image and the light source, the rapid measurement and analysis of the specular reflection light intensity of the specific area of the earth surface can be realized, and the environment condition and the characteristics of the specific area of the earth surface can be known.
In an embodiment of the present invention, the obtaining the scattered light intensity according to the radar image includes:
adjusting the numerical value of each pixel point in the radar image through a residual error network, and taking the adjusted numerical value as a scattering coefficient of each pixel point in the radar image;
and carrying out dot product according to the scattering coefficient and the light source of the near infrared band image to obtain the scattering light intensity of each pixel point in the radar image.
In this embodiment, for the radar image, since the radar has higher penetration capability, if the weaker the numerical value in the SAR data is, the stronger the absorption capability of the object is, reflecting the physical property of the object, and having higher absorption rate for energy. Meaning that when the light irradiates the object, most of the light will be absorbed, the scattered light is weaker and the scattering coefficient is lower. The magnitude of the scattering coefficient is not completely accurately reflected in the radar image. The values in the radar image are adjusted using the residual network block. In the illumination model used in the method, the scattering intensity of the illumination in all directions of the hemisphere is considered to be uniform. The method assumes the adjusted radar image as the scattering coefficient of the object in the image. Referring to fig. 2, the light source term is used to perform dot product with the radar image after the adjustment of the residual block data, so as to obtain the scattered light intensity.
According to the bimodal target detection method, the residual error network block is utilized to adjust the numerical value in the radar image, so that a more accurate scattering coefficient is obtained. In the illumination model, it is assumed that the scattering intensity of the illumination in various directions of the hemisphere is uniform. The scattered light intensity of each pixel point can be obtained by dot product of the light source item and the radar image adjusted by the residual network.
In the embodiment of the present invention, the obtaining the direct illumination characteristic of each pixel point in the illumination distribution image according to the illumination distribution image includes:
obtaining the environment light intensity through self-adaptive adjustment of preset learnable super parameters;
and adding the specular reflection light intensity, the scattering light intensity and the environment light intensity in the illumination distribution image to obtain the direct illumination characteristic of each pixel point in the illumination distribution image.
In this embodiment, the ambient light generally refers to a light source with no specific directivity, which is from all directions of the surrounding environment, so that the ambient light is smooth and uniform for the entire scene, and when the ambient light intensity is acquired, the super-parameters need to be adaptively adjusted by an optimization algorithm. As shown in connection with fig. 2, the ambient light intensity is obtained by learning the superparameter and providing a convolution block. The direct illumination characteristic of each pixel point in the illumination distribution image can be obtained by adding, and in the embodiment of the present invention, the direct illumination characteristic of each pixel point can be calculated by a light intensity formula:
；
wherein,for specularly reflecting the light intensity>For scattering light intensity +.>Is the ambient light intensity.
According to the bimodal target detection method, the direct illumination characteristics of each pixel point in the illumination distribution image obtained by the method can reflect the real illumination condition more accurately.
In the embodiment of the present invention, the constructing an indirect illumination characteristic of each pixel point in the illumination distribution image according to the direct illumination characteristic includes:
inputting the direct illumination characteristics into a graph network to obtain a weight matrix and an adjacent matrix of the direct illumination characteristics of each pixel point in the illumination distribution image;
and obtaining the indirect illumination characteristic of each pixel point in the illumination distribution image according to the weight matrix, the adjacent matrix and the activation function.
In this embodiment, as shown in fig. 2, the direct illumination feature is input into a graph network to obtain a weight matrix and an adjacent matrix of the direct illumination feature of each pixel point in the illumination distribution image, and finally output through a pyramid network; wherein, based on the graph network, the calculation mode is as follows:
；
wherein,for the weight matrix, fine tuning by learning, +.>For the regularized adjacency matrix, +.>Is->The characteristics of the layers ultimately result in illumination characteristics that combine direct illumination and indirect illumination.
According to the bimodal target detection method, the direct illumination characteristics are input into the graph network, so that the weight matrix and the adjacent matrix of the direct illumination characteristics of each pixel point can be obtained through learning. The weights and the adjacency relations can help the model to understand the association between the pixel points, so that the propagation and influence of illumination in a scene can be simulated more truly, and the illumination distribution image is more realistic.
In an embodiment of the present invention, the performing object detection according to the depth feature of the visible light portion, the direct illumination feature, and the indirect illumination feature includes:
performing dot product according to the direct illumination characteristic and the indirect illumination characteristic to obtain the integral illumination characteristic of the non-visible light part in the initial remote sensing image data;
obtaining object features in the initial remote sensing image data according to the product of the integral illumination features and the depth features;
and detecting the target according to the object characteristics.
In this embodiment, the original image features are element multiplied with the illumination features.
；
Representing the characteristics of an object->Representing depth features>Representing the overall illumination characteristics. If no direct solar angle is affected, the characteristic of a certain point in the image is obvious and the value is large, but in actual situation, the shadow is covered during shooting, and the shadow place in the image is +.>Smaller, the original characteristic size can be reflected by the illumination characteristics constructed by the illumination model,/-degree>The method is relatively large, and features of the object in the original image can be amplified by a multiplication method, so that the texture features of the object are more obvious and are easier to distinguish.
According to the bimodal target detection method, the image data of the visible light part in the initial remote sensing image data is enhanced through the direct illumination characteristic and the indirect illumination characteristic, and the detection precision of the remote sensing target in the remote sensing image is improved.
The invention also provides a bimodal target detection system comprising a computer readable storage medium storing a computer program and a processor, the computer program implementing the bimodal target detection method as described above when read and run by the processor.
According to the bimodal target detection system, initial remote sensing image data are divided into visible light image data and non-visible light image data, the non-visible light image data are processed by utilizing a Phong model and an IEMBP model, direct illumination characteristics and indirect illumination characteristics of a non-visible light part in the initial remote sensing image data are extracted, illumination characteristics in the visible light image data are enhanced by utilizing the IEMBP model, noise caused by uneven illumination and shadow in the initial remote sensing image data is reduced, the problems of inaccurate target detection boundary and false detection are reduced, the influence of illumination conditions in an image on target detection is reduced, finally the direct illumination characteristics and the indirect illumination characteristics are fused with depth characteristics of a visible light part to be used for target detection, the image data of the visible light part in the initial remote sensing image data are enhanced by the direct illumination characteristics and the indirect illumination characteristics, and the detection accuracy of the remote sensing target in the remote sensing image is improved.
Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A bimodal target detection method, comprising:
obtaining bimodal target detection data according to the initial remote sensing image data, wherein the bimodal target detection data comprises visible light image data and non-visible light image data;
obtaining depth characteristics of a visible light part in the initial remote sensing image data according to the visible light image data;
obtaining an illumination distribution image of a non-visible light part in the initial remote sensing image data according to the non-visible light image data through a Phong model and an IEMBP model;
obtaining direct illumination characteristics of each pixel point in the illumination distribution image according to the illumination distribution image;
constructing indirect illumination characteristics of each pixel point in the illumination distribution image according to the direct illumination characteristics;
and performing target detection according to the depth characteristic of the visible light part, the direct illumination characteristic and the indirect illumination characteristic.
2. The method for detecting a bimodal target according to claim 1, wherein obtaining bimodal target detection data from the initial remote sensing image data comprises:
classifying the initial remote sensing image data according to a spectrum band;
taking an RGB wave band image in the initial remote sensing image data as the visible light image data;
and taking a near infrared band image, a radar image and a digital elevation image in the initial remote sensing image data as the non-visible light image data.
3. The method for detecting a bimodal target according to claim 2, wherein the obtaining depth features of a visible light portion in the initial remote sensing image data according to the visible light image data comprises:
inputting the RGB band image into a depth convolution network;
and obtaining the depth characteristic of the visible light part in the initial remote sensing image data through the output of the depth convolution network.
4. The method for detecting a bimodal target according to claim 2, wherein the obtaining, by using a Phong model and an IEMBP model, an illumination distribution image of a non-visible light portion in the initial remote sensing image data according to the non-visible light image data includes:
dividing the illumination intensity of each pixel point of the invisible light image data into specular reflection light intensity, scattering light intensity and environment light intensity through the Phong model and the IEMBP model;
obtaining the specular reflection light intensity according to the digital elevation image;
obtaining scattered light intensity according to the radar image;
and obtaining an illumination distribution image according to the specular reflection light intensity and the scattering light intensity of each pixel point of the invisible light image data.
5. The method of claim 4, wherein obtaining the specular reflection light intensity from the digital elevation image comprises:
obtaining the three-dimensional coordinates of each pixel point in the digital elevation image according to the two-dimensional coordinates of each pixel point in the preset coordinate system and the pixel value of each pixel point in the digital elevation image;
obtaining a three-dimensional curved surface of the digital elevation image according to the three-dimensional coordinates of each pixel point in the digital elevation image;
and carrying out dot product on the normal vector of each point in the three-dimensional curved surface and a tangential plane perpendicular to the normal vector and a light source of the near infrared band image to obtain the specular reflection light intensity of each pixel point in the digital elevation image.
6. The method of claim 4, wherein obtaining scattered light intensity from the radar image comprises:
adjusting the numerical value of each pixel point in the radar image through a residual error network, and taking the adjusted numerical value as a scattering coefficient of each pixel point in the radar image;
and carrying out dot product according to the scattering coefficient and the light source of the near infrared band image to obtain the scattering light intensity of each pixel point in the radar image.
7. The method according to claim 4, wherein the obtaining the direct illumination characteristic of each pixel point in the illumination distribution image according to the illumination distribution image includes:
obtaining the environment light intensity through self-adaptive adjustment of preset learnable super parameters;
and adding the specular reflection light intensity, the scattering light intensity and the environment light intensity in the illumination distribution image to obtain the direct illumination characteristic of each pixel point in the illumination distribution image.
8. The method according to claim 2, wherein the constructing an indirect illumination feature of each pixel point in the illumination distribution image according to the direct illumination feature comprises:
inputting the direct illumination characteristics into a graph network to obtain a weight matrix and an adjacent matrix of the direct illumination characteristics of each pixel point in the illumination distribution image;
and obtaining the indirect illumination characteristic of each pixel point in the illumination distribution image according to the weight matrix, the adjacent matrix and the activation function.
9. The method according to claim 2, wherein the performing object detection according to the depth feature of the visible light portion, the direct illumination feature, and the indirect illumination feature comprises:
performing dot product according to the direct illumination characteristic and the indirect illumination characteristic to obtain the integral illumination characteristic of the non-visible light part in the initial remote sensing image data;
obtaining object features in the initial remote sensing image data according to the product of the integral illumination features and the depth features;
and detecting the target according to the object characteristics.
10. A bimodal target detection system comprising a computer readable storage medium storing a computer program and a processor, the computer program when read and executed by the processor implementing a bimodal target detection method as claimed in any one of claims 1 to 9.
CN202410216848.1A 2024-02-28 2024-02-28 Bimodal target detection method and bimodal target detection system Active CN117809202B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410216848.1A CN117809202B (en) 2024-02-28 2024-02-28 Bimodal target detection method and bimodal target detection system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410216848.1A CN117809202B (en) 2024-02-28 2024-02-28 Bimodal target detection method and bimodal target detection system

Publications (2)

Publication Number Publication Date
CN117809202A true CN117809202A (en) 2024-04-02
CN117809202B CN117809202B (en) 2024-05-31

Family

ID=90434856

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410216848.1A Active CN117809202B (en) 2024-02-28 2024-02-28 Bimodal target detection method and bimodal target detection system

Country Status (1)

Country Link
CN (1) CN117809202B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180137635A1 (en) * 2015-06-08 2018-05-17 Commissariat A L'energie Atomique Et Aux Energies Alternatives Method for processing images having specularities and corresponding computer program product
CN109507685A (en) * 2018-10-15 2019-03-22 天津大学 The distance measuring method of the TOF sensor model of phong formula illumination model
EP3757890A1 (en) * 2019-06-28 2020-12-30 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and device for image processing, method and device for training object detection model
WO2023274254A1 (en) * 2021-06-29 2023-01-05 上海高德威智能交通系统有限公司 Object detection method, apparatus and system, electronic device, and storage medium
CN115775303A (en) * 2023-02-13 2023-03-10 南京航空航天大学 High-reflectivity object three-dimensional reconstruction method based on deep learning and illumination model
WO2023130650A1 (en) * 2022-01-04 2023-07-13 苏州浪潮智能科技有限公司 Image restoration method and apparatus, electronic device, and storage medium
CN117373138A (en) * 2023-11-02 2024-01-09 厦门熵基科技有限公司 Cross-modal living fusion detection method and device, storage medium and computer equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180137635A1 (en) * 2015-06-08 2018-05-17 Commissariat A L'energie Atomique Et Aux Energies Alternatives Method for processing images having specularities and corresponding computer program product
CN109507685A (en) * 2018-10-15 2019-03-22 天津大学 The distance measuring method of the TOF sensor model of phong formula illumination model
EP3757890A1 (en) * 2019-06-28 2020-12-30 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and device for image processing, method and device for training object detection model
WO2023274254A1 (en) * 2021-06-29 2023-01-05 上海高德威智能交通系统有限公司 Object detection method, apparatus and system, electronic device, and storage medium
WO2023130650A1 (en) * 2022-01-04 2023-07-13 苏州浪潮智能科技有限公司 Image restoration method and apparatus, electronic device, and storage medium
CN115775303A (en) * 2023-02-13 2023-03-10 南京航空航天大学 High-reflectivity object three-dimensional reconstruction method based on deep learning and illumination model
CN117373138A (en) * 2023-11-02 2024-01-09 厦门熵基科技有限公司 Cross-modal living fusion detection method and device, storage medium and computer equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GAODIAN ZHOU 等: "Split Depth-Wise Separable Graph-Convolution Network for Road Extraction in Complex Environments From High-Resolution Remote-Sensing Images", IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 31 December 2022 (2022-12-31), pages 5614115 *

Also Published As

Publication number Publication date
CN117809202B (en) 2024-05-31

Similar Documents

Publication Publication Date Title
Grau et al. Estimation of 3D vegetation density with Terrestrial Laser Scanning data using voxels. A sensitivity analysis of influencing parameters
KR102018789B1 (en) evaluation method of topographic normalization models, Method AND Apparatus for topographic correction of normalized vegetation index map BY USING the SAME
US20100002910A1 (en) Method and Apparatus for Developing Synthetic Three-Dimensional Models from Imagery
CN108280440A (en) A kind of fruit-bearing forest recognition methods and system
CN109376600A (en) Multi-spectrum remote sensing image comprehensive characteristics cloud detection method of optic and device
Malmgren-Hansen et al. Convolutional neural networks for SAR image segmentation
CN103871039B (en) Generation method for difference chart in SAR (Synthetic Aperture Radar) image change detection
CN104050643B (en) Remotely sensing image geometric and radiation integrated relative correction method and system
Perrin et al. 2D and 3D vegetation resource parameters assessment using marked point processes
CN116295285A (en) Shallow sea water depth remote sensing inversion method based on region self-adaption
CN115761463A (en) Shallow sea water depth inversion method, system, equipment and medium
Gichu et al. Segmentation of dust storm areas on Mars images using principal component analysis and neural network
Tao et al. Deep Learning for Hyperspectral Image Analysis and Classification
CN117115669B (en) Object-level ground object sample self-adaptive generation method and system with double-condition quality constraint
Wang et al. Lithological information extraction and classification in hyperspectral remote sensing data using Backpropagation Neural Network
CN117809202B (en) Bimodal target detection method and bimodal target detection system
Wang et al. Classification and extent determination of rock slope using deep learning
Kuter et al. Modern applied mathematics for alternative modeling of the atmospheric effects on satellite images
Ye et al. Building detection by local region features in SAR images
Bartoš Cloud and shadow detection in satellite imagery
KR20170122935A (en) Equipment and Method for topographically corrected image generation using quantitative analysis and Apparatus Thereof
Nimbalkar et al. Optimal band configuration for the roof surface characterization using hyperspectral and LiDAR imaging
CN115546393A (en) Open GL-based infrared virtual scene radiance drawing method
Schott et al. Incorporation of texture in multispectral synthetic image generation tools
Barcomb et al. High-resolution slant-angle scene generation and validation of concealed targets in DIRSIG

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CB03 Change of inventor or designer information

Inventor after: Li Xianju

Inventor after: Li Yao

Inventor after: Han Xu

Inventor after: Chang Yiran

Inventor after: Feng Jian

Inventor after: Ding Huijun

Inventor after: Zhang Xiaokai

Inventor before: Li Xianju

Inventor before: Li Yao

Inventor before: Han Xu

Inventor before: Chang Yiran

Inventor before: Feng Jian

Inventor before: Ding Huijun

Inventor before: Zhang Xiaokai