CN117889867B - Path planning method based on local self-attention moving window algorithm - Google Patents
Path planning method based on local self-attention moving window algorithm Download PDFInfo
- Publication number
- CN117889867B CN117889867B CN202410304943.7A CN202410304943A CN117889867B CN 117889867 B CN117889867 B CN 117889867B CN 202410304943 A CN202410304943 A CN 202410304943A CN 117889867 B CN117889867 B CN 117889867B
- Authority
- CN
- China
- Prior art keywords
- attention
- self
- image
- layer
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000011218 segmentation Effects 0.000 claims abstract description 78
- 230000033001 locomotion Effects 0.000 claims abstract description 29
- 239000011159 matrix material Substances 0.000 claims abstract description 21
- 230000004913 activation Effects 0.000 claims abstract description 15
- 238000010586 diagram Methods 0.000 claims abstract description 14
- 238000003708 edge detection Methods 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000000605 extraction Methods 0.000 claims description 75
- 238000004364 calculation method Methods 0.000 claims description 43
- 230000006870 function Effects 0.000 claims description 41
- 230000004927 fusion Effects 0.000 claims description 41
- 238000010606 normalization Methods 0.000 claims description 36
- 230000009191 jumping Effects 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 7
- 230000003213 activating effect Effects 0.000 claims description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 3
- 230000002708 enhancing effect Effects 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M1/00—Testing static or dynamic balance of machines or structures
- G01M1/12—Static balancing; Determining position of centre of gravity
- G01M1/122—Determining position of centre of gravity
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P13/00—Indicating or recording presence, absence, or direction, of movement
- G01P13/02—Indicating direction only, e.g. by weather vane
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Aviation & Aerospace Engineering (AREA)
- Computational Linguistics (AREA)
- Automation & Control Theory (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a path planning method based on a local self-attention moving window algorithm, which comprises the steps of obtaining an RGB image of a current working environment, preprocessing and converting the RGB image into a gray scale image; after edge detection, a binarized image is obtained, and an edge self-attention weight is obtained through an activation function; obtaining curvature self-attention weight according to entropy of sea plug matrix eigenvalue of the image block after dividing the gray level diagram; the self-supervision attention semantic segmentation network is improved through the edge self-attention weight and the curvature self-attention weight, an image containing semantic information is obtained through the improved network, and the movement direction of the obstacle is predicted according to the images obtained at continuous moments, so that the movement direction of the robot is selected. Introducing entropy of the sea plug matrix to obtain new self-attention weight, and enhancing segmentation of the object edges; the self-attention weight obtained by edge detection is introduced, the segmentation of a part with larger color change in the image is enhanced, and the accuracy of edge segmentation is improved, so that the success rate of obstacle avoidance is improved.
Description
Technical Field
The invention relates to robot vision path planning, in particular to a path planning method based on a local self-attention moving window algorithm.
Background
With the continuous development of computer vision technology, semantic segmentation based on deep learning is also widened in the field of robot vision path planning. The technology can utilize the deep learning model to sense the local environment in real time, and further improves the recognition accuracy of the obstacle through semantic information, so that more reliable environment sensing data is provided for path planning. However, the existing partial path planning technology still has the problems of inaccurate identification target and inaccurate edge segmentation.
At present, path planning is used as a key technology in the fields of autonomous mobile robots, automatic driving vehicles and the like, and aims to realize safe and efficient movement in complex and dynamic environments. The traditional path planning method is mainly based on a classical planning algorithm, and has good effect in a static environment, but is difficult to process environmental information changing in real time in a dynamic environment. Along with the continuous development of artificial intelligence technologies such as deep learning, reinforcement learning and the like, a new breakthrough is made to the path planning technology. The deep learning method can better understand environmental characteristics and dynamic changes by learning a large amount of real scene data, so that the robustness and adaptability of path planning are improved. However, the current path planning method adopting deep learning has the problems of dimensional change and self movement of the obstacle under the dynamic complex condition, and the problem of error in obstacle avoidance caused by inaccurate edge segmentation is easy to occur.
Disclosure of Invention
The invention aims to: aiming at the defects, the invention provides the path planning method based on the local self-attention moving window algorithm, which has the advantages of strong instantaneity, high barrier segmentation accuracy and strong background interference resistance.
The technical scheme is as follows: in order to solve the problems, the invention adopts a path planning method based on a local self-attention moving window algorithm, which comprises the following steps:
(1) Acquiring an RGB image of a current operation environment, and preprocessing to acquire an RGB image of the preprocessed operation environment;
(2) Converting the RGB image of the preprocessed working environment into a gray scale image;
(3) Performing edge detection on the converted gray level image to obtain a binary image containing edge characteristic information, and obtaining the edge self-attention weight of the binary image through an activation function; dividing the converted gray image to obtain a plurality of gray image blocks, calculating entropy of sea plug matrix eigenvalues corresponding to each divided gray image block, and calculating curvature self-attention weight of each gray image block through an activation function according to the entropy;
(4) The self-supervision attention semantic segmentation network is improved through edge self-attention weights and curvature self-attention weights, the self-supervision attention semantic segmentation network comprises two continuous self-attention window layers, and the edge self-attention weights and the curvature self-attention weights are respectively added into the attention calculation of the two continuous self-attention window layers;
(5) The RGB image of the operation environment after preprocessing is processed through the improved self-supervision attention semantic segmentation network, the operation environment RGB image containing semantic information is obtained, the position of an obstacle substance center is calculated according to the operation environment RGB image containing semantic information obtained at continuous moments, the movement direction of the obstacle is predicted, and the movement direction of the obstacle is selected according to the predicted movement direction of the obstacle.
Further, the preprocessing of the RGB image of the current working environment in the step (1) includes scaling the RGB image of the current working environment, and then performing flipping, affine transformation and noise addition on the scaled RGB image to obtain the RGB image of the preprocessed working environment.
Further, in the step (3), performing edge detection on the transformed gray scale image to obtain a binary image containing edge feature information, and obtaining the edge self-attention weight of the binary image through an activation function includes:
(3.11) carrying out smoothing treatment on the gray level image, then calculating the gradient amplitude of each pixel in the image, carrying out maximum value inhibition operation on the gradient amplitude, and finally carrying out edge detection through a double-threshold algorithm to obtain a binarized image containing edge information;
(3.12) smoothing the confidence coefficient of each pixel point in the binarized image containing the edge information, wherein the confidence coefficient is calculated according to the formula:
；
wherein, Represents the/>Confidence of individual pixel points,/>Representing the/>, in a binarized image containing edge informationGray value of each pixel/(Representing the/>, in a binarized image containing edge informationPixel dot and/>Distance of individual pixel points,/>Divider/>, representing a binarized image containing edge informationFirst pixel out ofConfidence of individual pixel points,/>Representing the total number of pixel points in the binarized image containing the edge information;
(3.13) converting the confidence into edge self-attention weight of each pixel point through an activation function, wherein the calculation formula is as follows:
；
wherein, Edge self-attention weight value representing first pixel point in binarized image containing edge information,/>For/>The function is activated.
Further, the calculation formula of the entropy of the sea plug matrix eigenvalue corresponding to each gray image block after segmentation in the step (3) is as follows:
；
wherein, Represents the/>Entropy of sea plug matrix eigenvalue corresponding to each gray image block,/>Representing the number of pixels in each gray image block,/>Represents the/>And the characteristic values of the sea plug matrix corresponding to the gray image blocks.
Further, the calculation formula of the curvature self-attention weight in the step (3) is as follows:
；
wherein, Represents the/>Curvature self-attention weight corresponding to each gray image block,/>The total number of the gray image blocks after the segmentation processing is defined.
Further, the processing the RGB image of the pre-processed working environment through the improved self-supervision attention semantic segmentation network in the step (5) includes:
(5.1) reducing the RGB image of the preprocessed working environment, and inputting the images before and after reduction into an image block segmentation layer from an original image channel and a reduced image channel respectively for segmentation to obtain a plurality of non-overlapping image blocks respectively;
(5.2) inputting the image blocks obtained by segmentation into a first linear self-attention feature extraction module, wherein the first linear self-attention feature extraction module comprises a linear embedding layer and two continuous self-attention window layers, and obtaining a feature map after the first feature extraction;
(5.3) sequentially inputting the feature map after the first feature extraction into three identical fusion self-attention feature extraction modules, wherein each fusion self-attention feature extraction module comprises a patch fusion layer and two continuous self-attention window layers, and obtaining a feature map after the fourth feature extraction;
and (5.4) inputting the feature images obtained in the original image channel and the contracted image channel after the feature extraction of each time into a decoder for semantic segmentation after up-sampling and jump connection operation, and obtaining the operation environment RGB image containing semantic information.
Further, the patch fusion layer of the first fused self-attention feature extraction module includes:
extracting the first characteristic in the original image channel Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the second feature extraction; Feature map/>, after first feature extraction, in a contracted map channelInput to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the second feature extraction，/>For image depth,/>Is the height of the image,/>Is the width of the image;
The second patch fusion layer fused with the self-attention feature extraction module comprises:
Extracting the second characteristic in the original image channel Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the third feature extraction; Feature map/>, after feature extraction of the second time, in the contracted map channelInput to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the third feature extraction；
The third patch fusion layer fused with the self-attention feature extraction module includes:
extracting the third feature in the original image channel to obtain a feature image Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the fourth feature extraction; Feature map/>, after feature extraction of third time, in contracted map channelInput to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the fourth feature extraction。
Further, the two consecutive self-attention window layers include:
Inputting the feature images output by the linear embedding layer or the patch fusion layer into a first layer normalization layer for normalization processing, inputting the feature images after layer normalization into a window self-attention calculation layer for window allocation operation, adding curvature self-attention weight into self-attention calculation, sequentially inputting the feature images output by the window self-attention calculation layer and the feature images output by the linear embedding layer or the patch fusion layer into a full-connection layer, a second layer normalization layer and a multi-layer perceptron layer, and finally performing full-connection operation on the feature images output by the multi-layer perceptron layer and the feature images output by the window self-attention calculation layer to obtain the feature images output by a first attention window module, wherein the self-attention calculation formula is as follows:
；
wherein, To add the self-attention calculation function of curvature self-attention weight, variablesTo query vectors, variables/>Is a key vector,/>As a transpose of the key vector, variable/>Is a value vector, variable/>Represents the offset, variable/>Representing the dimensions of the key,/>For the/>, in the feature map of the current working environment after layer normalizationCurvature self-attention weight of each pixel point,/>Representing the total number of pixel points in the feature map of the current working environment after layer normalization,/> />, In feature map representing current working environment after layer normalizationPixel dot,/>To be improved afterThe function is activated.
Inputting the feature map output by the first attention window module into a third layer normalization layer for normalization processing, inputting the feature map after layer normalization into a moving window multi-head self-attention layer, adding edge self-attention weight into self-attention calculation, sequentially inputting the feature map output by the moving window multi-head self-attention layer and the feature map output by the first attention window module into a full-connection layer, a fourth layer normalization layer and a multi-layer perceptron layer, and performing full-connection operation on the feature map output by the multi-layer perceptron layer and the feature map output by the moving window multi-head self-attention layer through the processing of an activation function of the multi-layer perceptron layer to obtain the feature map output by the whole of two continuous attention window modules; the self-attention calculation formula is as follows:
；
wherein, To add the self-attention computation function of the edge self-attention weight,For the/>, in the feature map of the current working environment after layer normalizationThe edges of the individual pixels are weighted from the attention.
Further, in the step (5.4), the step of inputting the feature map obtained by extracting each feature in the original map channel and the contracted map channel into the decoder for semantic segmentation after upsampling and jump connection operation includes:
Upsampling the feature map after the fourth feature extraction in the contracted map channel to make the feature map have the following size The image is connected with the feature image after the fourth feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>The extended self-attention module comprises a patch extension layer and two continuous self-attention layers;
Performing up-sampling operation on the feature map after the third feature extraction in the contracted map channel to make the size of the feature map be the same as that of the contracted map channel The image is connected with the feature image after the third feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>；
Performing up-sampling operation on the feature map after the second feature extraction in the contracted map channel to make the feature map have the size ofAnd the image is connected with the feature image after the second feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>；
Upsampling the feature map after the first feature extraction in the contracted map channel to make the feature map have the following sizeThe image is connected with the feature image after the first feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>。
Further, the calculating the position of the center of the obstacle according to the RGB images of the working environment containing semantic information obtained at successive times, predicting the movement direction of the obstacle, and selecting the movement direction according to the predicted movement direction of the obstacle includes:
(5.51) calculating position coordinates of the center point of the obstacle in the operation environment image containing semantic information at each moment;
(5.52) predicting the position of the obstacle at the next moment according to the prediction function and the position coordinates of the center point of the obstacle, wherein the formula is as follows:
；
wherein, Represents the abscissa of the predicted obstacle center point at the next moment,/>Represents the/>Abscissa of obstacle center point in operation environment RGB image containing semantic information at moment,/>Representing the number of co-acquired obstacle coordinates;
(5.53) determining the movement direction of the robot at the next moment according to the predicted position of the center point of the obstacle at the next moment in the working environment image containing semantic information, wherein the calculation formula is as follows:
；
；
wherein, Abscissa representing center point of RGB image of a work environment containing semantic information,/>Represents the abscissa of the predicted obstacle center point at the next moment,/>For predicting the distance between the center point of the obstacle at the next moment and the center point of the RGB image of the operation environment containing semantic information,/>The robot rotation angle during path planning is set;
(5.54) the robot adjusts the corresponding movement direction and moves forward by a preset distance, and then returns to execute the step (1).
The beneficial effects are that: compared with the prior art, the invention has the remarkable advantages that the self-supervision attention semantic segmentation network is more focused on the segmentation of the edges of the object by introducing new self-attention weights obtained by entropy calculation according to the sea plug matrix; the self-supervision attention semantic segmentation network is led in to pay more attention to segmentation of the part with larger color change in the image according to the self-supervision attention weight obtained by edge detection and calculation, and the accuracy of edge segmentation is improved, so that the success rate of obstacle avoidance is improved. The improved activation function is used for calculating the self-attention weight, so that the self-supervision attention semantic segmentation network is more suitable for an application scene of path planning, the accuracy of edge segmentation is improved, and the success rate of obstacle avoidance is improved; smoothing the binarized image obtained by edge detection to avoid excessive attention to the edge part during self-attention calculation; the speed of identifying the obstacle is high, the accuracy is high, the background interference resistance is high, and the path planning accuracy is improved.
Drawings
FIG. 1 is a schematic workflow diagram of a path planning method of the present invention.
Fig. 2 is a gray scale image of the current work environment captured by the camera of the present invention.
FIG. 3 is a binarized image of the present invention for edge detection of the current operating environment.
FIG. 4 is a diagram of a self-supervising attention semantic segmentation network model in accordance with the present invention.
Fig. 5 is a semantic segmentation effect diagram of an RGB image of a current work environment obtained using the prior art UperNet.
Fig. 6 is a semantic division effect diagram of an RGB image of the current working environment obtained by the present invention.
Detailed Description
In the path planning method based on the local self-attention moving window algorithm, a REALSENSE D435i depth binocular camera is used for collecting an RGB image of a current working environment, processing the image through software of an upper computer, displaying a processing result by utilizing a liquid crystal display screen, and inputting the processing result into a robot processor for path planning. The method can be applied to the field of robot path planning.
As shown in fig. 1, a path planning method based on a local self-attention moving window algorithm in this embodiment includes the following steps:
(1) The RGB image of the current working environment is obtained through the camera, the RGB image of the current working environment is preprocessed, and the RGB image of the preprocessed working environment is obtained.
(1.1) Scaling the acquired color image to an image size acceptable to the backbone network.
(1.2) Inverting, affine transforming and noise adding the scaled color image to obtain a preprocessed RGB image of the current working environment.
(2) The RGB image of the current working environment is converted into a gray image, and the gray image of the current working environment is obtained, as shown in fig. 2.
(3) Performing edge detection on the converted gray level image to obtain a binary image containing edge characteristic information, and obtaining the edge self-attention weight of the binary image through an activation function; the method comprises the following specific steps:
(3.11) smoothing the preprocessed gray image of the current working environment, and using a Gaussian kernel function to the original image by using a Gaussian filter to obtain the smoothed gray image of the current working environment. Wherein the formula of the gaussian kernel function is as follows:
；
wherein, As a Gaussian kernel function,/>Representing bandwidth,/>And the coordinates of each pixel point in the preprocessed gray-scale image of the current working environment are represented.
(3.12) Calculating a first-order partial derivative finite difference by using a Sobel edge detection operator to obtain a gradient amplitude of each pixel in a gray level image of the current working environment, performing maximum value inhibition operation on the gradient amplitude, and finally performing edge detection by a double-threshold algorithm, wherein the specific method can refer to Yanglin. The image edge extraction [ J ]. Information of the improved operator of the local self-adaptive threshold method and a computer (theoretical version), 2023,35 (14): 78-80, which are not described herein any more, so as to obtain a binary image of the current working environment containing edge information, as shown in fig. 3.
(3.13) Taking the pixel value of each pixel point in the binarized image of the current working environment containing the edge information as a confidence coefficient, and smoothing the confidence coefficient, wherein a confidence coefficient calculation formula is as follows:
；
wherein, Represents the/>Confidence of individual pixel points,/>Gray value representing first pixel point in binarized image containing edge information,/>, andRepresenting the/>, in a binarized image containing edge informationPixel dot and/>Distance of individual pixel points,/>Divider/>, representing a binarized image containing edge informationFirst pixel out ofConfidence of individual pixel points,/>The total number of pixels in the binarized image containing the edge information is represented.
Calculating the edge attention weight of the confidence coefficient of each pixel point in the smoothed binarized image of the current working environment containing the edge information by using the improved activation function, wherein the calculation formula is as follows:
；
wherein, Represents the/>Confidence of individual pixel points,/>First/>, in a binarized image representing a current working environment containing edge informationEdge self-attention weight value of each pixel point,/>Representing the total number of pixel points in a binarized image of a current working environment containing edge information,/>, andTo be improved/>The function is activated.
Dividing the converted gray image to obtain a plurality of gray image blocks, calculating entropy of sea plug matrix eigenvalues corresponding to each divided gray image block, and calculating curvature self-attention weight of each gray image block through an activation function according to the entropy, wherein the specific steps comprise:
(3.21) extracting image blocks from the gray level image of the current operation environment by taking the step length as 8, wherein each image block comprises 64 pixel points, the size of the image block is 8 multiplied by 8, and a plurality of gray level image blocks of the current operation environment after segmentation processing are obtained;
And (3.22) calculating the sea plug matrix of the feature map corresponding to each gray image block, and filtering out the punctiform structures and noise points. The pixel points in each gray image block of the current operation environment after each division processing are convolved with the second derivative of the Gaussian function corresponding to the pixel points to obtain a sea plug matrix corresponding to the gray image block of the current operation environment after each division processing, so that the characteristic value solution of the sea plug matrix corresponding to each gray image block of the current operation environment after the division processing is obtained, and the calculation formula is as follows:
；
；
；
；
；
wherein, A gray value function corresponding to the gray image block of the current working environment after the segmentation processing is represented,Representing gray value corresponding to each pixel point in gray image block of current working environment after segmentation processing,/>Representing the number of pixel points in the gray image block of the current working environment after the segmentation processing,/>Representing sea plug matrix corresponding to each gray image block of the current working environment after segmentation processing,/>Gaussian function corresponding to gray image block representing current working environment after segmentation processing,/>Respectively representing the horizontal and vertical coordinates of each pixel point in the gray image block of the current working environment after preprocessing and segmentation processing,/>Represents the standard deviation in Gaussian distribution,/>And the characteristic values of the sea plug matrix corresponding to the gray image blocks of the current working environment after the segmentation processing are represented.
(3.23) Calculating entropy of characteristic values corresponding to each gray image block of the current working environment after the segmentation processing, wherein the basic formula is as follows:
；
wherein, Represents the/>Entropy of characteristic value corresponding to gray image block of current working environment after segmentation processingRepresenting the number of pixel points in each gray image block of the current working environment after the segmentation processing,/>Represents the/>, of the current working environment after the segmentation processAnd the characteristic values of the sea plug matrix corresponding to the gray image blocks.
(3.24) Entropy application of sea plug matrix corresponding to gray image blocks of the current working environment after the segmentation processingAnd activating the function to obtain the curvature self-attention weight corresponding to the gray image block of the current working environment after each segmentation process. The calculation formula is as follows:
；
wherein, Represents the/>Entropy of characteristic value corresponding to gray image block of current working environment after segmentation processingRepresents the/>Curvature self-attention weight corresponding to gray image blocks of current working environment,/>The total number of the gray image blocks after the segmentation processing is defined.
(4) The self-supervision attention semantic segmentation network is improved by edge self-attention weights and curvature self-attention weights, wherein the self-supervision attention semantic segmentation network comprises two continuous self-attention window layers, and the edge self-attention weights and the curvature self-attention weights are respectively added into the attention calculation of the two continuous self-attention window layers.
(5) The RGB image of the operation environment after preprocessing is processed through the improved self-supervision attention semantic segmentation network, the operation environment RGB image containing semantic information is obtained, the position of an obstacle substance center is calculated according to the operation environment RGB image containing semantic information obtained at continuous moments, the movement direction of the obstacle is predicted, and the movement direction of the obstacle is selected according to the predicted movement direction of the obstacle. The specific steps are shown in fig. 4, including:
(5.1) reducing the RGB image of the current working environment after preprocessing by two times, and inputting the images before and after reduction from the original image channel and the reduced image channel respectively to an image block dividing layer, and dividing the image block into a plurality of non-overlapping image blocks. The original image obtained has an image block size of The original image obtained has an image block size of/>Wherein, the method comprises the steps of, wherein,For compressed image depth,/>For the height of the RGB image of the current working environment after preprocessing,/>Is the width of the RGB image of the current working environment after preprocessing.
(5.2) Inputting the image block output by the image segmentation layer into a first linear self-attention feature extraction module, wherein the module comprises a linear embedding layer and two continuous self-attention window layers, and a feature map after the first feature extraction is obtained.
And (5.3) sequentially inputting the feature map after the first feature extraction into three identical fusion self-attention feature extraction modules, wherein the modules comprise a patch fusion layer and two continuous self-attention window layers, so as to obtain the feature map after the fourth feature extraction.
The first patch fusion layer fused with the self-attention feature extraction module comprises:
extracting the first characteristic in the original image channel Input to the first patch fusion layer, downsampled, and output dimension set to/>. The size of the obtained characteristic diagram is/>. Feature map/>, after first feature extraction, in a contracted map channelInput to the first patch fusion layer, downsampled, and output dimension set to/>. The size of the obtained characteristic diagram is/>。
The second patch fusion layer fused with the self-attention feature extraction module comprises:
Extracting the second characteristic in the original image channel Input to the second patch fusion layer, downsampled, and output dimension set to/>. The size of the obtained characteristic diagram is/>. Inputting the feature map extracted from the second feature in the contracted map channel to a second patch fusion layer, downsampling, and setting the output dimension as. The size of the obtained characteristic diagram is/>。
The third patch fusion layer fused with the self-attention feature extraction module includes:
extracting the third feature in the original image channel to obtain a feature image Input to the third patch fusion layer, downsampled, and output dimension set to/>. The size of the obtained characteristic diagram is/>. Feature map/>, after feature extraction of third time, in contracted map channelInput to the third patch fusion layer, downsampled, and output dimension set to/>. The size of the obtained characteristic diagram is/>。
The two successive self-attention window layers include:
A first self-attention window layer: inputting the feature images output by the linear embedding layer or the patch fusion layer into a first layer normalization layer for normalization processing, inputting the feature images of the current working environment after layer normalization into a window self-attention calculation layer for window allocation operation, adding the obtained curvature self-attention weight into attention calculation, sequentially inputting the feature images output by the window self-attention calculation layer and the feature images output by the linear embedding layer or the patch fusion layer into a full-connection layer, a second layer normalization layer and a multi-layer perceptron layer, and finally performing full-connection operation on the feature images output by the multi-layer perceptron layer and the feature images output by the window self-attention calculation layer to obtain the feature images output by a first attention module, wherein the self-attention calculation formula is as follows:
；
wherein, To add the self-attention calculation function of curvature self-attention weight, variablesTo query vectors, variables/>Is a key vector,/>As a transpose of the key vector, variable/>Is a value vector, variable/>Represents the offset, variable/>Representing the dimensions of the key,/>For the/>, in the feature map of the current working environment after layer normalizationCurvature self-attention weight of each pixel point,/>Representing the total number of pixel points in the feature map of the current working environment after layer normalization,/>/>, In feature map representing current working environment after layer normalizationPixel dot,/>To be improved/>The function is activated.
A second self-attention window layer: the method comprises the steps of inputting a feature map output by a first attention window module into a third layer normalization layer for normalization processing, inputting the feature map of the current working environment after layer normalization into a moving window multi-head self-attention layer, adding the obtained edge self-attention weight into self-attention calculation, sequentially inputting the feature map output by the moving window multi-head self-attention layer and the feature map output by the first attention window module into a full-connection layer, and processing activation functions of the multi-layer perceptron layer in a fourth layer normalization layer and the multi-layer perceptron layer, and performing full-connection operation on the feature map output by the multi-layer perceptron layer and the feature map output by the moving window multi-head self-attention layer to obtain the feature map output by the whole of two continuous attention window modules. The basic formula is as follows:
；
wherein, To add the self-attention computation function of the edge self-attention weight,For the/>, in the feature map of the current working environment after layer normalizationThe edges of the individual pixels are weighted from the attention.
And (5.4) sequentially inputting the feature images after the fourth feature extraction into a decoder for semantic segmentation to obtain an RGB image of the current working environment containing semantic information, as shown in fig. 6. The method specifically comprises the following steps:
Upsampling the feature map obtained by the fourth feature extraction in the contracted map channel to obtain a feature map with the size of And jumping connection is carried out on the feature map obtained by the fourth feature extraction in the original map channel, and the feature map is input to an extended self-attention module which comprises a patch extended layer and two continuous self-attention layers. Output picture size is/>。
Performing up-sampling operation on the feature map obtained by the third feature extraction in the contracted map channel to make the feature map have the size ofAnd jumping connection is carried out on the feature map obtained by extracting the third feature in the original map channel, and the feature map is input to an extended self-attention module which comprises a patch extended layer and two continuous self-attention layers. Output picture size is/>。
Performing up-sampling operation on the feature map obtained by the second feature extraction in the contracted map channel to make the feature map have the size ofAnd jumping connection is carried out on the feature map obtained by extracting the second feature in the original map channel, and the feature map is input to an extended self-attention module which comprises a patch extended layer and two continuous self-attention layers. The output image has a size of。
Upsampling the feature map obtained by the first feature extraction in the contracted map channel to obtain a feature map of the same size as the feature mapAnd jumping connection is carried out on the feature map obtained by the first feature extraction in the original map channel, and the feature map is input to an extended self-attention module which comprises a patch extended layer and two continuous self-attention layers. The output image has a size of。
And (5.5) calculating the position of the obstacle substance center according to the RGB images of the working environment containing semantic information, which are obtained at continuous moments, predicting the movement direction of the obstacle, and selecting the movement direction of the obstacle according to the predicted movement direction of the obstacle. The method comprises the following specific steps:
(5.51) calculating the position coordinates of the center point of the obstacle in the image of the current working environment containing semantic information obtained at each time. The basic calculation formula is as follows:
；
wherein, Represents the/>Abscissa of center point of obstacle in RGB image of current working environment containing semantic information at each moment,/>Represents the/>Left edge point abscissa,/>, of obstacle in RGB image of current working environment containing semantic information at each momentRepresents the/>The right edge point abscissa of the obstacle in the RGB image of the current working environment containing semantic information at each moment.
(5.52) Predicting the position of the obstacle at the next moment according to the prediction function and the obtained center point coordinates of the obstacle, wherein the basic formula is as follows:
；
wherein, Represents the abscissa of the center point of the obstacle at the next time predicted,/>Represents the/>Abscissa of center point of obstacle in RGB image of current working environment containing semantic information at each moment,/>Representing the number of co-acquired obstacle coordinates.
(5.53) Calculating the direction of robot movement at the next moment according to the predicted position of the center point of the obstacle at the next moment in the RGB image of the current working environment containing semantic information, wherein the basic calculation formula is as follows:
；
wherein, Abscissa of center point of RGB image representing current working environment containing semantic information,/>Represents the abscissa of the center point of the predicted obstacle at the next moment,/>Distance between center point of obstacle and center point of RGB image of current working environment containing semantic information is predicted.
And (5.54) calculating the motion direction which the robot itself should adjust according to the motion direction, and moving forward by 0.5 meter. The above operation is repeated to realize path planning, and the basic calculation formula is as follows:
；
wherein, For the angle of rotation in path planning, in this scenario/>Taking a constant value of 30 degrees.
The invention also provides a semantic segmentation effect graph comparison of RGB images of the current working environment, which is the same as the working environment of the embodiment but obtained by adopting the prior art UperNet. FIG. 5 is a semantic segmentation effect diagram of an RGB image of a current work environment obtained using prior art UperNet; FIG. 6 is a semantic segmentation effect diagram of an RGB image of a current working environment obtained by adopting the local self-attention moving window algorithm in the invention; the comparison shows that the effect of the object edge segmentation in fig. 6 is obviously better than that in fig. 5, so that in the embodiment, the self-supervision attention semantic segmentation network provided by the invention effectively improves the accuracy of semantic segmentation, thereby improving the obstacle avoidance accuracy of the robot.
Claims (8)
1. A path planning method based on a local self-attention moving window algorithm, comprising the steps of:
(1) Acquiring an RGB image of a current operation environment, and preprocessing to acquire an RGB image of the preprocessed operation environment;
(2) Converting the RGB image of the preprocessed working environment into a gray scale image;
(3) Performing edge detection on the converted gray level image to obtain a binary image containing edge characteristic information, and obtaining the edge self-attention weight of the binary image through an activation function, wherein the calculation formula is as follows:
；
wherein, Represents the/>Confidence of individual pixel points,/>Representing the/>, in a binarized image containing edge informationEdge self-attention weight value of each pixel point,/>For/>Activating a function,/>Representing the total number of pixel points in the binarized image containing the edge information;
dividing the converted gray image to obtain a plurality of gray image blocks, calculating entropy of sea plug matrix eigenvalues corresponding to each divided gray image block, and calculating curvature self-attention weight of each gray image block through an activation function according to the entropy; the method comprises the following specific steps:
(3.21) dividing the converted gray image to obtain a plurality of gray image blocks;
(3.22) calculating a sea plug matrix of the feature map corresponding to each gray image block, and filtering out punctiform structures and noise points; the pixel points in each gray image block of the current operation environment after each division processing are convolved with the second derivative of the Gaussian function corresponding to the pixel points to obtain a sea plug matrix corresponding to the gray image block of the current operation environment after each division processing, so that the characteristic value solution of the sea plug matrix corresponding to each gray image block of the current operation environment after the division processing is obtained, and the calculation formula is as follows:
；
；
；
；
；
wherein, A gray value function corresponding to the gray image block of the current working environment after the segmentation processing is represented,Representing gray value corresponding to each pixel point in gray image block of current working environment after segmentation processing,/>Representing the number of pixel points in the gray image block of the current working environment after the segmentation processing,/>Representing sea plug matrix corresponding to each gray image block of the current working environment after segmentation processing,/>Gaussian function corresponding to gray image block representing current working environment after segmentation processing,/>Respectively representing the horizontal and vertical coordinates of each pixel point in the gray image block of the current working environment after preprocessing and segmentation processing,/>Represents the standard deviation in Gaussian distribution,/>Characteristic values of sea plug matrixes corresponding to all gray image blocks of the current working environment after segmentation processing are represented;
(3.23) calculating entropy of characteristic values corresponding to each gray image block of the current working environment after the segmentation processing, wherein the basic formula is as follows:
；
wherein, Represents the/>Entropy of characteristic value corresponding to gray image block of current working environment after segmentation processingRepresenting the number of pixel points in each gray image block of the current working environment after the segmentation processing,/>Represents the/>, of the current working environment after the segmentation processCharacteristic values of sea plug matrixes corresponding to the gray image blocks;
(3.24) entropy application of sea plug matrix corresponding to gray image blocks of the current working environment after the segmentation processing Activating a function to obtain curvature self-attention weights corresponding to gray image blocks of the current working environment after each segmentation process; the calculation formula is as follows:
；
wherein, Represents the/>Entropy of a characteristic value corresponding to the gray image block of the current working environment after the segmentation processing,Represents the/>Curvature self-attention weight corresponding to gray image blocks of current working environment,/>The total number of the gray image blocks after the segmentation processing;
(4) The self-supervision attention semantic segmentation network is improved through edge self-attention weights and curvature self-attention weights, the self-supervision attention semantic segmentation network comprises two continuous self-attention window layers, and the edge self-attention weights and the curvature self-attention weights are respectively added into the attention calculation of the two continuous self-attention window layers;
(5) The RGB image of the operation environment after preprocessing is processed through the improved self-supervision attention semantic segmentation network, the operation environment RGB image containing semantic information is obtained, the position of an obstacle substance center is calculated according to the operation environment RGB image containing semantic information obtained at continuous moments, the movement direction of the obstacle is predicted, and the movement direction of the obstacle is selected according to the predicted movement direction of the obstacle.
2. The path planning method according to claim 1, wherein the preprocessing of the RGB image of the current working environment in step (1) includes scaling the RGB image of the current working environment, and then performing flipping, affine transformation and noise addition on the scaled RGB image to obtain the RGB image of the preprocessed working environment.
3. The path planning method according to claim 1, wherein the step (3) of performing edge detection on the transformed gray scale map to obtain a binary image containing edge feature information, and the step of obtaining the edge self-attention weight of the binary image by activating the function comprises:
(3.11) carrying out smoothing treatment on the gray level image, then calculating the gradient amplitude of each pixel in the image, carrying out maximum value inhibition operation on the gradient amplitude, and finally carrying out edge detection through a double-threshold algorithm to obtain a binarized image containing edge information;
(3.12) smoothing the confidence coefficient of each pixel point in the binarized image containing the edge information, wherein the confidence coefficient is calculated according to the formula:
；
wherein, Represents the/>Confidence of individual pixel points,/>Representing the/>, in a binarized image containing edge informationGray value of each pixel/(Representing the/>, in a binarized image containing edge informationPixel dot and/>Distance of individual pixel points,/>Divider/>, representing a binarized image containing edge informationFirst pixel out ofConfidence of individual pixel points,/>Representing the total number of pixel points in the binarized image containing the edge information;
(3.13) converting the confidence into an edge self-attention weight for each pixel point by an activation function.
4. The path planning method according to claim 1, wherein the processing of the RGB image of the pre-processed work environment through the modified self-supervised attention semantic segmentation network in step (5) comprises:
(5.1) reducing the RGB image of the preprocessed working environment, and inputting the images before and after reduction into an image block segmentation layer from an original image channel and a reduced image channel respectively for segmentation to obtain a plurality of non-overlapping image blocks respectively;
(5.2) inputting the image blocks obtained by segmentation into a first linear self-attention feature extraction module, wherein the first linear self-attention feature extraction module comprises a linear embedding layer and two continuous self-attention window layers, and obtaining a feature map after the first feature extraction;
(5.3) sequentially inputting the feature map after the first feature extraction into three fusion self-attention feature extraction modules, wherein each fusion self-attention feature extraction module comprises a patch fusion layer and two continuous self-attention window layers, and obtaining a feature map after the fourth feature extraction;
and (5.4) inputting the feature images obtained in the original image channel and the contracted image channel after the feature extraction of each time into a decoder for semantic segmentation after up-sampling and jump connection operation, and obtaining the operation environment RGB image containing semantic information.
5. The path planning method of claim 4, wherein the first patch fusion layer that fuses the self-attention feature extraction module comprises:
extracting the first characteristic in the original image channel Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the second feature extraction; Feature map/>, after first feature extraction, in a contracted map channelInput to the patch fusion layer, downsampled, and output dimension set toObtaining a feature map/>, after the second feature extraction，/>For image depth,/>Is the height of the image,/>Is the width of the image;
The second patch fusion layer fused with the self-attention feature extraction module comprises:
Extracting the second characteristic in the original image channel Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the third feature extraction; Feature map/>, after feature extraction of the second time, in the contracted map channelInput to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the third feature extraction；
The third patch fusion layer fused with the self-attention feature extraction module includes:
extracting the third feature in the original image channel to obtain a feature image Input to the patch fusion layer, downsampled, and output dimension set to/>Obtaining a feature map/>, after the fourth feature extraction; Inputting the feature map after the third feature extraction in the contracted map channel to a patch fusion layer, performing downsampling, and setting the output dimension as/>Obtaining a feature map/>, after the fourth feature extraction。
6. The path planning method of claim 5, wherein the two consecutive self-attention window layers comprise:
Inputting the feature images output by the linear embedding layer or the patch fusion layer into a first layer normalization layer for normalization processing, inputting the feature images after layer normalization into a window self-attention calculation layer for window allocation operation, adding curvature self-attention weight into self-attention calculation, sequentially inputting the feature images output by the window self-attention calculation layer and the feature images output by the linear embedding layer or the patch fusion layer into a full-connection layer, a second layer normalization layer and a multi-layer perceptron layer, and finally performing full-connection operation on the feature images output by the multi-layer perceptron layer and the feature images output by the window self-attention calculation layer to obtain the feature images output by a first attention window module, wherein the self-attention calculation formula is as follows:
；
wherein, To add the self-attention computation function of curvature self-attention weight, variable/>To query vectors, variables/>Is a key vector,/>As a transpose of the key vector, variable/>Is a value vector, variable/>Represents the offset, variable/>Representing the dimensions of the key,/>For the/>, in the feature map of the current working environment after layer normalizationCurvature self-attention weight of each pixel point,/>The total number of pixel points in the characteristic diagram of the current working environment after the layer normalization is represented,/>, In feature map representing current working environment after layer normalizationPixel dot,/>For/>Activating a function;
Inputting the feature map output by the first attention window module into a third layer normalization layer for normalization processing, inputting the feature map after layer normalization into a moving window multi-head self-attention layer, adding edge self-attention weight into self-attention calculation, sequentially inputting the feature map output by the moving window multi-head self-attention layer and the feature map output by the first attention window module into a full-connection layer, a fourth layer normalization layer and a multi-layer perceptron layer, and performing full-connection operation on the feature map output by the multi-layer perceptron layer and the feature map output by the moving window multi-head self-attention layer through the processing of an activation function of the multi-layer perceptron layer to obtain the feature map output by the whole of two continuous attention window modules; the self-attention calculation formula is as follows:
；
wherein, To add the self-attention computation function of the edge self-attention weight,For the/>, in the feature map of the current working environment after layer normalizationThe edges of the individual pixels are weighted from the attention.
7. The path planning method according to claim 6, wherein the step (5.4) of inputting the feature map obtained in the original map channel and the contracted map channel after each feature extraction into the decoder for semantic segmentation after upsampling and jump connection operations comprises:
Upsampling the feature map after the fourth feature extraction in the contracted map channel to make the feature map have the following size The image is connected with the feature image after the fourth feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>The extended self-attention module comprises a patch extension layer and two continuous self-attention layers;
Performing up-sampling operation on the feature map after the third feature extraction in the contracted map channel to make the size of the feature map be the same as that of the contracted map channel The image is connected with the feature image after the third feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>；
Performing up-sampling operation on the feature map after the second feature extraction in the contracted map channel to make the feature map have the size ofThe image is connected with the feature image after the second feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>；
Upsampling the feature map after the first feature extraction in the contracted map channel to make the feature map have the following sizeThe image is connected with the feature image after the first feature extraction in the original image channel in a jumping way, and is input to the extended self-attention module, and the output image size is/>。
8. The path planning method according to claim 7, wherein calculating the heart position of the obstacle from the RGB images of the work environment containing semantic information obtained at successive times and predicting the movement direction of the obstacle, and selecting the movement direction of the obstacle based on the predicted movement direction comprises:
(5.51) calculating position coordinates of the center point of the obstacle in the RGB image of the working environment containing semantic information at each moment;
(5.52) predicting the position of the obstacle at the next moment according to the prediction function and the position coordinates of the center point of the obstacle, wherein the formula is as follows:
；
wherein, Represents the abscissa of the predicted obstacle center point at the next moment,/>Represents the/>Abscissa of obstacle center point in operation environment RGB image containing semantic information at moment,/>Representing the number of co-acquired obstacle coordinates;
(5.53) determining the movement direction of the robot at the next moment according to the predicted position of the center point of the obstacle at the next moment in the RGB image of the working environment containing semantic information, wherein the calculation formula is as follows:
，
，
wherein, Abscissa representing center point of RGB image of a work environment containing semantic information,/>Represents the abscissa of the predicted obstacle center point at the next moment,/>For predicting the distance between the center point of the obstacle at the next moment and the center point of the RGB image of the operation environment containing semantic information,/>The robot rotation angle during path planning is set;
(5.54) the robot adjusts the corresponding movement direction and moves forward by a preset distance, and then returns to execute the step (1).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410304943.7A CN117889867B (en) | 2024-03-18 | 2024-03-18 | Path planning method based on local self-attention moving window algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410304943.7A CN117889867B (en) | 2024-03-18 | 2024-03-18 | Path planning method based on local self-attention moving window algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117889867A CN117889867A (en) | 2024-04-16 |
| CN117889867B true CN117889867B (en) | 2024-05-24 |
Family
ID=90639927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410304943.7A Active CN117889867B (en) | 2024-03-18 | 2024-03-18 | Path planning method based on local self-attention moving window algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117889867B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807257A (en) * | 2010-05-12 | 2010-08-18 | 上海交通大学 | Method for identifying information of image tag |
| CN104778721A (en) * | 2015-05-08 | 2015-07-15 | 哈尔滨工业大学 | Distance measuring method of significant target in binocular image |
| CN109766924A (en) * | 2018-12-20 | 2019-05-17 | 东南大学 | Image detecting method based on image information entropy Yu adaptive threshold DAISY characteristic point |
| WO2021011581A1 (en) * | 2019-07-15 | 2021-01-21 | Memorial Sloan Kettering Cancer Center | Image-based predictive model for lung cancer |
| CN115482382A (en) * | 2022-09-17 | 2022-12-16 | 北京工业大学 | Image semantic segmentation method based on Transformer architecture |
| CN116189180A (en) * | 2023-04-28 | 2023-05-30 | 青岛理工大学 | Urban streetscape advertisement image segmentation method |
| CN116402742A (en) * | 2023-01-17 | 2023-07-07 | 有件(嘉兴)网络科技有限公司 | Visual detection method and system for surface defects of automobile sheet metal part |
| CN117036686A (en) * | 2023-06-29 | 2023-11-10 | 南京邮电大学 | Semantic segmentation method based on self-attention and convolution feature fusion |
| CN117522896A (en) * | 2023-11-22 | 2024-02-06 | 深圳职业技术大学 | Self-attention-based image segmentation method and computer equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8285019B2 (en) * | 2006-02-10 | 2012-10-09 | Synarc Inc. | Breast tissue density measure |
| US9799098B2 (en) * | 2007-04-24 | 2017-10-24 | Massachusetts Institute Of Technology | Method and apparatus for image processing |
| US20150032449A1 (en) * | 2013-07-26 | 2015-01-29 | Nuance Communications, Inc. | Method and Apparatus for Using Convolutional Neural Networks in Speech Recognition |
-
2024
- 2024-03-18 CN CN202410304943.7A patent/CN117889867B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807257A (en) * | 2010-05-12 | 2010-08-18 | 上海交通大学 | Method for identifying information of image tag |
| CN104778721A (en) * | 2015-05-08 | 2015-07-15 | 哈尔滨工业大学 | Distance measuring method of significant target in binocular image |
| CN109766924A (en) * | 2018-12-20 | 2019-05-17 | 东南大学 | Image detecting method based on image information entropy Yu adaptive threshold DAISY characteristic point |
| WO2021011581A1 (en) * | 2019-07-15 | 2021-01-21 | Memorial Sloan Kettering Cancer Center | Image-based predictive model for lung cancer |
| CN115482382A (en) * | 2022-09-17 | 2022-12-16 | 北京工业大学 | Image semantic segmentation method based on Transformer architecture |
| CN116402742A (en) * | 2023-01-17 | 2023-07-07 | 有件(嘉兴)网络科技有限公司 | Visual detection method and system for surface defects of automobile sheet metal part |
| CN116189180A (en) * | 2023-04-28 | 2023-05-30 | 青岛理工大学 | Urban streetscape advertisement image segmentation method |
| CN117036686A (en) * | 2023-06-29 | 2023-11-10 | 南京邮电大学 | Semantic segmentation method based on self-attention and convolution feature fusion |
| CN117522896A (en) * | 2023-11-22 | 2024-02-06 | 深圳职业技术大学 | Self-attention-based image segmentation method and computer equipment |
Non-Patent Citations (1)
| Title |
|---|
| 考虑结构模态识别不确定性的传感器优化布置理论研究;裴雪扬;《中国博士学位论文全文数据库 信息科技辑》;20210115(第01期);I140-98 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117889867A (en) | 2024-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112132156B (en) | Image saliency target detection method and system based on multi-depth feature fusion | |
| CN106845478B (en) | A kind of secondary licence plate recognition method and device of character confidence level | |
| CN109035319B (en) | Monocular image depth estimation method, monocular image depth estimation device, monocular image depth estimation apparatus, monocular image depth estimation program, and storage medium | |
| CN107274445B (en) | Image depth estimation method and system | |
| CN114565655B (en) | Depth estimation method and device based on pyramid segmentation attention | |
| CN114724155B (en) | Scene text detection method, system and equipment based on deep convolutional neural network | |
| CN112947419B (en) | Obstacle avoidance method, device and equipment | |
| CN112883850A (en) | Multi-view aerospace remote sensing image matching method based on convolutional neural network | |
| CN112381061A (en) | Facial expression recognition method and system | |
| CN114445715A (en) | Crop disease identification method based on convolutional neural network | |
| CN110827320A (en) | Target tracking method and device based on time sequence prediction | |
| CN112633274A (en) | Sonar image target detection method and device and electronic equipment | |
| US11367206B2 (en) | Edge-guided ranking loss for monocular depth prediction | |
| CN114677479A (en) | Natural landscape multi-view three-dimensional reconstruction method based on deep learning | |
| CN114332644B (en) | Large-view-field traffic density acquisition method based on video satellite data | |
| CN113989604A (en) | Tire DOT information identification method based on end-to-end deep learning | |
| CN118251698A (en) | Novel view synthesis of robust NERF model for sparse data | |
| CN112184731A (en) | Multi-view stereo depth estimation method based on antagonism training | |
| Zuo et al. | A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields | |
| Gao | Performance evaluation of automatic object detection with post-processing schemes under enhanced measures in wide-area aerial imagery | |
| CN108647605B (en) | Human eye gaze point extraction method combining global color and local structural features | |
| Chen et al. | An image restoration and detection method for picking robot based on convolutional auto-encoder | |
| CN112115786B (en) | Monocular vision odometer method based on attention U-net | |
| CN113421210A (en) | Surface point cloud reconstruction method based on binocular stereo vision | |
| CN117392508A (en) | Target detection method and device based on coordinate attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |