CN112541930A - Image super-pixel target pedestrian segmentation method based on cascade connection - Google Patents

Image super-pixel target pedestrian segmentation method based on cascade connection Download PDF

Info

Publication number
CN112541930A
CN112541930A CN201910900391.5A CN201910900391A CN112541930A CN 112541930 A CN112541930 A CN 112541930A CN 201910900391 A CN201910900391 A CN 201910900391A CN 112541930 A CN112541930 A CN 112541930A
Authority
CN
China
Prior art keywords
superpixel
segmentation
image
target
result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910900391.5A
Other languages
Chinese (zh)
Inventor
杨大伟
马雪
毛琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian Minzu University
Original Assignee
Dalian Minzu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian Minzu University filed Critical Dalian Minzu University
Priority to CN201910900391.5A priority Critical patent/CN112541930A/en
Publication of CN112541930A publication Critical patent/CN112541930A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

本发明属于图像分割技术领域，具体的说是一种基于级联式的图像超像素目标行人分割方法，其步骤包括：S1、以Mask R‑CNN作为粗粒度一阶段分割，结合能量滤波准则提取目标区域及分割结果；S2、将单目标区域送至超像素分割通道，输出带标签的超像素结果；S3、计算超像素分割图像中相邻超像素块的特征相似差异值，将超像素块聚集为一个超系数像素块，聚集出更精准的目标对象轮廓；S4、利用能量高低频融合规则将一阶段粗粒度分割提取的目标结果和二阶段细粒度分割提取的目标对象轮廓进行融合操作，重构出最终的融合图像。

Figure 201910900391

The invention belongs to the technical field of image segmentation, in particular to a cascading-based image superpixel target pedestrian segmentation method. The target area and the segmentation result; S2, send the single target area to the superpixel segmentation channel, and output the superpixel result with the label; S3, calculate the feature similarity difference value of the adjacent superpixel blocks in the superpixel segmentation image, and divide the superpixel block Aggregate into a super-coefficient pixel block to gather a more accurate target object contour; S4, use the energy high-frequency fusion rule to fuse the target result extracted by the first-stage coarse-grained segmentation and the target object contour extracted by the second-stage fine-grained segmentation. The final fused image is reconstructed.

Figure 201910900391

Description

Image super-pixel target pedestrian segmentation method based on cascade connection
Technical Field
The invention belongs to the technical field of image segmentation, and particularly relates to a pedestrian segmentation method based on a cascading super-pixel target.
Background
The target pedestrian segmentation technology is an important research subject in the field of machine vision, and with the development of autonomous automobiles and driving assistance systems, how to protect the safety of pedestrians and vehicles becomes a research hotspot at present. The image segmentation technology is a preprocessing part in an image recognition and computer vision system, and can provide target characteristics for recognition and tracking by effectively segmenting an image, so that the interference of redundant information on post-image processing is avoided.
Currently, object segmentation based on color features can be divided into two categories, color image and grayscale image, and image segmentation techniques based on generation can be divided into two categories, region and feature. Aiming at different fields, the segmentation precision can be effectively improved by adopting a suitable image segmentation method. The current mainstream segmentation methods can be mainly divided into a region method, a threshold value method, an edge method and an interactive segmentation method. The patent application number is CN201910020730, which is entitled "an image segmentation method and apparatus", and discloses that an image to be segmented is obtained from a first image to be segmented and a second image to be segmented and the segmented images are input into a pre-trained image segmentation model, and correlation parameters of the segmented images are calculated by extracting feature vectors of the segmented images, so as to predict the segmented images. The patent application number is CN201811472018 entitled "chest segmentation and processing method, system and electronic device", which discloses that an image data set is trained by using a deep learning algorithm to obtain an image segmentation model based on deep learning, then an image to be segmented is processed by the image segmentation model to obtain a segmented lung region and a thoracic region, and finally a lung-chest ratio is calculated according to the lung region and the thoracic region. The method focuses on calculating the lung-chest ratio, but the precision of the method is still not accurate enough in terms of the segmentation result, and if the precision can be improved, the calculated lung-chest ratio result can be obviously more accurate. The patent application number is CN201810860392, which is named as a target detection network design method fusing image segmentation features and discloses a method for realizing small target image segmentation by combining a general target detection frame Mask-RCNN and image segmentation features. When the method is applied to small targets, the segmentation precision is relatively high, but the segmentation precision of large targets is still not accurate enough. The patent application number is CN201810860454, which is named as a pedestrian detection method based on Mask-RCNN, and aims at the condition that an in-vehicle target is mistakenly detected as a pedestrian, the method combines the characteristics that the Mask-RCNN can simultaneously carry out target detection and target segmentation, and provides an optimization algorithm combined with a target segmentation result.
Disclosure of Invention
In order to improve the accuracy of image segmentation of the target pedestrian, the invention provides a cascading-type image superpixel target pedestrian segmentation method, which provides more accurate preprocessing information for subsequent work of a computer vision system by establishing a target pedestrian segmentation model.
The invention realizes the above aim by the following technical scheme: a cascading-based image super-pixel target pedestrian segmentation method comprises the following steps:
step 1, sending a source image to an example segmentation channel, outputting an example segmentation image, splitting the example segmentation image, and extracting a single-target area and a segmentation result:
(M，R，S0)＝MASKRCNN(I0) (1)
wherein MASKRCNN is an example partition function, I0For inputting a source image, R is a single target area split and extracted from an example segmentation image, S0Splitting and extracting a segmentation result in the example segmentation image, wherein M is the example segmentation image;
wherein:
the example segmentation image M is an image which is not processed after the source image is subjected to example segmentation;
the single target area R is a single target area image obtained by splitting and extracting the example segmentation image M, and the range of the single target area R is larger than that of the target detection frame; formula (2) is a calculation formula of the number of the single target regions R:
B＝A±X(A∈(0，N+)，B∈(0，N+)，X∈(0，N+)) (2)
the source image contains A target objects, and B single target areas R after example segmentation, wherein X represents the number of error detection people of example segmentation targets;
segmentation result S0For example, a contour image obtained by splitting and extracting an image M is segmented, and a segmentation result S is represented by formula (3)0The calculation formula of the number is as follows:
N＝J±X(N∈(0，N+)，J∈(0，N+)，X∈(0，N+)) (3)
the example segmentation image M comprises J target objects, and N segmentation results S after the J target objects are split and extracted0Wherein X represents the number of error detection persons of the example division target;
step 2, sending the single target region R to a super-pixel segmentation channel, and outputting a super-pixel segmentation image with a label;
QGK＝SLIC(R) (4)
wherein SLIC is a superpixel segmentation function, R is an extracted single-target region in an example segmentation image, QGKSegmenting the image for K tagged superpixels;
step 3, dividing the super pixel into an image QGKMerging the super-pixel blocks with similar characteristics in the middle adjacent super-pixel blocks, replacing K super-pixel blocks in the super-pixel segmentation image with N super-pixel coloring information blocks, and finally reconstructing a more accurate target object outline;
PN＝Cslic(QGK) (5)
wherein, CslicFor combining functions, Q, of SLICGKSegmenting an image for K tagged superpixels, PNIs a reconstructed target object contour;
step 4, dividing the result S0And a target object profile PNFusing and reconstructing a cascaded segmentation fused image Ei。
Ei＝NSST(PN，S0) (6)
Wherein NSST is a non-down-shear wave transform multi-scale analysis function, PNIs the contour of the target object, S0For example segmentation results split and extracted in segmented images, EiIs a reconstructed final fused image;
the image fusion uses an energy filtering high-low frequency fusion rule: for the registered segmentation result S0And pre-fusing the target object contour P by adopting an energy filtering high-low frequency fusion rule, fusing the low-frequency coefficient by adopting a fusion rule based on an image guide filter in the low-frequency information fusion, and obtaining the low-frequency fusion coefficient. High frequency information fusion for superpixel QGKAnd then, the coefficients with the same label are gathered into a super-coefficient block, and the spatial frequency of each super-coefficient block is solved to obtain a high-frequency fusion coefficient. Finally, NSST inverse transformation is carried out on the high-frequency fusion coefficient and the low-frequency fusion coefficient, and a final fusion image E is reconstructedi。
Further, the super-pixel block feature merging step is as follows:
1) setting and sequencing superpixel blocks, and calculating the characteristic difference of color and space distance of adjacent superpixel blocks in the graph by the following calculation formula:
Figure BDA0002211642230000031
in the formula (9), the LAB vector adopts a CIELAB color space model, DLAB(Ri) For the inter-superpixel block color space distance, R' denotes the non-target region, liAnd ljIs a component of the pixel brightness, ai、aj、bi、bjBeing a component of a color, DXY(Ri) Is a position space distance, xi、xj、yi、yjThe vector obtains the spatial coordinate value of the pixel, D (R)i) Is the superpixel distance, δ is the distance weight coefficient, and δ belongs to (0, 1);
2) comparing the calculation result with a preset threshold, merging the target super-pixel block and the adjacent super-pixel block if the characteristic result is smaller than the threshold, ignoring the target pixel block if the characteristic result is larger than the threshold, and continuing to perform characteristic inspection on the next super-pixel block;
determining the correlation degree of the super-pixel area according to the super-pixel distance, wherein the calculation formula is as follows:
C(Ri)＝1-exp(-D(Ri)) (10)
in the formula (10), C (R)i) Representing the degree of super-pixel area correlation, D (R)i) The super-pixel distance is inversely related to the area correlation. Determining whether the superpixel blocks accord with the characteristic information of the same target or not according to the correlation;
according to the calculation of the regional relevance of all superpixels, a regional relevance threshold value is calculated by utilizing a maximum inter-class difference method, all superpixel blocks meeting the relevance threshold value are extracted as target superpixels, and the calculation formula is as follows:
Figure BDA0002211642230000032
in the formula (11), R*Representing the set of target superpixels finally acquired, RiIs the target superpixel at i, C (R)i) Representing the degree of correlation of the super-pixel area,
Figure BDA0002211642230000033
the method comprises the steps of obtaining a region correlation threshold value, wherein epsilon is a correlation threshold value coefficient, epsilon is 0.5, when epsilon is 0.5, characteristic information can be better divided into different pixel sets, each obtained subset forms a region corresponding to a real scene, the interior of each region has consistent attributes, and adjacent regions do not have the consistent attributes;
3) iterating the steps until all the superpixel blocks in the image complete one-time feature comparison, and generating a first merging result image at the moment;
4) before the second combination, refreshing the characteristic information and the rearrangement sequence of the superpixel blocks, and then combining the first combination result as an object of the combination operation until the superpixel blocks in the first combination result complete characteristic comparison to generate a second combination result graph.
Through the technical scheme, the invention has the beneficial effects that: the existing image segmentation method is basically used for segmenting a source image, the extracted target feature result is not accurate enough, and especially the edge contour effect of the segmented target feature is not ideal. The method adopts a cascading-type super-pixel segmentation method to carry out cascading type segmentation on a source image, finally uses an energy filtering high-low frequency fusion rule to realize sparse representation on the image in each direction and each scale, overcomes the pseudo Gibbs effect, finally improves the segmentation precision of image preprocessing, and provides a beneficial segmentation basis for subsequent identification and tracking
Drawings
FIG. 1 is a block diagram of cascaded superpixel splitting logic;
FIG. 2 is a simplified model diagram of imaging by a vehicle-mounted camera;
FIG. 3 is a single person source image from an in-vehicle perspective;
FIG. 4 is a vehicle-mounted perspective single-pedestrian example segmentation image
FIG. 5 is a vehicle view single person superpixel segmentation image;
FIG. 6 is a schematic view of a vehicle-mounted perspective single-target pedestrian fusion sign;
FIG. 7 is a vehicle view dual target pedestrian source image;
FIG. 8 is an on-board perspective dual-target pedestrian example segmentation image;
FIG. 9 is a schematic view of a vehicle-mounted perspective dual-target pedestrian fusion sign;
FIG. 10 is a dim dual target pedestrian source image;
FIG. 11 is a dim binocular example segmented image of a pedestrian;
fig. 12 is a schematic diagram of a dim-out dual target pedestrian fusion marker.
Detailed Description
The invention is further described with reference to the accompanying drawings and the specific classification procedures:
a logic block diagram of a cascade-based image super-pixel target pedestrian segmentation method is shown in FIG. 1, and the method comprises the following specific implementation steps:
step 1: sending the source image to an example segmentation channel, outputting an example segmentation image, and splitting and extracting a single target region R and a segmentation result S on the basis of the example segmentation image0；
Step 2: sending the single target region R to the super-pixel segmentation channel, and outputting the super-pixel segmentation image Q with labelsGK；
And 3, step 3: segmenting the superpixel into an image QGKMerging and reconstructing super pixel blocks with similar characteristics in middle and adjacent super pixel blocks to obtain more accurate target object contour QGK；
Step 4, dividing the result S0And a target object profile PNFusing and reconstructing a final cascade segmentation fused image Ei。
The specific scheme is as follows:
the invention is different from the existing target pedestrian segmentation algorithm, provides a cascading-type super-pixel-based target pedestrian segmentation method, and provides more accurate preprocessing information for the follow-up work of a computer vision system by establishing a target pedestrian segmentation model.
The invention realizes the above aim by the following technical scheme:
step 1, sending a source image to an example segmentation channel, outputting an example segmentation image, and splitting and extracting a single target area and a segmentation result on the basis of the example segmentation image.
(M，R，S0)＝MASKRCNN(I0) (1)
Wherein MASKRCNN is an example partition function, I0For inputting a source image (length and width 2)6Multiple source images), R is an extracted single-target region in the example segmentation image, S0The extracted segmentation result is split in example segmentation, and M is an example segmentation image.
Definition 1: the example segmentation image M is an image which is not processed after the source image is subjected to example segmentation.
Definition 2: the single target area R is an image obtained by splitting and extracting the example segmentation image M, and the range of the single target area R is certainly larger than that of the target detection frame. Formula (2) is a calculation formula of the number of the single target regions R:
B＝A±X(A∈(0，N+)，B∈(0，N+)，X∈(0，N+)) (2)
the source image comprises A target objects, B single target areas R after example segmentation, and X represents the number of error detection persons of the example segmentation target.
Definition 3: segmentation result S0For example, the contour image obtained by splitting and extracting the image M is divided, and the formula (3) is a division result S0The calculation formula of the number is as follows:
N＝J±X(N∈(0，N+)，J∈(0，N+)，X∈(0，N+)) (3)
here, the example segmented image M contains J target objects, and after the splitting extraction, N segmentation results S0In the formula, X represents the number of detection errors of the example division target.
And 2, sending the single target region R to a superpixel segmentation channel, and outputting a super pixel segmentation image with a label.
QGK＝SLIC(R) (4)
Wherein SLIC is a superpixel segmentation function, R is an extracted single-target region in an example segmentation image, QGKThe image is segmented for the superpixel containing K labeled superpixels.
Step 3, dividing the super pixel into an image QGKAnd combining the super-pixel blocks with similar characteristics in the middle adjacent super-pixel blocks to realize that K super-pixel blocks in the super-pixel segmentation image are replaced by N super-pixel coloring information blocks, and finally reconstructing a more accurate target object outline.
PN＝Cslic(QGK) (5)
Wherein, CslicFor combining functions, Q, of SLICGKSegmenting an image for K tagged superpixels, PNIs the reconstructed contour of the target object.
Step 4, dividing the result S0And a target object profile PNFusing and reconstructing a final cascade segmentation fused image Ei。
Ei＝NSST(PN，S0) (6)
Wherein NSST is a non-down-shear wave transform multi-scale analysis function, PNIs the contour of the target object, S0For splitting the extracted segmentation result in the instance segmentation, EiIs the reconstructed final fused image.
Energy filtering high-low frequency fusion rule: for the registered segmentation result S0And pre-fusing the target object contour P by adopting an energy filtering high-low frequency fusion rule, fusing the low-frequency coefficient by adopting a fusion rule based on an image guide filter in the low-frequency information fusion, and obtaining the low-frequency fusion coefficient. High frequency information fusion for superpixel QGKAnd then, the coefficients with the same label are gathered into a super-coefficient block, and the spatial frequency of each super-coefficient block is solved to obtain a high-frequency fusion coefficient. Finally, NSST inverse transformation is carried out on the high-frequency fusion coefficient and the low-frequency fusion coefficient, and a final fusion image E is reconstructedi。
The super pixel block feature merging step is as follows:
1) setting and sequencing superpixel blocks, and calculating the characteristic difference of color and space distance of adjacent superpixel blocks in the graph by the following calculation formula:
Figure BDA0002211642230000061
in the formula (9), the LAB vector adopts a CIELAB color space model, DLAB(Ri) For the inter-superpixel block color space distance, R' denotes the non-target region, liAnd ljIs a component of the pixel brightness, ai、aj、bi、bjBeing a component of a color, DXY(Ri) Is a position space distance, xi、xj、yi、yjThe vector obtains the spatial coordinate value of the pixel, D (R)i) Is the superpixel distance, δ is the distance weight coefficient, and δ belongs to (0, 1);
2) comparing the calculation result with a preset threshold, merging the target super-pixel block and the adjacent super-pixel block if the characteristic result is smaller than the threshold, ignoring the target pixel block if the characteristic result is larger than the threshold, and continuing to perform characteristic inspection on the next super-pixel block;
determining the correlation degree of the super-pixel area according to the super-pixel distance, wherein the calculation formula is as follows:
C(Ri)＝1-exp(-D(Ri)) (10)
in the formula (10), C (R)i) Representing the degree of super-pixel area correlation, D (R)i) The super-pixel distance is inversely related to the area correlation. Determining whether the superpixel blocks accord with the characteristic information of the same target or not according to the correlation;
according to the calculation of the regional relevance of all superpixels, a regional relevance threshold value is calculated by utilizing a maximum inter-class difference method, all superpixel blocks meeting the relevance threshold value are extracted as target superpixels, and the calculation formula is as follows:
Figure BDA0002211642230000062
in the formula (11), R*Representing the set of target superpixels finally acquired, RiIs the target superpixel at i, C (R)i) Representing the degree of correlation of the super-pixel area,
Figure BDA0002211642230000063
the method comprises the steps of obtaining a region correlation threshold value, wherein epsilon is a correlation threshold value coefficient, epsilon is 0.5, when epsilon is 0.5, characteristic information can be better divided into different pixel sets, each obtained subset forms a region corresponding to a real scene, the interior of each region has consistent attributes, and adjacent regions do not have the consistent attributes;
3) iterating the steps until all the superpixel blocks in the image complete one-time feature comparison, and generating a first merging result image at the moment;
4) before the second combination, refreshing the characteristic information and the rearrangement sequence of the superpixel blocks, and then combining the first combination result as an object of the combination operation until the superpixel blocks in the first combination result complete characteristic comparison to generate a second combination result graph.
The existing image segmentation method is basically used for segmenting a source image, the extracted target feature result is not accurate enough, and especially the edge contour effect of the segmented target feature is not ideal. The method adopts a cascading-type super-pixel segmentation method to carry out cascading type segmentation on a source image, and finally uses an energy filtering high-low frequency fusion rule to realize sparse representation on the image in each direction and each scale, so that the pseudo Gibbs effect is overcome, the segmentation precision of image preprocessing is finally improved, and a beneficial segmentation basis is provided for subsequent identification and tracking. That is to say, the method has the problems of small target false detection, missing detection, inaccurate segmentation result of the overlapped part and the like due to the fact that the Mask-RCNN is independently used, and the method segments the image by establishing a cascading super-pixel segmentation system. Firstly, carrying out example segmentation through Mask-RCNN, splitting and extracting a single target region R and a segmentation result S after segmentation0Then, the single target region R is subjected to superpixel segmentation to obtain a superpixel segmentation image QGKFinally, corresponding fusion rules are formulated to divide the superpixel into the image QGKAnd the segmentation result S0Fusing to reconstruct the final fused image Ei. According to the method, the super-pixel single-target segmentation is carried out on the result of the Mask-RCNN example segmentation, so that the segmentation precision can be improved, and more accurate preprocessing information is provided for the follow-up work of a computer vision system.
Example 1:
vehicle-mounted visual angle single-target pedestrian segmentation condition
A vehicle-mounted camera model is established by utilizing a geometrical relation, wherein the height of a target in an image plane is set to be h, the height of the target in the real world is 168cm, the focal length of a camera is set to be 12.25cm, the actual distance between the target and the camera is 145cm, and the pedestrian target moves at the speed of about 1.5m/s in a video and keeps moving linearly without changing the moving speed. By way of example segmentation, it can be observed that the segmentation of the contour of the target person in fig. 4 is not accurate enough. On the basis of the above-mentioned accuracy, it can be raised, and transferred into super-pixel input channel, and the null can be setThe inter-distance weight value is 65, the number of divided blocks is 225, the initial step size is 5, and superpixel division is performed. After the segmentation is finished, the registered single target region R and the segmentation result S are subjected to energy filtering high-low frequency fusion rule0Fusing to reconstruct the final fused image EiThe image contour accuracy of the cascade segmentation fusion can be obviously improved.
Example 2:
vehicle-mounted visual angle dual-target pedestrian segmentation condition
The vehicle-mounted camera model is established by using a geometrical relation, wherein the height of a target in an image plane is set to be h, the height of a target A in the real world is 168cm, the height of a target B in the real world is 165cm, the focal length of a camera is set to be 11.25cm, the actual distance between the target A and the camera is 120cm, and the actual distance between the target B and the camera is 195 cm. The video contains the double-pedestrian target, moves oppositely at the speed of about 1.4m/s, and keeps moving linearly without changing the moving speed. By way of example segmentation, it can be observed that the segmentation of the contour of the target person in fig. 8 is not accurate enough. On the basis of the above, the precision is improved, and the super-pixel is sent to a super-pixel input channel, the spatial distance weighted value is set to be 75, the number of the segmentation blocks is set to be 150, the initial step size is set to be 5, and super-pixel segmentation is carried out. After the segmentation is finished, the registered single target region R and the segmentation result S are subjected to energy filtering high-low frequency fusion rule0Fusing to reconstruct the final fused image EiThe image contour accuracy of the cascading segmentation and fusion can be obviously improved.
Example 3:
dim environment dual-target pedestrian segmentation condition
And establishing a camera model by using a geometrical relation, wherein the height of the target in an image plane is set as h, the height of a target A in the real world is 175cm, the height of a target B in the real world is 165cm, the focal length of the camera is set as 12.45cm, the actual distance between the target A and the camera is 115cm, and the actual distance between the target B and the camera is 105 cm. The twin pedestrian objects A, B move in opposite directions at a speed of about 0.5m/s, and both keep moving straight without changing the moving speed. By example segmentation, a graph can be observedThe segmentation of the contour of the target person in 11 is not accurate enough. On the basis, the precision is improved, the super-pixel is sent to a super-pixel input channel, a spatial distance weighted value is set to be 80, the number of segmentation blocks is set to be 200, the initial step size is set to be 6, and super-pixel segmentation is carried out. After the segmentation is finished, the registered single target region R and the segmentation result S are subjected to energy filtering high-low frequency fusion rule0Fusing to reconstruct the final fused image EiThe image contour accuracy of the cascading segmentation and fusion can be obviously improved.

Claims (2)

1.一种基于级联式的图像超像素目标行人分割方法，其特征在于，包括如下步骤：1. a kind of image superpixel target pedestrian segmentation method based on cascade, is characterized in that, comprises the steps: 第1步，将源图像送至实例分割通道，输出实例分割图像，对实例分割图像拆分并提取单目标区域和分割结果：The first step is to send the source image to the instance segmentation channel, output the instance segmentation image, split the instance segmentation image and extract the single target area and segmentation result: (M,R,S0)＝MASKRCNN(I0) (1)(M,R,S 0 )=MASKRCNN(I 0 ) (1) 其中，MASKRCNN为实例分割函数，I0为输入源图像，R为实例分割图像中拆分提取的单目标区域，S0为实例分割图像中拆分并提取的分割结果，M为实例分割图像；Among them, MASKRCNN is the instance segmentation function, I 0 is the input source image, R is the single target region split and extracted in the instance segmentation image, S 0 is the segmentation result split and extracted in the instance segmentation image, and M is the instance segmentation image; 其中：in: 实例分割图像M为源图像经过实例分割之后不进行任何处理的图像；The instance segmentation image M is an image that does not perform any processing on the source image after instance segmentation; 单目标区域R为实例分割图像M进行拆分并提取后得到的单目标区域图，其范围大于目标检测框范围；式(2)为单目标区域R个数的计算公式：The single target area R is the single target area map obtained by splitting and extracting the instance segmentation image M, and its range is larger than the range of the target detection frame; formula (2) is the calculation formula for the number of single target areas R: B＝A±X(A∈(0，N+)，B∈(0，N+)，X∈(0，N+)) (2)B=A±X(A∈(0, N + ), B∈(0, N + ), X∈(0, N + )) (2) 源图像含有A个目标对象，经过实例分割之后含有B个单目标区域R，式中X表示实例分割目标检测误差人数；The source image contains A target objects, and after instance segmentation, contains B single-target regions R, where X represents the number of people with instance segmentation target detection errors; 分割结果S0为实例分割图像M经过拆分并提取得到的轮廓图像，式(3)为分割结果S0个数的计算公式：The segmentation result S 0 is the contour image obtained by splitting and extracting the instance segmentation image M, and formula (3) is the calculation formula of the number of segmentation results S 0 : N＝J±X(N∈(0，N+)，J∈(0，N+)，X∈(0，N+)) (3)N=J±X(N∈(0, N + ), J∈(0, N + ), X∈(0, N + )) (3) 实例分割图像M含有J个目标对象，经过拆分并提取之后含有N个分割结果S0，式中X表示实例分割目标检测误差人数；The instance segmentation image M contains J target objects, and after being split and extracted, it contains N segmentation results S 0 , where X represents the number of people with instance segmentation target detection errors; 第2步，将单目标区域R送至超像素分割通道，输出带标签的超像素分割图像；The second step is to send the single target area R to the superpixel segmentation channel, and output the superpixel segmentation image with labels; QGK＝SLIC(R) (4)Q GK = SLIC(R) (4) 其中，SLIC为超像素分割函数，R为实例分割图像中的提取的单目标区域，QGK为含有K个带标签的超像素分割图像；Among them, SLIC is the superpixel segmentation function, R is the extracted single target region in the instance segmentation image, and Q GK is the superpixel segmentation image containing K labels; 第3步，将超像素分割图像QGK中相邻超像素块中特征相似的超像素块进行合并，用N块超像素着色信息块代替超像素分割图像中的K个超像素块，最后重构出更精准的目标对象轮廓；The third step is to merge the superpixel blocks with similar characteristics in the adjacent superpixel blocks in the superpixel segmentation image Q GK , and replace the K superpixel blocks in the superpixel segmentation image with N superpixel coloring information blocks. Construct a more accurate outline of the target object; PN＝Cslic(QGK) (5)P N =C slic (Q GK ) (5) 其中，Cslic为SLIC合并函数，QGK为含有K个带标签的超像素分割图像，PN为重构的目标对象轮廓；Among them, C slic is the SLIC merging function, Q GK is the superpixel segmentation image containing K labels, and P N is the reconstructed target object contour; 第4步，将分割结果S0和目标对象轮廓PN融合，重构出级联式分割融合图像Ei。In the fourth step, the segmentation result S 0 and the target object outline P N are fused to reconstruct a cascaded segmentation and fusion image E i . Ei＝NSST(PN,S0) (6)E i =NSST(P N ,S 0 ) (6) 其中，NSST为非下剪切波变换多尺度分析函数，PN为目标对象轮廓，S0为实例分割图像中拆分并提取的分割结果，Ei为重构的最终融合图像；Among them, NSST is the non-lower shearlet transform multi-scale analysis function, P N is the contour of the target object, S 0 is the segmentation result split and extracted from the instance segmentation image, and E i is the reconstructed final fusion image; 图像融合中使用能量滤波高低频融合规则：对经过配准的分割结果S0和目标对象轮廓P采用能量滤波高低频融合规则进行预融合，低频信息融合采用基于图像引导滤波器的融合规则对低频系数进行融合，得到低频融合系数。高频信息融合对于超像素QGK中的所有像素点，利用其空间坐标找出与其对应的高频系数，并赋予其对应的标签超像素块，然后将具有相同标签的系数聚集为一个超系数块并求取每个超系数块的空间频率得到高频融合系数。最后对高频融合系数和低频融合系数进行NSST逆变换，重构出最终的融合图像Ei。The energy filtering high and low frequency fusion rules are used in image fusion: the registered segmentation result S 0 and the target object contour P are pre-fused using the energy filtering high and low frequency fusion rules, and the low frequency information fusion adopts the fusion rule based on the image guided filter to fuse the low frequency information. The coefficients are fused to obtain low-frequency fusion coefficients. High-frequency information fusion For all the pixels in the superpixel Q GK , use its spatial coordinates to find the corresponding high-frequency coefficients, and assign the corresponding label superpixel blocks, and then gather the coefficients with the same label into a super-coefficient block and obtain the high-frequency fusion coefficient by calculating the spatial frequency of each super-coefficient block. Finally, perform NSST inverse transformation on the high-frequency fusion coefficient and the low-frequency fusion coefficient to reconstruct the final fusion image E i . 2.如权利要求1所述的基于级联式的图像超像素目标行人分割方法，其特征在于，超像素块特征合并步骤如下：2. the image superpixel target pedestrian segmentation method based on cascade type as claimed in claim 1, is characterized in that, superpixel block feature merging step is as follows: 1)将超像素块设定排序，对图中相邻超像素块进行颜色与空间距离的特征差异计算，计算公式如下：1) Set and sort the superpixel blocks, and calculate the feature difference between color and spatial distance between adjacent superpixel blocks in the figure. The calculation formula is as follows:
Figure FDA0002211642220000021
Figure FDA0002211642220000021
 式(9)中，LAB向量采用的是CIELAB的色度空间模型，DLAB(Ri)为超像素块间颜色空间距离，R′表示非目标区域，li与lj为像素亮度的分量，ai、aj、bi、bj为颜色的分量，DXY(Ri)为位置空间距离，xi、xj、yi、yj向量获取的是像素的空间坐标值，D(Ri)为超像素距离，δ是距离权值系数，且δ∈(0,1)；In formula (9), the LAB vector adopts the chromaticity space model of CIELAB, D LAB (R i ) is the color space distance between superpixel blocks, R′ represents the non-target area, and l i and l j are the components of pixel brightness. , a i , a j , b i , b j are the color components, D XY (R i ) is the position space distance, the x i , x j , y i , y j vectors obtain the spatial coordinate value of the pixel, D (R i ) is the superpixel distance, δ is the distance weight coefficient, and δ∈(0,1); 2)将计算结果与预先设定的阈值进行比较，若特征结果小于阈值则将目标超像素块与相邻超像素块合并，若大于阈值则忽略该目标像素块，继续为下一超像素块进行特征检验；2) Compare the calculation result with the preset threshold, if the feature result is less than the threshold, merge the target superpixel block with the adjacent superpixel block, if it is greater than the threshold, ignore the target pixel block, and continue to be the next superpixel block. carry out feature inspection; 根据超像素距离确定超像素区域相关度，计算公式如下：The correlation degree of the superpixel region is determined according to the superpixel distance, and the calculation formula is as follows: C(Ri)＝1-exp(-D(Ri)) (10)C(R i )=1-exp(-D(R i )) (10) 式(10)中，C(Ri)表示超像素区域相关度，D(Ri)为超像素距离，超像素距离与区域相关度成负相关。根据相关度，确定超像素块是否符合同一目标的特征信息；In formula (10), C(R i ) represents the superpixel regional correlation, D(R i ) is the superpixel distance, and the superpixel distance is negatively correlated with the regional correlation. According to the correlation, determine whether the superpixel block conforms to the characteristic information of the same target; 根据计算所有超像素的区域相关度，利用最大类间差法求出区域相关度阈值，提取所有符合相关度阈值的超像素块作为目标超像素，计算公式如下：According to the calculation of the regional correlation of all superpixels, the maximum inter-class difference method is used to obtain the regional correlation threshold, and all superpixel blocks that meet the correlation threshold are extracted as target superpixels. The calculation formula is as follows:
Figure FDA0002211642220000031
Figure FDA0002211642220000031
 式(11)中，R*表示最终获取的目标超像素的集合，Ri为i处的目标超像素，C(Ri)表示超像素区域相关度，
Figure FDA0002211642220000032
为区域相关度阈值，ε为相关度阈值系数，取ε＝0.5，当ε为0.5时，特征信息能更好地划分为不同的像素集合，得到的每个子集形成一个与现实景物相对应的区域，各个区域内部具有一致的属性，而相邻区域不具有所述一致的属性；In formula (11), R * represents the set of target superpixels finally obtained, R i is the target superpixel at i, C(R i ) represents the superpixel region correlation,
Figure FDA0002211642220000032
is the regional correlation threshold, ε is the correlation threshold coefficient, and ε=0.5. When ε is 0.5, the feature information can be better divided into different pixel sets, and each subset obtained forms a corresponding to the real scene. Areas, each area has consistent attributes, but adjacent areas do not have the same attributes; 3)迭代上述步骤，直到图像中所有超像素块都完成一次特征比较，此时生成首次合并结果图；3) Iterate the above steps until all superpixel blocks in the image complete a feature comparison, and then generate the first merged result map at this time; 4)第二次合并前，刷新超像素块的特征信息及重新编排顺序，之后将首次合并结果作为本次合并操作的对象进行合并，直至第一次合并结果中的超像素块都完成特征比较，生成第二次合并结果图。4) Before the second merging, refresh the feature information of the superpixel blocks and rearrange the order, and then use the first merged result as the object of this merge operation to merge until the superpixel blocks in the first merged result have completed the feature comparison. , to generate the second merged result graph.
CN201910900391.5A 2019-09-23 2019-09-23 Image super-pixel target pedestrian segmentation method based on cascade connection Pending CN112541930A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910900391.5A CN112541930A (en) 2019-09-23 2019-09-23 Image super-pixel target pedestrian segmentation method based on cascade connection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910900391.5A CN112541930A (en) 2019-09-23 2019-09-23 Image super-pixel target pedestrian segmentation method based on cascade connection

Publications (1)

Publication Number Publication Date
CN112541930A true CN112541930A (en) 2021-03-23

Family

ID=75012996

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910900391.5A Pending CN112541930A (en) 2019-09-23 2019-09-23 Image super-pixel target pedestrian segmentation method based on cascade connection

Country Status (1)

Country Link
CN (1) CN112541930A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298809A (en) * 2021-06-25 2021-08-24 成都飞机工业（集团）有限责任公司 Composite material ultrasonic image defect detection method based on deep learning and superpixel segmentation
CN113807349A (en) * 2021-09-06 2021-12-17 海南大学 Multi-view target recognition method and system based on Internet of Things

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150234863A1 (en) * 2014-02-18 2015-08-20 Environmental Systems Research Institute (ESRI) Automated feature extraction from imagery
CN106780505A (en) * 2016-06-20 2017-05-31 大连民族大学 Super-pixel well-marked target detection algorithm based on region energy
US20180247126A1 (en) * 2017-02-24 2018-08-30 Beihang University Method and system for detecting and segmenting primary video objects with neighborhood reversibility
CN109360163A (en) * 2018-09-26 2019-02-19 深圳积木易搭科技技术有限公司 A kind of fusion method and emerging system of high dynamic range images

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150234863A1 (en) * 2014-02-18 2015-08-20 Environmental Systems Research Institute (ESRI) Automated feature extraction from imagery
CN106780505A (en) * 2016-06-20 2017-05-31 大连民族大学 Super-pixel well-marked target detection algorithm based on region energy
US20180247126A1 (en) * 2017-02-24 2018-08-30 Beihang University Method and system for detecting and segmenting primary video objects with neighborhood reversibility
CN109360163A (en) * 2018-09-26 2019-02-19 深圳积木易搭科技技术有限公司 A kind of fusion method and emerging system of high dynamic range images

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DAWEI YANG等: "A Superpixel Segmentation Algorithm with Region Correlation Saliency Analysis for Video Pedestrian Detection", PROCEEDINGS OF THE 36TH CHINESE CONTROL CONFERENCE *
马雪 等: "粗细粒度超像素行人目标分割算法", 大连民族大学学报 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298809A (en) * 2021-06-25 2021-08-24 成都飞机工业（集团）有限责任公司 Composite material ultrasonic image defect detection method based on deep learning and superpixel segmentation
CN113298809B (en) * 2021-06-25 2022-04-08 成都飞机工业（集团）有限责任公司 Composite material ultrasonic image defect detection method based on deep learning and superpixel segmentation
CN113807349A (en) * 2021-09-06 2021-12-17 海南大学 Multi-view target recognition method and system based on Internet of Things
CN113807349B (en) * 2021-09-06 2023-06-20 海南大学 Multi-view target recognition method and system based on Internet of Things

Similar Documents

Publication Publication Date Title
Bhagat et al. Indian sign language gesture recognition using image processing and deep learning
Bruls et al. The right (angled) perspective: Improving the understanding of road scenes using boosted inverse perspective mapping
CN111079685A (en) A 3D Object Detection Method
CN108021889A (en) A kind of binary channels infrared behavior recognition methods based on posture shape and movable information
CN104966054B (en) Detection method of small target in unmanned plane visible images
CN112766291A (en) Matching method of specific target object in scene image
Liu et al. Vision-based environmental perception for autonomous driving
Ling et al. Optimization of autonomous driving image detection based on RFAConv and triplet attention
CN111582232A (en) A SLAM method based on pixel-level semantic information
Ouyang et al. A cgans-based scene reconstruction model using lidar point cloud
Hwang et al. Lidar depth completion using color-embedded information via knowledge distillation
CN104751111A (en) Method and system for recognizing human action in video
Tarchoun et al. Hand-Crafted Features vs Deep Learning for Pedestrian Detection in Moving Camera.
Lee et al. Improved real-time monocular SLAM using semantic segmentation on selective frames
Burlacu et al. Obstacle detection in stereo sequences using multiple representations of the disparity map
CN115063717A (en) Video target detection and tracking method based on key area live-action modeling
Kan et al. Target localization based on cross-view matching between UAV and satellite
CN112541930A (en) Image super-pixel target pedestrian segmentation method based on cascade connection
Harianto et al. Data augmentation and faster rcnn improve vehicle detection and recognition
Barodi et al. An enhanced artificial intelligence-based approach applied to vehicular traffic signs detection and road safety enhancement
CN115880662A (en) 3D target detection method for autonomous driving by utilizing synergistic effect of heterogeneous sensors
CN114494594A (en) Astronaut operating equipment state identification method based on deep learning
Xue et al. Multiscale feature extraction network for real-time semantic segmentation of road scenes on the autonomous robot
Lin et al. Enhancing deep-learning object detection performance based on fusion of infrared and visible images in advanced driver assistance systems
CN106650814A (en) Vehicle-mounted monocular vision-based outdoor road adaptive classifier generation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210323