CN113379922A - Foreground extraction method, device, storage medium and equipment - Google Patents
Foreground extraction method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN113379922A CN113379922A CN202110692092.4A CN202110692092A CN113379922A CN 113379922 A CN113379922 A CN 113379922A CN 202110692092 A CN202110692092 A CN 202110692092A CN 113379922 A CN113379922 A CN 113379922A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- depth image
- cloud data
- foreground
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
One or more embodiments of the present invention provide a foreground extraction method, apparatus, storage medium, and device, where the foreground extraction method includes: when a target object does not appear in a viewing area, acquiring first point cloud data aiming at the viewing area; converting the first point cloud data to obtain a first depth image; when the target object appears in a viewing area, acquiring second point cloud data aiming at the viewing area; converting the second point cloud data to obtain a second depth image; and comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result, so that the efficiency of foreground extraction is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a foreground extraction method, apparatus, storage medium, and device.
Background
In the target detection and extraction, under the condition that the background is static, any meaningful moving object is the foreground, and the foreground, namely the moving target, can be obtained through background segmentation. At present, the background segmentation speed for three-dimensional space point cloud is slow, and the foreground and the background cannot be distinguished.
Disclosure of Invention
In view of this, one or more embodiments of the present invention provide a method, an apparatus, a storage medium, and a device for foreground extraction, which improve the efficiency of foreground extraction.
One or more embodiments of the present invention further provide a foreground extraction method, including: when a target object does not appear in a viewing area, acquiring first point cloud data aiming at the viewing area; converting the first point cloud data to obtain a first depth image; when the target object appears in a viewing area, acquiring second point cloud data aiming at the viewing area; converting the second point cloud data to obtain a second depth image; and comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result.
Optionally, comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result, including: calculating the difference value of the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image; determining that the corresponding pixel in the second depth image is a foreground point when the difference value is greater than a threshold value; determining that the corresponding pixel in the second depth image is a background point when the difference is not greater than the threshold; and converting the foreground point into a point cloud to obtain the foreground point cloud in the second point cloud data.
Optionally, the method further includes: after a first depth image is obtained through conversion according to the first point cloud data, establishing a mixed Gaussian model for each pixel in the first depth image to obtain a mixed Gaussian background model corresponding to each pixel; comparing each pixel in the second depth image with each corresponding pixel in the first depth image respectively to obtain a comparison result, and determining foreground point cloud in the second point cloud data according to the comparison result, wherein the method comprises the following steps: matching each pixel in the second depth image with a Gaussian mixture background model of a corresponding pixel in the first depth image; determining points in the second depth image, which are matched with the Gaussian mixture background model of the corresponding pixels in the first depth image, as background points, and determining points which are not matched with the Gaussian mixture background model of the corresponding pixels in the first depth image as foreground points; and converting the foreground point into a point cloud to obtain a foreground point in the second point cloud data.
Optionally, the method further includes: and after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud.
Optionally, the method further includes: after foreground point clouds in the second point cloud data are determined, clustering the sight spot clouds to obtain at least two types of point clouds; and segmenting the at least two types of point clouds to obtain at least two types of independent point clouds.
Optionally, the method further includes: after at least two types of independent point clouds are obtained, the centroid and the size information of the target corresponding to each independent point cloud are calculated.
Optionally, the first point cloud data includes multi-frame point cloud data, and a first depth image is obtained according to the first point cloud data by conversion, including: after acquiring a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting the collected new frame of point cloud data into a fourth depth image; comparing first distance values of each point in the fourth depth image with second distance values of each point in the third depth image, and replacing the second distance values in the third depth image with the first distance values if the first distance values are greater than the second distance values;
and continuously acquiring new point cloud data of one frame until point cloud data of a preset frame number is acquired.
According to one or more embodiments of the present invention, there is provided a foreground extraction apparatus including: a first acquisition module configured to acquire first point cloud data for a viewing area when a target object is not present within the viewing area; the first conversion module is configured to obtain a first depth image according to the first point cloud data; a second acquisition module configured to acquire second point cloud data for a viewing area when the target object appears within the viewing area; the second conversion module is configured to obtain a second depth image according to the second point cloud data; the determining module is configured to compare each pixel in the second depth image with each corresponding pixel in the first depth image respectively to obtain a comparison result, and determine the foreground point cloud in the second point cloud data according to the comparison result.
According to one or more embodiments of the present invention, there is provided an electronic apparatus including: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space enclosed by the shell, and the processor and the memory are arranged on the circuit board; the power supply circuit is used for supplying power to each circuit or device of the electronic equipment; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, and is used for executing any one of the foreground extraction methods.
According to one or more embodiments of the invention, a non-transitory computer-readable storage medium is provided, which stores computer instructions for causing the computer to perform any one of the above-described foreground extraction methods.
According to the foreground extraction method in one or more embodiments of the invention, a first depth map of a background is obtained based on first point cloud data of the background, a second depth map is obtained based on second point cloud data containing a target object, foreground points in the second depth map are identified by comparing pixels in the first depth map with pixels in the second depth map, and processing speed and efficiency of foreground extraction are improved by converting three-dimensional point cloud data into two-dimensional depth maps for comparison.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flow diagram illustrating a foreground extraction method in accordance with one or more embodiments of the invention.
Fig. 2 is a flow diagram illustrating a foreground extraction method in accordance with one or more embodiments of the invention.
Fig. 3 is a schematic structural diagram of a foreground extracting apparatus according to one or more embodiments of the present invention.
Fig. 4 is a schematic structural diagram of an electronic device according to one or more embodiments of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart illustrating a foreground extraction method according to one or more embodiments of the present invention, as shown in fig. 1, the method including:
step 101: when a target object does not appear in a viewing area, acquiring first point cloud data aiming at the viewing area;
for example, for a detected scene, the first point cloud data may be acquired with the point clouds in the scene all static, or with no dynamic objects present in the scene.
For example, the first point cloud data may be acquired using a lidar as a sensor.
Step 102: converting the first point cloud data to obtain a first depth image;
the first point cloud data may include, for example, a plurality of frames of point clouds, and each frame of point cloud in the first point cloud data may be converted into a depth map, that is, a background may be modeled as a two-dimensional matrix, and each element in the matrix stores a distance value of a corresponding angle.
For example, according to the characteristics of the sensor acquiring the first point cloud data, the horizontal field angle h _ fov, the horizontal angular resolution h _ res, the vertical field angle v _ fov, and the vertical angular resolution v _ res, the first point cloud data is converted into the depth image, and the first depth image is obtained, and the parameters of the first depth image may be represented as follows:
number of lines of depth image
Number of columns of depth image
The formula for converting to a depth image for one point (x, y, z) is as follows:
r_idx＝rows/2-atan2(y，x)*180/pi/v_res；
depthmap[r_idx，c_idx]＝dis tan ce；
where distance represents a distance value of the depth map, r _ idx represents an index of a row that projects the point cloud data onto the depth image, and c _ idx represents an index of a column that projects the point cloud data onto the depth image.
Because multiple frames of point cloud data are adopted during background modeling, the data of each frame needs to be compared with the distance value of the background depth image to judge whether to update the data of the corresponding position of the background depth image, and the whole background modeling process can be as shown in fig. 2.
Step 103: when the target object appears in a viewing area, acquiring second point cloud data aiming at the viewing area;
the target object may be, for example, a moving object present in the scene to be detected, i.e. the second point cloud data is acquired when a moving object is present in the detected scene.
Likewise, a lidar may be used to acquire the second point cloud data.
Step 104: converting the second point cloud data to obtain a second depth image;
for example, the second depth image may be obtained by converting the second point cloud data in the same manner as the first depth image obtained by converting the first point cloud data in step 102, which is not described herein again.
Step 105: and comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result.
For example, a pixel in the second depth image that is at the same position as in the first depth image may be taken as the corresponding pixel in the first depth image.
In step 105, by comparing the difference between corresponding pixels in the two depth images, a pixel point belonging to the target object in the second depth image, i.e., a foreground point, can be accurately identified, and the identified foreground point is converted into a point cloud, so as to obtain a foreground point cloud in the second point cloud data.
According to the foreground extraction method in one or more embodiments of the invention, a first depth map of a background is obtained based on first point cloud data of the background, a second depth map is obtained based on second point cloud data containing a target object, foreground points in the second depth map are identified by comparing pixels in the first depth map with pixels in the second depth map, and processing speed and efficiency of foreground extraction are improved by converting three-dimensional point cloud data into two-dimensional depth maps for comparison.
In one or more embodiments of the present invention, comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result may include: calculating the difference value of the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image; determining that the corresponding pixel in the second depth image is a foreground point when the difference value is greater than a threshold value; determining that the corresponding pixel in the second depth image is a background point when the difference is not greater than the threshold; and converting the foreground point into a point cloud to obtain the foreground point cloud in the second point cloud data. For example, assume that the pixel value of the pixel point a in the second depth map is fabs | Depthmap [ i, j ], the pixel value of the pixel point corresponding to the pixel point a in the first depth map is Background [ i, j ], the threshold is tolerance, if fabs | Depthmap [ i, j ] -Background [ i, j ] | > tolerance is determined through comparison, the pixel point a is considered as a foreground point, otherwise, the pixel point a is considered as a Background point, and after the foreground point is determined each time, the foreground point cloud can be output.
In one or more embodiments of the present invention, the foreground extraction method may further include: and after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud to eliminate noise points and abnormal points. When determining foreground points and background points in the second depth image by comparing pixel values of pixel points in the first depth image and the second depth image, some background points may be mistakenly identified as foreground points due to factors such as sensor acquisition noise, but such mistakenly determined points are fewer in number and more discrete compared with actual foreground points. The radius filtering is based on the filtering operation of a three-dimensional space, and because the number of points to be filtered is relatively small, the speed of radius filtering performed at the moment is much higher than that of radius filtering performed on original second point cloud data, and the real-time requirement can be met.
In one or more embodiments of the present invention, the collected second point cloud data may include a plurality of target objects, and the point clouds of the plurality of target objects in the second point cloud data may be extracted together through the processing, but the point cloud data are not segmented according to different objects, so in one or more embodiments of the present invention, the foreground extracting method may further include: after foreground point clouds in the second point cloud data are determined, clustering the sight spot clouds to obtain at least two types of point clouds; and segmenting the at least two types of point clouds to obtain at least two types of independent point clouds. For example, the determined foreground point cloud can be segmented into a plurality of independent point clouds according to different objects by using distance-based euclidean clustering, so that each independent point cloud can be conveniently and independently analyzed subsequently.
In one or more embodiments of the present invention, the foreground extraction method may further include: after at least two types of independent point clouds are obtained, the centroid and the size information of the target corresponding to each independent point cloud are calculated. For example, the coordinates and length, width, high-level attribute information of the centroid of each individual point cloud may be calculated, or the position and volume of each target object may also be calculated.
In one or more embodiments of the present invention, since the background distance value in the depth image may not be in a unimodal state, it may appear in a multimodal state, for example, in a scene with wind-impulse leaves, the distance value detected by the sensor is different, but the detected distance value in the scene cannot be regarded as a foreground object. For the scene, a mixed Gaussian model can be used for processing a depth map represented by the distance, the mixed Gaussian model can be established for the collected multi-frame first point cloud data, and the model is used for representing background point cloud data, so that the mixed Gaussian background model can be called. Based on this, the foreground extraction method may further include: after a first depth image is obtained through conversion according to the first point cloud data, a mixed Gaussian background model is established based on each pixel in the first depth image, and a mixed Gaussian background model corresponding to each pixel is obtained;
for example, for each range pixel x in the first depth image described aboveiThe established Gaussian mixture model is as follows:
wherein λ isi，kDenotes xiThe weight of the kth gaussian component of the gaussian mixture model of (1); mu.si，kDenotes xiThe mean value of the kth Gaussian component of the Gaussian mixture model; sigmai，kDenotes xiThe variance of the kth gaussian component of the gaussian mixture model of (1);
initializing parameters:
and initializing a first Gaussian component in the first frame depth image by using a Gaussian mixture model of 5 Gaussian component combinations with k being 5, wherein the mean value is the value of the current pixel, the variance can take a larger value, the weight is 1, and the mean value, the variance and the weight of other Gaussian components except one Gaussian component can be 0.
Updating parameters:
the first frame depth image data is initialized by basic parameters, and the subsequent depth image updates each component parameter, wherein the updating process comprises the following steps:
step a: pixel x of each frameiCalculating the model mean mu of each Gaussian componenti，kE.g. less than 2.5 times
The current pixel is considered to match the kth gaussian component, otherwise it is considered to not match.
Updating the weight coefficient:
λi，k＝(1-α)λi，k+aMi，k；
where α is the learning rate, M at matchi，kIs 1, otherwise is 0;
after the matching is determined, the step b is carried out, if the matching is not determined, k is equal to k +1, and the step a is carried out again; if all the components have tried to match, then go to step c;
b step of updating Norm [ mu ]i，k，∑i，k]After the parameters are finished, the step a is carried out;
ui，k＝(1-α)ui，k+αxi
∑i，k＝(1-α)∑i，k+α(xi-ui，k)T(xi-ui，k)；
c, step (c): normalizing the sum of the weights of the k Gaussian components;
d, step: if all the Gaussian components in the step a are not matched, adding one Gaussian component to replace the minimum Gaussian component in the k Gaussian components;
e, step (e): after all the depth images for establishing the mixed Gaussian background model are subjected to the steps a, B, c and d, B Gaussian components with the maximum weight are selected from the k Gaussian components of each pixel to serve as the mixed Gaussian components of the real background model.
Comparing each pixel in the second depth image with each corresponding pixel in the first depth image respectively to obtain a comparison result, and determining foreground point cloud in the second point cloud data according to the comparison result, wherein the method comprises the following steps:
matching each pixel in the second depth image with a Gaussian mixture background model of a corresponding pixel in the first depth image;
determining points in the second depth image, which are matched with the Gaussian mixture background model of the corresponding pixels in the first depth image, as background points, and determining points which are not matched with the Gaussian mixture background model of the corresponding pixels in the first depth image as foreground points;
for example, the pixel point S in the second depth image and the pixel point S 'in the first depth image are corresponding pixels, and the gaussian mixture background model corresponding to the pixel point S' is M, when it is determined that the pixel point B matches the model M, the pixel point S is determined as a background point, otherwise, the pixel point S is determined as a foreground point.
When determining that each pixel point in the second depth image is a foreground point or a background point, each pixel point may be sequentially matched with the gaussian mixture background model (B components) of each pixel point in the first depth image, for example, for pixel point xiCalculating the model mean value mu of the calculated value and each Gaussian componenti，kE.g. less than 2.5 times
Then consider pixel point xiMatching with the kth Gaussian component, and considering the pixel point xiIs a background point, otherwise determines a pixel point xiIs a foreground point.
And converting the foreground point into a point cloud to obtain a foreground point in the second point cloud data.
In one or more embodiments of the present invention, the converting the first point cloud data to obtain the first depth image according to the first point cloud data may include: after acquiring a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting the collected new frame of point cloud data into a fourth depth image; comparing first distance values of each point in the fourth depth image with second distance values of each point in the third depth image, and replacing the second distance values in the third depth image with the first distance values if the first distance values are greater than the second distance values; and continuously acquiring new point cloud data of one frame until point cloud data of a preset frame number is acquired. Taking fig. 2 as an example, a process of obtaining a first depth image according to the first point cloud data conversion will be described. As shown in fig. 2, after the background modeling is started, step 201 is performed: collecting point cloud data; step 202: converting the collected point cloud data into a depth image; step 203: determining whether the current frame is the first frame, if so, executing step 204: directly taking the converted depth image as a background image, returning to step 201, and if the current frame is not the first frame, executing step 205: comparing the distance values of the points in the current converted depth image (such as the distance values of the points in the fourth depth image with the distance values of the points in the Background image (such as the third depth image), wherein the distance values of the points in the current depth image can be represented as Depthmap [ r _ idx, c _ idx ], the distance values of the points in the Background image can be represented as Background [ r _ idx, c _ idx ], if Depthmap [ r _ idx, c _ idx ] is greater than Background [ r _ idx, c _ idx ], executing step 206, replacing the distance values in the Background image at the corresponding positions with the distance values in the current depth image, judging whether the collected point cloud data is enough, if not, returning to step 201, if not, for example, reaching a preset frame number of collected point cloud data, the background modeling is ended.
Fig. 3 is a schematic structural diagram of a foreground extracting apparatus according to one or more embodiments of the present invention, and as shown in fig. 3, the apparatus 30 includes:
a first acquisition module 31 configured to acquire first point cloud data for a viewing area when a target object is not present within the viewing area;
a first conversion module 32 configured to obtain a first depth image according to the first point cloud data;
a second acquisition module 33 configured to acquire second point cloud data for a viewing area when the target object appears within the viewing area;
a second conversion module 34 configured to convert the second point cloud data into a second depth image;
a determining module 35 configured to compare each pixel in the second depth image with each corresponding pixel in the first depth image, respectively, to obtain a comparison result, and determine a foreground point cloud in the second point cloud data according to the comparison result.
In one or more embodiments of the present invention, the determining module may be specifically configured to: calculating the difference value of the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image; determining that the corresponding pixel in the second depth image is a foreground point when the difference value is greater than a threshold value; determining that the corresponding pixel in the second depth image is a background point when the difference is not greater than the threshold; and converting the foreground point into a point cloud to obtain the foreground point cloud in the second point cloud data.
In one or more embodiments of the present invention, the foreground extracting apparatus may further include:
the establishing module is configured to establish a Gaussian mixture model for each pixel in the first depth image after the first depth image is obtained through conversion according to the first point cloud data, and obtain a Gaussian mixture background model corresponding to each pixel; the determination module is specifically configured to: matching each pixel in the second depth image with a Gaussian mixture background model of a corresponding pixel in the first depth image; determining points in the second depth image, which are matched with the Gaussian mixture background model of the corresponding pixels in the first depth image, as background points, and determining points which are not matched with the Gaussian mixture background model of the corresponding pixels in the first depth image as foreground points; and converting the foreground point into a point cloud to obtain a foreground point in the second point cloud data.
In one or more embodiments of the present invention, the foreground extracting apparatus may further include: a filtering module configured to: and after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud.
In one or more embodiments of the present invention, the foreground extracting apparatus may further include: a clustering module configured to: after foreground point clouds in the second point cloud data are determined, clustering the sight spot clouds to obtain at least two types of point clouds; and the segmentation module is configured to segment the at least two types of point clouds to obtain at least two types of independent point clouds.
In one or more embodiments of the present invention, the foreground extracting apparatus may further include: a computing module configured to: after at least two types of independent point clouds are obtained, the centroid and the size information of the target corresponding to each independent point cloud are calculated.
In one or more embodiments of the present invention, the foreground extracting apparatus may further include: the first point cloud data comprises a plurality of frames of point cloud data, the conversion module configured to: after acquiring a frame of point cloud data, converting the frame of point cloud data into a third depth image; converting the collected new frame of point cloud data into a fourth depth image; comparing first distance values of each point in the fourth depth image with second distance values of each point in the third depth image, and replacing the second distance values in the third depth image with the first distance values if the first distance values are greater than the second distance values; and continuously acquiring new point cloud data of one frame until point cloud data of a preset frame number is acquired.
One or more embodiments of the present invention also provide an electronic device including: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space enclosed by the shell, and the processor and the memory are arranged on the circuit board; the power supply circuit is used for supplying power to each circuit or device of the electronic equipment; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, and is used for executing any one of the foreground extraction methods.
One or more embodiments of the present invention also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform any one of the foreground extraction methods described above.
Accordingly, as shown in fig. 4, one or more embodiments of the present invention further provide an electronic device, which may include: the device comprises a shell 41, a processor 42, a memory 43, a circuit board 44 and a power circuit 45, wherein the circuit board 44 is arranged inside a space enclosed by the shell 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; a power supply circuit 45 for supplying power to each circuit or device of the server; the memory 43 is used for storing executable program code; the processor 42 executes a program corresponding to the executable program code by reading the executable program code stored in the memory 43, for executing any one of the foreground extracting methods provided by the foregoing embodiments.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, features in the above embodiments or in different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the present disclosure as described above, which are not provided in detail for the sake of brevity.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.
In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
For convenience of description, the above devices are described separately in terms of functional division into various units/modules. Of course, the functionality of the units/modules may be implemented in one or more software and/or hardware implementations of the invention.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A foreground extraction method, comprising:
when a target object does not appear in a viewing area, acquiring first point cloud data aiming at the viewing area;
converting the first point cloud data to obtain a first depth image;
when the target object appears in a viewing area, acquiring second point cloud data aiming at the viewing area;
converting the second point cloud data to obtain a second depth image;
and comparing each pixel in the second depth image with each corresponding pixel in the first depth image to obtain a comparison result, and determining the foreground point cloud in the second point cloud data according to the comparison result.
2. The method of claim 1, wherein comparing each pixel in the second depth image with a corresponding pixel in the first depth image to obtain a comparison result, and determining a foreground point cloud in the second point cloud data according to the comparison result comprises:
calculating the difference value of the pixel value of each pixel in the second depth image and the pixel value of each corresponding pixel in the first depth image;
determining that the corresponding pixel in the second depth image is a foreground point when the difference value is greater than a threshold value;
determining that the corresponding pixel in the second depth image is a background point when the difference is not greater than the threshold;
and converting the foreground point into a point cloud to obtain the foreground point cloud in the second point cloud data.
3. The method of claim 1, further comprising:
after a first depth image is obtained through conversion according to the first point cloud data, establishing a mixed Gaussian model for each pixel in the first depth image to obtain a mixed Gaussian background model corresponding to each pixel;
comparing each pixel in the second depth image with each corresponding pixel in the first depth image respectively to obtain a comparison result, and determining foreground point cloud in the second point cloud data according to the comparison result, wherein the method comprises the following steps:
matching each pixel in the second depth image with a Gaussian mixture background model of a corresponding pixel in the first depth image;
determining points in the second depth image, which are matched with the Gaussian mixture background model of the corresponding pixels in the first depth image, as background points, and determining points which are not matched with the Gaussian mixture background model of the corresponding pixels in the first depth image as foreground points;
and converting the foreground point into a point cloud to obtain a foreground point in the second point cloud data.
4. The method of claim 1, further comprising:
and after determining the foreground point cloud in the second point cloud data, performing radius filtering on the foreground point cloud.
5. The method of claim 1, further comprising:
after foreground point clouds in the second point cloud data are determined, clustering the sight spot clouds to obtain at least two types of point clouds;
and segmenting the at least two types of point clouds to obtain at least two types of independent point clouds.
6. The method of claim 5, further comprising:
after at least two types of independent point clouds are obtained, the centroid and the size information of the target corresponding to each independent point cloud are calculated.
7. The method of any one of claims 1 to 6, wherein the first point cloud data comprises a plurality of frames of point cloud data, and the converting from the first point cloud data to obtain the first depth image comprises:
after acquiring a frame of point cloud data, converting the frame of point cloud data into a third depth image;
converting the collected new frame of point cloud data into a fourth depth image;
comparing first distance values of each point in the fourth depth image with second distance values of each point in the third depth image, and replacing the second distance values in the third depth image with the first distance values if the first distance values are greater than the second distance values;
and continuously acquiring new point cloud data of one frame until point cloud data of a preset frame number is acquired.
8. A foreground extraction apparatus, comprising:
a first acquisition module configured to acquire first point cloud data for a viewing area when a target object is not present within the viewing area;
the first conversion module is configured to obtain a first depth image according to the first point cloud data;
a second acquisition module configured to acquire second point cloud data for a viewing area when the target object appears within the viewing area;
the second conversion module is configured to obtain a second depth image according to the second point cloud data;
the determining module is configured to compare each pixel in the second depth image with each corresponding pixel in the first depth image respectively to obtain a comparison result, and determine the foreground point cloud in the second point cloud data according to the comparison result.
9. An electronic device, characterized in that the electronic device comprises: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space enclosed by the shell, and the processor and the memory are arranged on the circuit board; the power supply circuit is used for supplying power to each circuit or device of the electronic equipment; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the foreground extracting method of any one of the preceding claims 1 to 7.
10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the foreground extraction method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110692092.4A CN113379922A (en) | 2021-06-22 | 2021-06-22 | Foreground extraction method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110692092.4A CN113379922A (en) | 2021-06-22 | 2021-06-22 | Foreground extraction method, device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113379922A true CN113379922A (en) | 2021-09-10 |
Family
ID=77578307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110692092.4A Pending CN113379922A (en) | 2021-06-22 | 2021-06-22 | Foreground extraction method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113379922A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113963000A (en) * | 2021-10-21 | 2022-01-21 | 北京字节跳动网络技术有限公司 | Image segmentation method, device, electronic equipment and program product |
-
2021
- 2021-06-22 CN CN202110692092.4A patent/CN113379922A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113963000A (en) * | 2021-10-21 | 2022-01-21 | 北京字节跳动网络技术有限公司 | Image segmentation method, device, electronic equipment and program product |
| CN113963000B (en) * | 2021-10-21 | 2024-03-15 | 抖音视界有限公司 | Image segmentation method, device, electronic equipment and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111199564B (en) | Indoor positioning method and device of intelligent mobile terminal and electronic equipment | |
| CN108268823B (en) | Target re-identification method and device | |
| CN112967341B (en) | Indoor visual positioning method, system, equipment and storage medium based on live-action image | |
| CN106815323B (en) | Cross-domain visual retrieval method based on significance detection | |
| CN106534616A (en) | Video image stabilization method and system based on feature matching and motion compensation | |
| CN108109163A (en) | A kind of moving target detecting method for video of taking photo by plane | |
| CN107451156B (en) | Image re-identification method and identification device | |
| CN111091101B (en) | High-precision pedestrian detection method, system and device based on one-step method | |
| CN113065397B (en) | Pedestrian detection method and device | |
| CN115240149A (en) | Three-dimensional point cloud detection and identification method and device, electronic equipment and storage medium | |
| CN110930453A (en) | Target object positioning method and device and readable storage medium | |
| CN110942473A (en) | Moving target tracking detection method based on characteristic point gridding matching | |
| CN112802108A (en) | Target object positioning method and device, electronic equipment and readable storage medium | |
| CN112287906A (en) | Template matching tracking method and system based on depth feature fusion | |
| CN115620022A (en) | Object detection method, device, equipment and storage medium | |
| CN111783834A (en) | Heterogeneous image matching method based on joint graph spectrum feature analysis | |
| CN113379922A (en) | Foreground extraction method, device, storage medium and equipment | |
| CN117495891B (en) | Point cloud edge detection method and device and electronic equipment | |
| CN108764248B (en) | Image feature point extraction method and device | |
| CN104104911A (en) | Timestamp eliminating and resetting method in panoramic image generation process and system thereof | |
| CN116091781B (en) | Data processing method and device for image recognition | |
| CN110930519B (en) | Semantic ORB-SLAM sensing method and device based on environment understanding | |
| CN114494887A (en) | Remote sensing image classification method and device, computer equipment and readable storage medium | |
| CN111428565B (en) | Point cloud identification point positioning method and device based on deep learning | |
| CN113129332A (en) | Method and apparatus for performing target object tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |