CN109886864B - Privacy mask processing method and device - Google Patents
Privacy mask processing method and device Download PDFInfo
- Publication number
- CN109886864B CN109886864B CN201711273872.5A CN201711273872A CN109886864B CN 109886864 B CN109886864 B CN 109886864B CN 201711273872 A CN201711273872 A CN 201711273872A CN 109886864 B CN109886864 B CN 109886864B
- Authority
- CN
- China
- Prior art keywords
- target
- confidence
- target area
- privacy
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a privacy-mask processing method and device, and belongs to the field of image processing. The method comprises the following steps: determining a target region in which an object to be shielded possibly exists and a confidence coefficient of the target region from the image, wherein the confidence coefficient of the target region is used for indicating the probability that the target region possibly has the object to be shielded; determining privacy masking processing parameters of the target area based on the confidence of the target area; and performing privacy masking processing on the target area based on the privacy masking processing parameter of the target area. The invention realizes the privacy shielding processing with variable intensity on the target area according to the confidence coefficient of the target area, and improves the flexibility and the accuracy of the privacy shielding processing.
Description
Technical Field
The embodiment of the invention relates to the field of image processing, in particular to a privacy masking processing method and device.
Background
With the increasing importance of the public on privacy protection, the application of privacy masking in the field of image processing is also more and more extensive. The privacy masking is a method for performing masking or blurring processing on an area where a person or a vehicle in an image waits for a masking target in order to protect personal privacy, and is mainly applied to video monitoring images.
In the related art, a privacy-mask processing method is provided, including: and detecting the target to be shielded in the image by adopting intelligent means such as computer vision and the like so as to detect the area with the target to be shielded, and then carrying out uniform privacy shielding treatment on the detected area.
However, in practical applications, due to the limitation of the detection capability, and the situation that the target to be masked exists in a local area or the target to be masked is too small often occurs in an image, the situation that false detection or missing detection occurs often, and the privacy masking accuracy is reduced. For example, when there is false detection, some unnecessary areas are masked, which may cause information loss; some areas that should be obscured when there is an omission are not obscured, possibly resulting in privacy disclosure.
Disclosure of Invention
In order to solve the problem of low privacy masking accuracy in the related art, embodiments of the present invention provide a privacy masking processing method and apparatus. The technical scheme is as follows:
in a first aspect, a privacy-mask processing method is provided, and the method includes:
determining a target region in which an object to be shielded possibly exists and a confidence coefficient of the target region from an image, wherein the object to be shielded is the object to be shielded, and the confidence coefficient of the target region is used for indicating the probability that the target region possibly exists in the object to be shielded;
determining privacy masking processing parameters for the target area based on the confidence level of the target area;
and carrying out privacy masking processing on the target area based on the privacy masking processing parameter of the target area.
Optionally, the determining, from the image, a target region where an object to be occluded may exist and a confidence of the target region includes:
detecting a candidate frame possibly having the target to be shielded and the confidence degree of the candidate frame from the image based on a target detection algorithm aiming at the target to be shielded;
determining a region corresponding to the candidate frame as the target region;
determining the confidence of the candidate box as the confidence of the target area.
Optionally, the determining, from the image, a target region where an object to be occluded may exist and a confidence of the target region includes:
determining position information and confidence degrees of a plurality of target pixel points from the image based on a target segmentation algorithm aiming at the target to be shielded, wherein the target pixel points refer to the pixel points belonging to the target to be shielded in the image;
determining the target area based on the position information of the target pixels;
and determining the confidence of the target area based on the confidence of the target pixel points.
Optionally, the determining the confidence of the target region based on the confidences of the plurality of target pixel points includes:
determining the average confidence of the target pixel points;
determining the average confidence as the confidence of the target region.
Optionally, the determining the confidence of the target region based on the confidences of the plurality of target pixel points includes:
determining the confidence of each pixel point in the target area based on the confidence of the target pixel points;
the determining privacy masking processing parameters for the target area based on the confidence level for the target area comprises:
determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area;
the privacy-mask processing on the target area based on the privacy-mask processing parameter of the target area comprises:
and respectively carrying out privacy masking processing on each pixel point in the target area based on the privacy masking processing parameter of each pixel point in the target area.
Optionally, the determining privacy-mask processing parameters of the target area based on the confidence level of the target area includes:
determining privacy masking processing parameters corresponding to the confidence degrees of the target area based on the stored correspondence between the confidence degrees and the privacy masking processing parameters;
and determining the privacy masking processing parameter corresponding to the confidence degree of the target area as the privacy masking processing parameter of the target area.
Optionally, when the privacy-mask processing parameter of the target area is a parameter matrix, performing privacy-mask processing on the target area based on the privacy-mask processing parameter of the target area includes:
performing convolution processing on the pixel value of each pixel point in the target area based on the parameter matrix;
and replacing the pixel value of each pixel point in the target area with the pixel value after convolution processing so as to finish privacy masking processing of the target area.
In a second aspect, there is provided a privacy-mask processing apparatus, the apparatus comprising:
the device comprises a first determining module, a second determining module and a third determining module, wherein the first determining module is used for determining a target region in which an object to be shielded possibly exists and a confidence coefficient of the target region, the object to be shielded is the object to be shielded, and the confidence coefficient of the target region is used for indicating the probability that the object to be shielded possibly exists in the target region;
a second determination module to determine privacy-mask processing parameters for the target area based on the confidence level of the target area;
and the processing module is used for carrying out privacy masking processing on the target area based on the privacy masking processing parameters of the target area.
Optionally, the first determining module includes:
a first determining unit, configured to detect, from the image, a candidate frame in which the object to be occluded may exist and a confidence of the candidate frame based on an object detection algorithm for the object to be occluded;
a second determining unit, configured to determine a region corresponding to the candidate frame as the target region;
a third determining unit, configured to determine the confidence level of the candidate box as the confidence level of the target region.
Optionally, the first determining module includes:
a fourth determining unit, configured to determine, based on a target segmentation algorithm for the target to be masked, position information and confidence levels of a plurality of target pixel points from the image, where a target pixel point is a pixel point in the image that belongs to the target to be masked;
a fifth determining unit configured to determine the target area based on position information of the plurality of target pixel points;
a sixth determining unit, configured to determine, based on the confidences of the multiple target pixel points, the confidence of the target region.
Optionally, the sixth determining unit is specifically configured to:
determining the average confidence of the target pixel points;
determining the average confidence as the confidence of the target region.
Optionally, the sixth determining unit is specifically configured to:
determining the confidence of each pixel point in the target area based on the confidence of the target pixel points;
the second determining module is specifically configured to:
determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area;
the processing module is specifically configured to:
and respectively carrying out privacy masking processing on each pixel point in the target area based on the privacy masking processing parameter of each pixel point in the target area.
Optionally, the second determining module is specifically configured to:
determining privacy masking processing parameters corresponding to the confidence degrees of the target area based on the stored correspondence between the confidence degrees and the privacy masking processing parameters;
and determining the privacy masking processing parameter corresponding to the confidence degree of the target area as the privacy masking processing parameter of the target area.
Optionally, when the privacy-mask processing parameter of the target area is a parameter matrix, the processing module is specifically configured to:
performing convolution processing on the pixel value of each pixel point in the target area based on the parameter matrix;
and replacing the pixel value of each pixel point in the target area with the pixel value after convolution processing so as to finish privacy masking processing of the target area.
In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor is configured to perform any of the privacy-mask processing methods described in the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the privacy-mask processing methods described in the first aspect above.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the target area possibly provided with the target to be shielded and the confidence coefficient of the target area can be determined from the image, then the privacy shielding processing parameter is determined based on the confidence coefficient of the target area, and the privacy shielding processing is carried out on the target area based on the determined privacy shielding processing parameter, so that the privacy shielding processing can be carried out on all the target areas possibly provided with the target to be shielded, the privacy shielding processing with variable strength can be carried out on the target area according to the confidence coefficient, the problem of low privacy shielding accuracy caused by the problems of false detection and missing detection in the target detection process is avoided, and the flexibility and the accuracy of the privacy shielding processing are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1A is a schematic logical structure diagram of a privacy-mask processing system according to an embodiment of the present invention;
fig. 1B is a flowchart of a privacy-mask processing method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a privacy-mask processing apparatus according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of another privacy-mask processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Before explaining the embodiments of the present invention in detail, an application scenario of the embodiments of the present invention will be described.
In the field of security monitoring and the like, in order to protect the privacy of a user, privacy shielding processing is generally required to be performed on a local area of a monitored image. For example, for a collected surveillance video, privacy shielding processing needs to be performed on an area where a to-be-shielded target exists, such as a human face, a vehicle window, a license plate and the like, which can reveal the privacy of a user, exists in each video frame in the surveillance video, and then the surveillance video is played according to the authority of the user after the privacy shielding processing. For example, for a user with a privacy viewing authority such as police, an original video image without privacy masking processing may be played for the user, and for a user without a privacy viewing authority, a video image after privacy masking processing may be played for the user. Of course, the method provided by the embodiment of the present invention may also be applied to other scenarios, and the embodiment of the present invention does not limit this.
After briefly introducing the application scenario of the embodiment of the present invention, a system architecture related to the embodiment of the present invention will be introduced next.
Fig. 1A is a schematic logical structure diagram of a privacy mask processing system according to an embodiment of the present invention, and as shown in fig. 1A, the privacy mask processing system includes a detection module 11, a mapping module 12, and a processing module 13.
The detection module 11 is configured to determine, from an image (input image) to be subjected to privacy masking processing, a target area where an object to be masked may exist and a confidence level of the target area, where the confidence level of the target area is used to indicate a probability that the target area may exist the object to be masked.
Specifically, the detection module 11 may perform intelligent analysis on the input image to obtain target parameter information of the target to be masked. The target parameter information may include position information a of the target to be masked and a confidence B of the corresponding target region, where the position information a of the target to be masked is used to indicate that the target region of the target to be masked may exist. The intelligent analysis method used by the detection module 11 may be a target detection algorithm or a target segmentation algorithm, and the like, which is not limited in the embodiment of the present invention.
The mapping module 12 is configured to determine privacy-mask processing parameters for the target area based on the confidence level of the target area. Specifically, the mapping module 12 may determine the privacy-mask processing parameter corresponding to the confidence of the target area based on the correspondence between the stored confidence and the privacy-mask processing parameter, and then determine the privacy-mask processing parameter corresponding to the confidence of the target area as the privacy-mask processing parameter of the target area.
The processing module 13 is configured to perform privacy masking processing on the target area in the input image based on the privacy masking processing parameter of the target area, and output a masked image. Specifically, the processing module 13 may perform privacy masking processing F on pixel points on an image included in the position information a of the target to be masked, and generate a masked privacy-masked image X, that is, X ═ F (a). It should be noted that, in the conventional privacy-mask processing method, privacy-mask processing is generally performed on all the pixel points in the pixel a according to the same method or the same privacy-mask processing parameter, and the processing procedure is unrelated to the confidence level. The embodiment of the present invention may perform privacy-mask processing using the privacy-mask processing parameters output by the mapping module 12, where the processing procedure is related to the confidence level, that is, X ═ F (a, B).
Fig. 1B is a flowchart of a privacy-mask processing method according to an embodiment of the present invention, where the method is applied to a privacy-mask processing apparatus, which may be a terminal, a server, a camera, or the like, and is not limited herein, as shown in fig. 1B, the method includes:
step 101: a target region in which an object to be occluded may exist and a confidence level of the target region indicating a probability that the object to be occluded may exist in the target region are determined from the image.
The image is an image to be subjected to privacy masking processing, and specifically may be a video frame image in a video or the like. The target to be masked is a preset target which needs privacy masking processing, such as a face, a window or a license plate, which may reveal the privacy of the user, and the target to be masked may be preset by the privacy masking processing device in advance or may be set by the user, which is not limited in the embodiment of the present invention.
Wherein determining the confidence level of the target region may comprise: and determining the confidence of the whole target area or determining the confidence of each pixel point in the target area. The confidence of the target area is used for indicating the probability that the target area may have the target to be shielded, and the confidence of each pixel point is used for indicating the probability that the pixel point belongs to the target to be shielded.
In the embodiment of the invention, the image can be intelligently analyzed to obtain the target area in which the target to be shielded possibly exists in the image and the confidence coefficient of the target area. Specifically, the image may be intelligently analyzed to obtain the position information and the corresponding confidence of the target to be shielded, then the region indicated by the position information of the target to be shielded is determined as the target region where the target to be shielded may exist, and the confidence corresponding to the position information of the target to be shielded is determined as the confidence of the target region.
In practical application, the used intelligent analysis algorithm may be a target detection algorithm or a target segmentation algorithm, and other intelligent analysis algorithms may also be adopted, which is not limited in the embodiment of the present invention. The method for determining the confidence degrees of the target region and the target region from the image is different according to different intelligent analysis algorithms, and may specifically include the following two implementation manners:
the first implementation mode comprises the following steps: detecting a candidate frame possibly having the target to be shielded and the confidence coefficient of the candidate frame from the image based on a target detection algorithm aiming at the target to be shielded; determining the area corresponding to the candidate frame as the target area; the confidence of the candidate box is determined as the confidence of the target region.
The second implementation mode comprises the following steps: determining position information and confidence degrees of a plurality of target pixel points from the image based on a target segmentation algorithm aiming at the target to be shielded, wherein the target pixel points refer to pixel points belonging to the target to be shielded in the image, determining the target area based on the position information of the plurality of target pixel points, and determining the confidence degree of the target area based on the confidence degrees of the plurality of target pixel points.
In the process of performing target segmentation on an image based on a target segmentation algorithm for the target to be masked, the category to which each pixel point in the image belongs can be determined first, then the pixel point of which the category is the category of the target to be masked is determined from all the pixel points included in the image and is used as a target pixel point, and the confidence coefficient of each target pixel point is determined.
Specifically, determining the confidence of the target region based on the confidence of the target pixel points includes: determining the average confidence of the target pixel points, and determining the average confidence as the confidence of the target area; or determining the confidence of each pixel point in the target area based on the confidence of the target pixel points. That is, the confidence of the entire target region may be determined based on the confidence of the plurality of target pixel points, or the confidence of each pixel point in the target region may be determined separately.
Step 102: determining privacy masking processing parameters for the target area based on the confidence level for the target area.
The privacy-mask processing parameter of the target area is a parameter used when privacy-mask processing is performed on the target area. In practical applications, the privacy-mask processing parameter may be a linear parameter or a non-linear parameter, which is not limited in the embodiment of the present invention.
Specifically, determining privacy-mask processing parameters for the target area based on the confidence level of the target area may include: determining privacy masking processing parameters corresponding to the confidence degrees of the target area based on the stored correspondence between the confidence degrees and the privacy masking processing parameters; and determining the privacy masking processing parameter corresponding to the confidence degree of the target area as the privacy masking processing parameter of the target area.
The correspondence between the confidence degrees and the privacy-mask processing parameters may be that the smaller the confidence degree is, the smaller the privacy-mask processing intensity indicated by the corresponding privacy-mask processing parameter is, and the larger the confidence degree is, the larger the privacy-mask processing intensity indicated by the corresponding privacy-mask processing parameter is, so that the intensity of the privacy-mask processing may be determined according to the confidence degree.
The correspondence between the stored confidence degrees and the privacy-mask processing parameters may be a correspondence list or a designated algorithm, and the correspondence list may store a plurality of confidence degrees with different sizes and corresponding privacy-mask processing parameters, and the designated algorithm is related to the confidence degrees.
Specifically, when the correspondence between the stored confidence level and the privacy-mask processing parameter is a correspondence list, the privacy-mask processing parameter corresponding to the confidence level of the target area may be searched from the list, and when the correspondence between the stored confidence level and the privacy-mask processing parameter is a specified algorithm, the confidence level of the target area may be substituted into the specified algorithm, and then the privacy-mask processing parameter of the target area is calculated by the specified algorithm.
For example, when the privacy-mask processing parameter is a parameter matrix M, and the number of non-0 parameters or the non-0 parameter variance in the parameter matrix M is related to the confidence B of the target region, the number of non-0 parameters or the non-0 parameter variance of the parameter matrix M may be determined according to the confidence B of the target region, and then the final parameter matrix M may be determined according to the number of non-0 parameters or the non-0 parameter variance. Wherein the number of non-0 parameters or the non-0 parameter variance is used to indicate the strength of the privacy masking process. Preferably, as B increases, the number of non-0 parameters of M increases or the non-0 parameter variance decreases.
Further, when the confidence of each pixel point in the target region is determined through step 101, step 102 may include: and determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area.
Specifically, determining the privacy masking processing parameter of each pixel point based on the confidence of each pixel point in the target region may include: determining privacy masking processing parameters corresponding to the confidence of each pixel point based on the corresponding relation between the stored confidence and the privacy masking processing parameters; and determining the privacy masking processing parameter corresponding to the confidence of each pixel point as the privacy masking processing parameter of the pixel point.
Step 103: and performing privacy-mask processing on the target area based on the privacy-mask processing parameter of the target area.
The privacy masking processing performed on the target area may be masking processing or fuzzification processing or the like on the target area to mask the content of the target area and avoid privacy disclosure. For example, when the object to be masked is a face object, the privacy masking processing mode may be a mosaic processing mode or the like.
Specifically, based on the privacy-mask processing parameter of the target area, privacy-mask processing may be performed on each pixel point in the target area. That is, the entire target area may be subjected to the uniform privacy-mask processing based on the determined privacy-mask processing parameter of the target area.
In a possible implementation manner, when the privacy-mask processing parameter is a parameter matrix, the privacy-mask processing parameter may act on a pixel point in the target area in a sliding window manner to complete the privacy-mask processing of the target area. For example, when the privacy masking processing parameter is a parameter matrix and the privacy masking processing is convolution processing, the convolution processing may be performed on the pixel value of each pixel point in the target area based on the parameter matrix to obtain a pixel value after the convolution processing, and then the pixel value of each pixel point in the target area is replaced with the pixel value after the convolution processing to complete the privacy masking processing of the target area.
In one embodiment, it is assumed that the privacy masking parameter of the pixel point with confidence levels a to B in the target area is set to 0, the privacy masking parameter of the pixel point with confidence levels B to C is set to 1, and the privacy masking parameter of the pixel point with confidence levels C to D is set to 2. Taking the privacy masking process as the blurring process as an example, the pixels with confidence levels from a to B may not be processed, the pixels with confidence levels from B to C may be subjected to the level 1 blurring process, and the pixels with confidence levels from C to D may be subjected to the level 2 blurring process, according to the set privacy masking process parameters. The way of the blurring process may include, but is not limited to, a mosaic process, a blacking process, and the like, and is not limited herein.
Alternatively, the color of the pixel point may be adjusted according to a preset mapping relationship between the privacy-mask processing parameter and the pixel value.
For example, the pixel points with confidence levels B to C may be blackened. For another example, the values of the pixels with the confidence levels C to D are normalized so that the colors thereof are aligned and the picture is blurred.
In a conventional privacy-masking processing method, target detection is generally performed on an object to be masked in an image, and in a detection process, only an area with a confidence level greater than a preset threshold is generally determined as an area where the object to be masked exists, whether privacy-masking processing is performed or not and which areas are subjected to privacy-masking processing are determined according to the existence of a detection result, and privacy-masking processing is performed on all detected areas by using the same method. For example, for a face target appearing from far to near in a surveillance video, the face target is small when the distance is far away, a face image may not be detected through target detection, and only when the distance is near and the face target is large, the face image can be accurately detected, so that privacy shielding can not be performed on the face target when the distance is far away, and privacy shielding can be performed only when the distance is near, so that privacy leakage is caused.
In the embodiment of the invention, privacy shielding processing can be carried out on all target areas where targets to be shielded possibly exist, and variable-strength privacy shielding processing can be carried out on the target areas according to the confidence coefficient, so that privacy disclosure is avoided, and the accuracy of the privacy shielding processing is improved. For example, for a face target appearing from far to near in a monitoring video, when the distance is far and the face target is small, the confidence coefficient is low, low-intensity privacy masking processing can be performed on the face target at the moment, when the distance is near and the face target is large, the confidence coefficient is large, high-intensity privacy masking processing can be performed on the face target at the moment, and privacy disclosure is avoided.
Further, when the privacy-masking processing parameter of each pixel point in the target area is determined based on step 102, privacy-masking processing may be performed on each pixel point based on the privacy-masking processing parameter of each pixel point. The privacy shielding processing is carried out on each pixel point based on the privacy shielding processing parameter of each pixel point, so that the accuracy of the privacy shielding processing is further improved.
According to the embodiment of the invention, the target area possibly provided with the target to be shielded and the confidence coefficient of the target area can be determined from the image, then the privacy shielding processing parameter is determined based on the confidence coefficient of the target area, and the privacy shielding processing is carried out on the target area based on the determined privacy shielding processing parameter, so that the privacy shielding processing can be carried out on all the target areas possibly provided with the target to be shielded, the privacy shielding processing with variable strength can be carried out on the target area according to the confidence coefficient, the problem of low privacy shielding accuracy caused by the problems of false detection and missing detection in the target detection process is avoided, and the flexibility and the accuracy of the privacy shielding processing are improved.
Fig. 2 is a schematic structural diagram of a privacy-mask processing apparatus according to an embodiment of the present invention, and as shown in fig. 2, the privacy-mask processing apparatus includes a first determining module 201, a second determining module 202, and a processing module 203.
A first determining module 201, configured to determine, from an image, a target region where an object to be occluded may exist and a confidence of the target region, where the confidence of the target region is used to indicate a probability that the target region may exist the object to be occluded;
a second determining module 202 for determining privacy-mask processing parameters for the target area based on the confidence level of the target area;
and the processing module 203 is configured to perform privacy masking processing on the target area based on the privacy masking processing parameter of the target area.
The first determining module 201 corresponds to the detecting module 11 in fig. 1A, the second determining module 202 corresponds to the mapping module 12 in fig. 1A, and the processing module 203 corresponds to the processing module 13 in fig. 1A.
Optionally, the first determining module 201 includes:
a first determining unit, configured to detect, from the image, a candidate frame in which the target to be occluded may exist and a confidence of the candidate frame based on a target detection algorithm for the target to be occluded;
a second determining unit, configured to determine a region corresponding to the candidate frame as the target region;
and the third determining unit is used for determining the confidence of the candidate frame as the confidence of the target area.
Optionally, the first determining module 201 includes:
a fourth determining unit, configured to determine, based on a target segmentation algorithm for the target to be masked, position information and confidence levels of a plurality of target pixel points from the image, where a target pixel point is a pixel point in the image that belongs to the target to be masked;
a fifth determining unit configured to determine the target area based on the position information of the target pixels;
and the sixth determining unit is used for determining the confidence of the target area based on the confidence of the target pixel points.
Optionally, the sixth determining unit is specifically configured to:
determining the average confidence of the target pixel points;
the average confidence is determined as the confidence of the target region.
Optionally, the sixth determining unit is specifically configured to:
determining the confidence of each pixel point in the target area based on the confidence of the target pixel points;
the second determining module 202 is specifically configured to:
determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area;
the processing module 203 is specifically configured to:
and respectively carrying out privacy masking processing on each pixel point in the target area based on the privacy masking processing parameter of each pixel point in the target area.
Optionally, the second determining module 202 is specifically configured to:
determining privacy masking processing parameters corresponding to the confidence degrees of the target area based on the corresponding relation between the stored confidence degrees and the privacy masking processing parameters;
and determining the privacy masking processing parameter corresponding to the confidence of the target area as the privacy masking processing parameter of the target area.
Optionally, when the privacy-mask processing parameter of the target area is a parameter matrix, the processing module 203 is specifically configured to:
performing convolution processing on the pixel value of each pixel point in the target area based on the parameter matrix;
and replacing the pixel value of each pixel point in the target area with the pixel value after convolution processing so as to finish the privacy masking processing of the target area.
According to the embodiment of the invention, the target area possibly provided with the target to be shielded and the confidence coefficient of the target area can be determined from the image, then the privacy shielding processing parameter is determined based on the confidence coefficient of the target area, and the privacy shielding processing is carried out on the target area based on the determined privacy shielding processing parameter, so that the privacy shielding processing can be carried out on all the target areas possibly provided with the target to be shielded, the privacy shielding processing with variable strength can be carried out on the target area according to the confidence coefficient, the problem of low privacy shielding accuracy caused by the problems of false detection and missing detection in the target detection process is avoided, and the flexibility and the accuracy of the privacy shielding processing are improved.
It should be noted that: in the privacy-mask processing apparatus provided in the above embodiment, when performing privacy-mask processing, only the division of the above functional modules is taken as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the above described functions. In addition, the privacy-mask processing apparatus and the privacy-mask processing method provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
Fig. 3 is a schematic structural diagram of another privacy-mask processing apparatus according to an embodiment of the present invention, where the apparatus may be an electronic device such as a terminal, a server, or a camera. Referring to fig. 3, the privacy-mask processing apparatus includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.
The processor 301 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more ics for controlling the execution of programs in accordance with the present invention.
The communication bus 302 may include a path that conveys information between the aforementioned components.
The Memory 303 may be a Read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these. The memory 303 may be separate and coupled to the processor 301 through a communication bus 302. The memory 303 may also be integrated with the processor 301.
In particular implementations, processor 301 may include one or more CPUs such as CPU0 and CPU1 shown in fig. 3 for one embodiment.
In particular implementations, the privacy-mask processing apparatus may include a plurality of processors, such as processor 301 and processor 305 shown in fig. 3, as one embodiment. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
In a specific implementation, the privacy-mask processing apparatus may further include an output device 306 and an input device 307, as an embodiment. An output device 306 is in communication with the processor 301 and may display information in a variety of ways. For example, the output device 306 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 307 is in communication with the processor 301 and may receive user input in a variety of ways. For example, the input device 307 may be a mouse, a keyboard, a touch screen device, a sensing device, or the like.
The memory 303 is used for storing program codes for executing the scheme of the application, and is controlled by the processor 301 to execute. The processor 301 is operable to execute program code stored in the memory 303. One or more software modules (e.g., detection module, mapping module, processing module, user interface presentation module, etc.) may be included in the program code.
In a specific implementation, the privacy-mask processing apparatus may be a desktop computer, a laptop computer, a web server, a Personal Digital Assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device or a server.
In another embodiment, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the privacy-mask processing method described in fig. 1B above is implemented.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (11)
1. A privacy mask processing method, the method comprising:
determining a target region in which an object to be shielded possibly exists and a confidence coefficient of the target region from an image, wherein the confidence coefficient of the target region is used for indicating the probability that the target region possibly exists the object to be shielded;
substituting the confidence degree of the target area into a stored specified algorithm, and calculating privacy masking processing parameters corresponding to the confidence degree of the target area through the specified algorithm, wherein the specified algorithm is used for indicating the corresponding relation between the confidence degree and the privacy masking processing parameters; determining privacy masking processing parameters corresponding to the confidence degrees of the target area as privacy masking processing parameters of the target area;
based on the privacy masking processing parameter of the target area, carrying out privacy masking processing on the target area;
the calculating of the privacy-masking processing parameter corresponding to the confidence of the target area through the specified algorithm includes:
determining the number of non-0 parameters or non-0 parameter variance according to the confidence degree of the target area, determining a parameter matrix according to the number of the non-0 parameters or the non-0 parameter variance, and determining the parameter matrix as a privacy masking processing parameter corresponding to the confidence degree of the target area, wherein the number of the non-0 parameters or the non-0 parameter variance is used for indicating the intensity of privacy masking processing;
when the privacy-masking processing parameter of the target area is the parameter matrix, the privacy-masking processing on the target area based on the privacy-masking processing parameter of the target area includes:
performing convolution processing on the pixel value of each pixel point in the target area based on the parameter matrix; and replacing the pixel value of each pixel point in the target area with the pixel value after convolution processing so as to finish privacy masking processing of the target area.
2. The method of claim 1, wherein determining from the image a target region where an object to be occluded may exist and a confidence level of the target region comprises:
detecting a candidate frame possibly having the target to be shielded and the confidence degree of the candidate frame from the image based on a target detection algorithm aiming at the target to be shielded;
determining a region corresponding to the candidate frame as the target region;
determining the confidence of the candidate box as the confidence of the target area.
3. The method of claim 1, wherein determining from the image a target region where an object to be occluded may exist and a confidence level of the target region comprises:
determining position information and confidence degrees of a plurality of target pixel points from the image based on a target segmentation algorithm aiming at the target to be shielded, wherein the target pixel points refer to the pixel points belonging to the target to be shielded in the image;
determining the target area based on the position information of the target pixels;
and determining the confidence of the target area based on the confidence of the target pixel points.
4. The method of claim 3, wherein said determining a confidence level for the target region based on the confidence levels for the plurality of target pixels comprises:
determining the average confidence of the target pixel points;
determining the average confidence as the confidence of the target region.
5. The method of claim 3, wherein said determining a confidence level for the target region based on the confidence levels for the plurality of target pixels comprises:
determining the confidence of each pixel point in the target area based on the confidence of the target pixel points;
the determining privacy masking processing parameters for the target area based on the confidence level for the target area comprises:
determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area;
the privacy-mask processing on the target area based on the privacy-mask processing parameter of the target area comprises:
and respectively carrying out privacy masking processing on each pixel point in the target area based on the privacy masking processing parameter of each pixel point in the target area.
6. A privacy mask processing apparatus, the apparatus comprising:
the device comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining a target region in which an object to be shielded possibly exists and a confidence coefficient of the target region from an image, and the confidence coefficient of the target region is used for indicating the probability that the object to be shielded possibly exists in the target region;
the second determination module is used for substituting the confidence degree of the target area into a stored specified algorithm, calculating privacy masking processing parameters corresponding to the confidence degree of the target area through the specified algorithm, and the specified algorithm is used for indicating the corresponding relation between the confidence degree and the privacy masking processing parameters; determining privacy masking processing parameters corresponding to the confidence degrees of the target area as privacy masking processing parameters of the target area;
the processing module is used for carrying out privacy masking processing on the target area based on the privacy masking processing parameter of the target area;
the second determining module is further configured to determine a non-0 parameter number or a non-0 parameter variance according to the confidence of the target area, determine a parameter matrix according to the non-0 parameter number or the non-0 parameter variance, and determine the parameter matrix as a privacy masking processing parameter corresponding to the confidence of the target area, where the non-0 parameter number or the non-0 parameter variance is used to indicate the intensity of privacy masking processing:
when the privacy-masking processing parameter of the target area is a parameter matrix, the processing module is specifically configured to perform convolution processing on the pixel value of each pixel point in the target area based on the parameter matrix; and replacing the pixel value of each pixel point in the target area with the pixel value after convolution processing so as to finish privacy masking processing of the target area.
7. The apparatus of claim 6, wherein the first determining module comprises:
a first determining unit, configured to detect, from the image, a candidate frame in which the object to be occluded may exist and a confidence of the candidate frame based on an object detection algorithm for the object to be occluded;
a second determining unit, configured to determine a region corresponding to the candidate frame as the target region;
a third determining unit, configured to determine the confidence level of the candidate box as the confidence level of the target region.
8. The apparatus of claim 6, wherein the first determining module comprises:
a fourth determining unit, configured to determine, based on a target segmentation algorithm for the target to be masked, position information and confidence levels of a plurality of target pixel points from the image, where a target pixel point is a pixel point in the image that belongs to the target to be masked;
a fifth determining unit configured to determine the target area based on position information of the plurality of target pixel points;
a sixth determining unit, configured to determine, based on the confidences of the multiple target pixel points, the confidence of the target region.
9. The apparatus of claim 8, wherein the sixth determining unit is specifically configured to:
determining the average confidence of the target pixel points;
determining the average confidence as the confidence of the target region.
10. The apparatus of claim 8, wherein the sixth determining unit is specifically configured to:
determining the confidence of each pixel point in the target area based on the confidence of the target pixel points;
the second determining module is specifically configured to:
determining privacy masking processing parameters of each pixel point in the target area based on the confidence of each pixel point in the target area;
the processing module is specifically configured to:
and respectively carrying out privacy masking processing on each pixel point in the target area based on the privacy masking processing parameter of each pixel point in the target area.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is configured to perform the steps of the method of any of claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711273872.5A CN109886864B (en) | 2017-12-06 | 2017-12-06 | Privacy mask processing method and device |
| PCT/CN2018/109450 WO2019109722A1 (en) | 2017-12-06 | 2018-10-09 | Privacy shielding processing method, device, electronic apparatus, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711273872.5A CN109886864B (en) | 2017-12-06 | 2017-12-06 | Privacy mask processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109886864A CN109886864A (en) | 2019-06-14 |
| CN109886864B true CN109886864B (en) | 2021-03-09 |
Family
ID=66750018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711273872.5A Active CN109886864B (en) | 2017-12-06 | 2017-12-06 | Privacy mask processing method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109886864B (en) |
| WO (1) | WO2019109722A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363172A (en) * | 2019-07-22 | 2019-10-22 | 曲靖正则软件开发有限公司 | A kind of method for processing video frequency, device, electronic equipment and readable storage medium storing program for executing |
| CN112672102B (en) * | 2019-10-15 | 2023-03-24 | 杭州海康威视数字技术股份有限公司 | Video generation method and device |
| CN114419720B (en) * | 2022-03-30 | 2022-10-18 | 浙江大华技术股份有限公司 | Image occlusion method and system and computer readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006155109A (en) * | 2004-11-26 | 2006-06-15 | Yec Co Ltd | Image processor |
| CN101540894A (en) * | 2009-04-10 | 2009-09-23 | 浙江工业大学 | Shadowing method of privacy area in video surveillance |
| CN102067175A (en) * | 2008-03-31 | 2011-05-18 | 谷歌公司 | Automatic face detection and identity masking in images, and applications thereof |
| CN102609693A (en) * | 2012-02-14 | 2012-07-25 | 南昌航空大学 | Human face recognition method based on fuzzy two-dimensional kernel principal component analysis |
| CN103907120A (en) * | 2011-12-16 | 2014-07-02 | 英派尔科技开发有限公司 | Automatic privacy management for image sharing networks |
| CN106023250A (en) * | 2016-05-16 | 2016-10-12 | 长春理工大学 | Target occlusion intensity evaluation method in image identification and tracking |
| CN106303233A (en) * | 2016-08-08 | 2017-01-04 | 西安电子科技大学 | A kind of video method for secret protection merged based on expression |
| CN106447634A (en) * | 2016-09-27 | 2017-02-22 | 中国科学院上海微系统与信息技术研究所 | Private part positioning and protection method based on active millimeter wave imaging |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5088161B2 (en) * | 2008-02-15 | 2012-12-05 | ソニー株式会社 | Image processing apparatus, camera apparatus, communication system, image processing method, and program |
| EP2291795A1 (en) * | 2008-07-02 | 2011-03-09 | C-True Ltd. | Face recognition system and method |
| CN102028482B (en) * | 2009-09-30 | 2012-11-14 | 同方威视技术股份有限公司 | Human body detection image processing method and human body detection apparatus |
| CN104966067B (en) * | 2015-06-29 | 2018-05-18 | 福建天晴数码有限公司 | Protect the image processing method and system of privacy |
-
2017
- 2017-12-06 CN CN201711273872.5A patent/CN109886864B/en active Active
-
2018
- 2018-10-09 WO PCT/CN2018/109450 patent/WO2019109722A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006155109A (en) * | 2004-11-26 | 2006-06-15 | Yec Co Ltd | Image processor |
| CN102067175A (en) * | 2008-03-31 | 2011-05-18 | 谷歌公司 | Automatic face detection and identity masking in images, and applications thereof |
| CN101540894A (en) * | 2009-04-10 | 2009-09-23 | 浙江工业大学 | Shadowing method of privacy area in video surveillance |
| CN103907120A (en) * | 2011-12-16 | 2014-07-02 | 英派尔科技开发有限公司 | Automatic privacy management for image sharing networks |
| CN102609693A (en) * | 2012-02-14 | 2012-07-25 | 南昌航空大学 | Human face recognition method based on fuzzy two-dimensional kernel principal component analysis |
| CN106023250A (en) * | 2016-05-16 | 2016-10-12 | 长春理工大学 | Target occlusion intensity evaluation method in image identification and tracking |
| CN106303233A (en) * | 2016-08-08 | 2017-01-04 | 西安电子科技大学 | A kind of video method for secret protection merged based on expression |
| CN106447634A (en) * | 2016-09-27 | 2017-02-22 | 中国科学院上海微系统与信息技术研究所 | Private part positioning and protection method based on active millimeter wave imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109886864A (en) | 2019-06-14 |
| WO2019109722A1 (en) | 2019-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109376667B (en) | Target detection method and device and electronic equipment | |
| WO2021068618A1 (en) | Method and device for image fusion, computing processing device, and storage medium | |
| CN113240630B (en) | Speckle image quality evaluation method and device, terminal equipment and readable storage medium | |
| CN109886864B (en) | Privacy mask processing method and device | |
| CN110909712B (en) | Moving object detection method and device, electronic equipment and storage medium | |
| CN112669344A (en) | Method and device for positioning moving object, electronic equipment and storage medium | |
| CN109286758B (en) | High dynamic range image generation method, mobile terminal and storage medium | |
| CN112672069B (en) | Exposure method and apparatus | |
| CN112308797A (en) | Corner detection method and device, electronic equipment and readable storage medium | |
| CN110390295B (en) | Image information identification method and device and storage medium | |
| CN109873980B (en) | Video monitoring method and device and terminal equipment | |
| CN111553302A (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| CN113657518B (en) | Training method, target image detection method, device, electronic device, and medium | |
| CN113052019A (en) | Target tracking method and device, intelligent equipment and computer storage medium | |
| CN108961293B (en) | Background subtraction method, device, equipment and storage medium | |
| CN116468914A (en) | Page comparison method and device, storage medium and electronic equipment | |
| CN113391779B (en) | Parameter adjusting method, device and equipment for paper-like screen | |
| CN113435393B (en) | Forest fire smoke root node detection method, device and equipment | |
| CN116128922A (en) | Object drop detection method, device, medium and equipment based on event camera | |
| CN115049572A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111754411B (en) | Image noise reduction method, image noise reduction device and terminal equipment | |
| CN111784607A (en) | Image tone mapping method, device, terminal equipment and storage medium | |
| CN111986144A (en) | Image blur judgment method and device, terminal equipment and medium | |
| CN110300253B (en) | Image processing apparatus and method, and storage medium storing instructions | |
| US20240037976A1 (en) | Information processing device, information processing method, and computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |