CN102708560B - A kind of method for secret protection based on mm-wave imaging - Google Patents

Abstract

The invention discloses a kind of method for secret protection based on mm-wave imaging, comprise the following steps: millimeter wave scanning is carried out to tested personnel and obtains original image; Human detection and concealment Articles detecting is carried out according to described original image; Determine the privacy places of human body; The privacy places of human body is shielded and indicates the concealment Item Information on human body.By the method for secret protection based on mm-wave imaging of the present invention, avoid the exposure to human body privacy when detecting concealment article, achieve the available protecting to privacy places of human body.

Description

Privacy protection method based on millimeter wave imaging

Technical Field

The invention relates to privacy protection technology in the field of security check, in particular to a privacy protection method based on millimeter wave imaging.

Background

In the security inspection field, there are several methods for detecting human bodies and their hidden objects: metal detectors, X-ray fluoroscopy, infrared detection, millimeter wave detection, and the like. The metal detector is realized by electromagnetic induction, can only judge whether a metal object exists, and cannot image or determine the position of the object. The X-ray perspective equipment has strong penetrability and is generally used for detecting luggage objects, and if the X-ray perspective equipment directly detects human bodies, the danger to the human bodies is high, so the X-ray perspective equipment is generally rarely used for detecting the human bodies in security check. The infrared detection is to use the thermal radiation characteristic of the object to image, and can be used for detecting the human body in the security check. The brightness of the object in the infrared image mainly depends on the temperature and the radiant heat of the object and the surface radiation characteristic of the object, and the infrared image is characterized by no obvious edge angle and edge information, smooth edge lines, slow gray level change and insensitivity to shape details and tiny posture change of the object. These features make it difficult to detect the human body in the infrared image.

Millimeter waves (3GHz-300GHz) are electromagnetic waves between light waves and radio waves. Millimeter waves can penetrate through all cloth of clothes, the millimeter wave energy radiated by a human body is stronger than that of metal, ceramic, plastic explosive, powdery explosive, clothes, insulating materials and the like, and various forbidden articles such as cutters, guns, explosives and the like hidden on the surface of the human body can be detected by utilizing an active/passive millimeter wave technology. Compared with a metal detection technology, the millimeter wave safety inspection technology is stronger in capability and safer than a ray technology, and the human body millimeter wave safety inspection technology is rapidly developed in the last 10 years. The passive focal plane array scanning technology, the multi-beam frequency scanning technology and the active three-dimensional holographic millimeter wave technology are tested and applied in sequence. After the active millimeter wave security inspection equipment is used for imaging a human body, human body characteristics and various articles carried by the human body can be displayed clearly in the image.

Firstly, in millimeter wave security inspection, analysis of human body images is an important component link. After human body millimeter wave imaging, how to detect and analyze a human body image is the basis for realizing target detection automation of a security inspection system, and the basis for marking the position of a hidden object on a human body and protecting the privacy of the human body image in subsequent processing.

Secondly, how to detect and mark hidden objects on human bodies after millimeter wave imaging is that in the prior art, an artificial analysis method is adopted, wherein an image enhancement technology and a multi-frame comparison technology are applied in the artificial analysis, but the hidden objects can be identified and positioned by interpretation and analysis of professionals. Although image segmentation techniques based on grayscale multi-threshold, boundary extraction, edge detection, region segmentation, wavelet transformation, morphology, fuzzy mathematics, genetic algorithms, neural networks, information entropy, etc. have been tried and applied in automatic detection of concealed objects, it still generates inconsistent conditions with human vision since the segmentation of an image is performed only by grayscale and spatial information in the image, which is disjointed from human vision mechanics. The positioning analysis method based on the human body prior model is applied to the motion tracking of the human body, and the tracking complexity is reduced, wherein the positioning analysis method mainly comprises a strip model shown in fig. 32, a rod model shown in fig. 33 and the like, but the strip model only comprises human body contour information such as structure, shape, posture and the like, and the rod model only comprises all joint points of the human body, and is only limited to the detection of the human body, and the automatic detection and identification problems of hidden objects in the human body cannot be directly solved.

Thirdly, through millimeter wave scanning imaging, the information of hidden objects on a human body can be detected, but the exposure and display of human body privacy (such as faces and privacy parts) can be caused at the same time, how to analyze and process images after millimeter wave imaging, and shielding the private information of the human body before displaying the hidden objects is also a technical problem to be solved in a security inspection system.

Disclosure of Invention

The invention aims to provide a human body detection method and a human body detection device based on millimeter wave imaging, which are used for identifying and positioning each part of a human body in millimeter wave scanning.

The method comprises the following steps: millimeter wave scanning is carried out on the person to be detected to obtain an original image; adjusting the original image to obtain a target image; segmenting and positioning a human body part according to the target image; and generating a human body model.

Further, the segmenting and positioning the human body part according to the target image further comprises the following sub-steps: determining a vertical center line of a human body; determining coordinates of each key point of the human body of the target image and obtaining a horizontal parting line between each part of the human body; determining the width and slope of each part of the human body.

Further, the generating the human body model comprises: and obtaining a human body model consisting of rectangles and/or parallelograms according to the coordinates of the key points and the width and slope of each part of the human body.

Further, the adjusting the original image to obtain the target image further includes the following sub-steps: preprocessing the original image to obtain a primary de-noised image; carrying out binarization on the preliminary de-noised image to obtain a preliminary binary image; and reprocessing the preliminary binary image to obtain the target image.

Further, the preprocessing the original image to obtain a preliminary denoised image further comprises the following substeps: performing difference operation on the gray values of the original image and the background image; carrying out image smoothing processing; linear gray scale transformation.

Further, the step of binarizing the preliminary de-noised image to obtain the preliminary binary image is to select a threshold value for binarization by using a pulse coupled neural network algorithm according to a criterion that the maximum entropy is the maximum.

Further, the reprocessing of the preliminary binary image to obtain the target image is through morphological filtering.

Further, the filtering by morphology comprises: carrying out corrosion operation by using a square structural element with the side length of 5 to eliminate bright noise points outside a human body; using a square structural element with the side length of 4 to carry out opening operation to keep the size of an image and eliminate isolated regions and burrs of human body edges; using square structural elements with the side length of 4 to perform closed operation to keep the size of an image, filling tiny holes in a human body and smoothing the edge of the human body; and performing expansion operation by using a square structural element with the side length of 5 to restore the image to the original size.

Further, the preprocessing the original image to obtain a preliminary denoised image further comprises the following substeps: and performing image enhancement on the original image.

Correspondingly, the human body detection device based on millimeter wave imaging comprises: the scanning device is used for performing millimeter wave scanning on the detected person to obtain an original image; the adjusting module is used for adjusting the original image to obtain a target image; the segmentation positioning module is used for segmenting and positioning the human body part according to the target image; and the human body model generating module is used for generating a human body model.

Further, the segmentation positioning module further comprises the following sub-modules: the vertical central line module is used for determining the vertical central line of the human body; the coordinate horizontal line module is used for determining the coordinates of each key point of the human body of the target image and obtaining a horizontal dividing line between each part of the human body; and the width slope module is used for determining the width and the slope of each part of the human body.

Further, the human body model generating module is further configured to obtain a human body model composed of rectangles and/or parallelograms according to the coordinates of the key points, and the widths and slopes of the parts of the human body.

Further, the adjusting module further comprises the following sub-modules: the preprocessing module is used for preprocessing the original image to obtain a preliminary de-noised image; a binarization module, configured to perform binarization on the preliminary de-noised image to obtain a preliminary binary image; and the reprocessing module is used for reprocessing the preliminary binary image to obtain the target image.

Further, the preprocessing module further comprises the following units:

the difference value operation unit is used for performing difference value operation on the gray values of the original image and the background image; a smoothing unit configured to perform image smoothing; and the linear change unit is used for performing linear gray scale conversion.

Further, the binarization module further selects a binarization threshold value by using a pulse coupling neural network algorithm according to the criterion that the entropy is the maximum.

Further, the reprocessing module further reprocesses through morphological filtering.

Further, the filtering by morphology comprises: carrying out corrosion operation by using a square structural element with the side length of 5 to eliminate bright noise points outside a human body; using a square structural element with the side length of 4 to carry out opening operation to keep the size of an image and eliminate isolated regions and burrs of human body edges; using square structural elements with the side length of 4 to perform closed operation to keep the size of an image, filling tiny holes in a human body and smoothing the edge of the human body; and performing expansion operation by using a square structural element with the side length of 5 to restore the image to the original size.

Further, the preprocessing module further comprises: and the image enhancement unit is used for carrying out image enhancement on the original image.

The millimeter wave imaging-based human body detection method and the millimeter wave imaging-based human body detection device realize the identification and processing of the human body part in the millimeter wave image, and provide a basis for the subsequent inspection of hidden articles and privacy protection.

The invention also aims to provide an automatic detection and identification method and device for the hidden articles, which realize that the distribution positioning and identification of the hidden articles on the human body in millimeter wave scanning are changed from manual operation to automatic operation, and reduce the use requirements of personnel.

The automatic detection and identification method of the concealed article comprises the following steps: millimeter wave scanning is carried out on the person to be detected to obtain an original image; adjusting the original image to obtain a target image; segmenting and positioning a human body part according to the target image; generating a bar combination model; detecting the non-human body target according to the original image to obtain a non-human body target distribution original image; obtaining the position distribution information of the non-human body target distribution original image relative to the human body by utilizing the bar and rod combination model; and carrying out category identification on the non-human body target and displaying the position distribution information of the hidden object relative to the human body.

Further, the generating a rowbar combination model comprises the sub-steps of: generating a rod-shaped model for providing each key point of the human body; generating a strip model providing human body contour information; combining the rod model and the ribbon model to generate a ribbon combination model.

Further, the detecting the non-human body target according to the original image to obtain the non-human body target distribution original image comprises the following substeps: carrying out edge detection on the original image, and preliminarily identifying a non-human body target; highlighting the non-human body target distribution area through mathematical morphology operation; selecting a minimum circumscribed rectangle according to the boundary of the non-human body target distribution area to obtain a non-human body target regular area distribution map; and fusing the non-human body target regular region distribution map and the original image to obtain the non-human body target original image.

Further, the obtaining of the position distribution information of the non-human body target distribution original image relative to the human body by using the bar combination model is performed by inputting the non-human body target distribution original image to the bar combination model.

Further, the performing the category identification on the non-human body target and displaying the position distribution information of the concealed article relative to the human body comprises the following substeps: positioning the exposed part of the human body; determining non-human body targets distributed on the exposed parts of the human body as non-hidden objects, and determining non-human body targets distributed outside the exposed parts of the human body as hidden objects; and eliminating the original image of the non-concealed object and displaying the distribution information of the original image of the concealed object on the bar combination model.

Accordingly, the automatic detection and identification device of concealed objects of the present invention comprises: the scanning device is used for performing millimeter wave scanning on the detected person to obtain an original image; the adjusting module is used for adjusting the original image to obtain a target image; the segmentation positioning module is used for segmenting and positioning the human body part according to the target image; the bar combination model generation module is used for generating a bar combination model; the non-human body target preliminary detection module is used for detecting the non-human body target according to the original image to obtain a non-human body target distribution original image; the non-human body target distribution module is used for obtaining the position distribution information of the non-human body target distribution original image relative to the human body by utilizing the bar combination model; and the category identification module is used for carrying out category identification on the non-human body target and displaying the position distribution information of the hidden object relative to the human body.

Further, the bar binding model generation module comprises the following sub-modules: the bar model generating module is used for generating bar models for providing key points of the human body; the strip model generating module is used for generating a strip model for providing human body contour information; a combining module for combining the rod model and the ribbon model.

Further, the non-human body target preliminary detection module comprises the following sub-modules: the edge detection module is used for carrying out edge detection on the original image and preliminarily identifying a non-human body target; the highlight display module is used for highlighting the non-human body target distribution area through mathematical morphology operation; the regularization module is used for selecting a minimum circumscribed rectangle according to the boundary of the non-human body target distribution area to obtain a non-human body target regular area distribution map; and the fusion module is used for fusing the non-human body target regular region distribution map and the original image to obtain the non-human body target original image.

Further, the non-human target distribution module is used for inputting the non-human target distribution original image to the bar combination model.

Further, the category identification module comprises the following sub-modules: the exposed part positioning module is used for positioning the exposed part of the human body; the classification module is used for determining the non-human body targets distributed on the exposed parts of the human body as non-hidden objects and determining the non-human body targets distributed outside the exposed parts of the human body as hidden objects; and the display module is used for eliminating the original image of the non-concealed object and displaying the distribution information of the original image of the concealed object on the bar combination model.

The method and the device for automatically detecting and identifying the concealed articles realize the change of the detection and the identification of the concealed articles from manual operation to automatic operation, reduce the use requirements of personnel, reduce the human error and shorten the detection and interpretation time.

The invention also aims to provide a privacy protection method and a privacy protection device based on millimeter wave imaging, so that privacy protection of the detected personnel in millimeter wave scanning is realized.

The privacy protection method based on millimeter wave imaging comprises the following steps: millimeter wave scanning is carried out on the person to be detected to obtain an original image; detecting human bodies and hidden objects according to the original images; determining a private part of a human body; the private parts of the human body are shielded and the information of the hidden objects on the human body is marked.

Further, the determining the private parts of the human body comprises: the sex of the person to be examined is judged, and when the person to be examined is male, the region of the head region and the human waist down trunk width 1/2 is determined as the privacy region, and when the person to be examined is female, the region of the human head region, the human waist down trunk width 1/2, and the region of the human trunk down from the upper end of the trunk to the trunk height 1/2 are determined as the privacy region.

Further, the step of masking the private parts of the human body and indicating the information of the hidden objects on the human body includes: fuzzifying the private part on the original image to form a partially fuzzified original image; the concealed item is delineated with a logo box over the partially obscured original image.

Further, the step of masking the private parts of the human body and indicating the information of the hidden objects on the human body includes: selecting a target image in the human body detection; judging whether the hidden article is in the private part of the human body, if so, using a color block with different color from the human body to represent the hidden article and marking the hidden article on the target image; and if not, displaying the original image of the concealed article on the destination image.

Further, the step of masking the private parts of the human body and indicating the information of the hidden objects on the human body includes: carrying out full blurring treatment on the original image to form a full blurring original image; judging whether the hidden object is positioned at the private part of the human body, if so, using a color block with the color different from that of the human body to represent the hidden object and marking the hidden object on the completely blurred original image; if not, the original image of the concealed item is displayed over the fully obscured original image.

Further, the step of masking the private parts of the human body and indicating the information of the hidden objects on the human body includes: selecting a human body model in the human body detection; concealed items are represented and marked on the mannequin with color blocks that are different from the mannequin color.

Correspondingly, the privacy protection device based on millimeter wave imaging comprises: the scanning device is used for performing millimeter wave scanning on the detected person to obtain an original image; the detection device is used for detecting human bodies and hidden objects according to the original images; the privacy part determining module is used for determining the privacy part of the human body; and the privacy shielding module is used for shielding the privacy part of the human body and marking the information of the hidden articles on the human body.

Further, the privacy location determination module is further to: the sex of the person to be examined is judged, and when the person to be examined is male, the region of the head region and the human waist down trunk width 1/2 is determined as the privacy region, and when the person to be examined is female, the region of the human head region, the human waist down trunk width 1/2, and the region of the human trunk down from the upper end of the trunk to the trunk height 1/2 are determined as the privacy region.

Further, the privacy-mask module further comprises the following sub-modules: the partial blurring module is used for performing blurring processing on the private part on the original image to form a partial blurring original image; a first marking module for marking out the concealed item with a marking frame on the partially obscured original image.

Further, the privacy-mask module further comprises: a target image selecting module for selecting a target image in the human body detection; the second marking module is used for judging whether the hidden article is positioned at the private part of the human body or not, if so, the color block with the color different from that of the human body is used for representing the hidden article and marking the hidden article on the target image; and if not, displaying the original image of the concealed article on the destination image.

Further, the privacy-mask module further comprises: the whole blurring module is used for performing whole blurring processing on the original image to form a whole blurred original image; the third marking module is used for judging whether the hidden object is positioned at the private part of the human body or not, if so, the hidden object is represented by using a color block with the color different from that of the human body and is marked on the whole fuzzified original image; if not, the original image of the concealed item is displayed over the fully obscured original image.

Further, the privacy-mask module further comprises: a selection model module for selecting a human body model in the human body detection; and the fourth marking module is used for representing the hidden objects by using color blocks with different colors from the human body model and marking the hidden objects on the human body model.

According to the privacy protection method and device based on millimeter wave imaging, exposure to human privacy when hidden objects are detected is avoided, and effective protection of human privacy parts is achieved.

Drawings

The present invention will be described in detail with reference to the following examples with reference to the accompanying drawings.

FIG. 1 is a basic flow chart of a human body detection method based on millimeter wave imaging;

FIG. 2 is a basic structure diagram of a human body detection device based on millimeter wave imaging;

FIG. 3 is an original image;

FIG. 4 is a preliminary denoised image;

FIG. 5 is a preliminary binary image;

fig. 6 is a flowchart of step S2 in the human body detection method based on millimeter wave imaging;

FIG. 7 is a schematic structural diagram of an adjustment module in the human body detection device based on millimeter wave imaging;

FIG. 8 is a destination image;

FIG. 9 is a human body skeleton diagram;

FIG. 10 is a diagram of human body segmentation effect;

fig. 11 is a flowchart of step S3 in the human body detecting method based on millimeter wave imaging;

FIG. 12 is a schematic structural diagram of a segmentation positioning module in the human body detection device based on millimeter wave imaging;

FIG. 13 is a diagram of a human body model obtained in the method and apparatus for detecting a human body based on millimeter wave imaging;

FIG. 14 is a corresponding effect diagram of the human body model diagram and the original image;

fig. 15 is a basic flow diagram of a method of automatic detection and identification of concealed items;

fig. 16 is a flowchart of step S5 of the method for automatic detection and identification of concealed items;

FIG. 17 is a drawing of a bar attachment model;

fig. 18 is a flowchart of step S6 of the method for automatic detection and identification of concealed items;

FIG. 19 is an image of a preliminary non-human target identification;

FIG. 20 is a diagram of a non-human target distribution area;

FIG. 21 is a non-human target regular region distribution map;

FIG. 22 is a non-human target raw image;

FIG. 23 is a distribution diagram of non-human target original images on a bar-bar combination model;

FIG. 24 is a distribution plot of an original image of a concealed item on a bar-and-stick model;

fig. 25 is a flowchart of step S8 of the method for automatic detection and identification of concealed items;

FIG. 26 is a basic flow diagram of a millimeter wave imaging based privacy preserving method;

fig. 27 is a schematic structural diagram of a privacy protecting apparatus based on millimeter wave imaging;

FIG. 28 is a diagram of the effectiveness of the first embodiment of privacy masking;

FIG. 29 is a diagram of the effectiveness of a second embodiment of privacy masking;

FIG. 30 is a diagram of the effectiveness of a third embodiment of privacy masking;

FIG. 31 is a diagram of the effectiveness of a fourth embodiment of privacy masking;

FIG. 32 is a diagram of a prior art stripe model;

fig. 33 is a schematic diagram of a rod model in the background art.

Detailed Description

The technical solution of the present invention is described in detail below with the help of embodiments of the present invention with reference to the accompanying drawings.

The invention relates to a human body detection method based on millimeter wave imaging, which comprises the following steps: s1, millimeter wave scanning is carried out on the person to be detected to obtain an original image; s2, adjusting the original image to obtain a target image; s3, segmenting and positioning the human body part according to the target image; and S4, generating a human body model. As shown in fig. 1.

Correspondingly, as shown in fig. 2, the present invention further provides a human body detection device based on millimeter wave imaging, comprising:

the scanning device 1 is used for executing the step S1, and performing millimeter wave scanning on the person to be detected to obtain an original image;

an adjusting module 2, configured to execute step S2, and adjust the original image to obtain a target image;

a segmentation and positioning module 3, configured to execute step S3, segment and position the human body part according to the target image;

and a human body model generating module 4 for executing step S4 to generate a human body model.

In step S1, the person to be detected is requested to enter the millimeter wave scanning detection area, and scanning detection is performed by the scanning device 1 in a millimeter wave active/passive manner, so as to obtain an original image as shown in fig. 3. The scanned original image generally has the following characteristics: the image is not clear enough overall and contains a lot of noise.

Therefore, the adjusting module 2 is required to perform step S2 to adjust the original image to obtain the target image suitable for image operation and segmentation, as shown in fig. 6, step S2 includes the following sub-steps: s21, preprocessing the original image to obtain a primary de-noised image; s22, carrying out binarization on the preliminary de-noised image to obtain a preliminary binary image; and S23, reprocessing the preliminary binary image to obtain the target image.

Accordingly, as shown in fig. 7, the adjusting module 2 further includes the following sub-modules:

a preprocessing module 21, configured to execute step S21, perform preprocessing on the original image to obtain a preliminary denoised image;

a binarization module 22, configured to perform step S22, to binarize the preliminary denoised image to obtain a preliminary binary image;

and a reprocessing module 23, configured to execute step S23, and reprocess the preliminary binary image to obtain the target image.

Further, the preprocessing module 21 further includes an image enhancement unit, a difference operation unit, a smoothing unit, and a linear variation unit. The execution of step S21 by the preprocessing module 21 requires the following substeps:

the image enhancement unit is used for enhancing the original image to increase the contrast between a human body area and a background area in the original image and improve the visual effect of the image.

The difference operation unit is used for carrying out difference operation or subtracting the gray values of the original image and the empty background image through the gray values of the original image and the empty background image, so that system noise is eliminated. The empty background image is an image formed by scanning when no person to be detected exists in the millimeter wave scanning detection area.

The smoothing unit is used for smoothing the image to remove random noise in the image $\frac{1}{10}\left[\begin{array}{ccc}1& 1& 1\\ 1& 2& 1\\ 1& 1& 1\end{array}\right]$ And the operator performs low-pass filtering on the image to realize smoothing operation.

The linear change unit is used for performing gray scale stretching or partition linear transformation on the image, compressing the gray scale range of the background region which is not interested in the image, and expanding the gray scale range of the human body region, so that the human body part is highlighted, the human body part is integrally clear, and finally the primary de-noised image is obtained, as shown in fig. 4.

Further, in the step S22 executed by the binarization module 22, the binarizing the preliminary de-noised image to obtain the preliminary binary image is to select a binarized threshold value according to the maximum entropy criterion by using a Pulse Coupled Neural Network (PCNN) algorithm, and convert the gray scale image of the preliminary de-noised image into the binarized image by using the threshold value, so as to implement the segmentation of the human body region and the background region in the image.

The key to the problem of how to accurately segment the human body from the background completely by selecting a threshold is to use a relatively mature Pulse Coupled Neural Network (PCNN) technology, which is a pulse coupled neural network-based model proposed by Eckhorn and other mammals in the 90 s based on the visual characteristic research of cats and the like, and the model is used in the iterative process of threshold selection of images, and the iterative formula is as follows:

<math> <mrow> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>F</mi> </msub> </mrow> </msup> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>F</mi> </msub> <munder> <mo>&Sigma;</mo> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </munder> <msub> <mi>M</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>I</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> </msub> </mrow> </math>

<math> <mrow> <msubsup> <mi>L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>L</mi> </msub> </mrow> </msup> <msubsup> <mi>L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>L</mi> </msub> <munder> <mo>&Sigma;</mo> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </munder> <msub> <mi>W</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> </mrow> </math>

<math> <mrow> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mrow> <mo>(</mo> <mn>1</mn> <mo>+</mo> <msubsup> <mi>&beta;L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>T</mi> </msub> </mrow> </msup> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>T</mi> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> </mrow> </math>

<math> <mrow> <msubsup> <mi>Y</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <mfenced open = '{' close = ''> <mtable> <mtr> <mtd> <mn>1</mn> <mo>(</mo> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>&gt;</mo> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> <mo>(</mo> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>&le;</mo> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mtd> </mtr> </mtable> </mfenced> </mrow> </math>

f is the n-times feedback input F of the ith and j neurons_i,j ⁽ⁿ⁾，I_i,jFor inputting a stimulus signal, the stimulus signal is the gray value of the ith and jth pixels in a matrix formed by image pixels, beta is a connection coefficient, and L_i,j ⁽ⁿ⁾Is a connecting item, T_i,j ⁽ⁿ⁾Is a dynamic threshold, i.e. the threshold to be solved, Y_i,j ⁽ⁿ⁾Is the PCNN pulse output value, U_i,j ⁽ⁿ⁾Is an internal activity item, where M of the internal connection matrix M, W_i,j,k,l、W_i,j,k,lAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾Middle Y_i,j ⁽ⁿ⁾Weighting coefficient of W＝M，α_F、α_L、α_TAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾、T_i,j ⁽ⁿ⁾Decay time constant of V_F、V_L、V_TAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾、T_i,j ⁽ⁿ⁾The intrinsic potential of (c).

Entropy is a representation of the statistical properties of an image, and reflects the amount of information contained in the image. For an image, the larger the entropy of the image after segmentation, the larger the amount of information obtained from the original image after segmentation, the more detailed the segmented image, and therefore the better the overall segmentation effect. The patent uses the maximum entropy criterion as the criterion for the PCNN iteration to end. The formula for calculating the entropy is:

H₁(P)＝-P₁×log₂P₁-P₀×log₂P₀。

wherein, P₁、P₀Respectively representing pulse output values Y [ n ]]Probability of 1, 0. The invention sets a large iteration number n, such as n being 100, uses PCNN algorithm to carry out iterative operation, and obtains corresponding entropy H after each operation₁(P), and comparing the entropy values obtained by n operations to obtain the value H with the maximum entropy_1maxNumber of iterations N in (P)_max. Number of iterations N_maxThen the threshold value T [ N ] is obtained_max]Y [ N ] output by PCNN at this time_max]And a binary image with the best overall segmentation effect is formed under the condition that other parameters of the PCNN are fixed. Wherein Y [ N ]_max]The portion of 1 is the background portion, YN_max]The portion of 0 is a human body portion.

The value ranges of the parameters in the PCNN formula adapted to the above calculation process are:

αF

αL

αT

VF

VL

VT

β

0.1～0.6

1～10

0.1～0.6

0.1～0.5

0.1～0.5

2～10

0.1～0.6

w, M two operators can use 1/r or 1/r²R represents the length of the matrix side of the operator.

Preferably, the invention can take the following parameter values: alpha is alpha_F＝0.2，α_L＝2，α_T＝0.1，V_F＝0.1，V_L＝0.5，V_T＝20，β＝0.5， $W=M=\left[\begin{array}{ccccc}1/8& 1/5& 1/4& 1/5& 1/8\\ 1/5& 1/2& 1& 1/2& 1/5\\ 1/4& 1& 1& 1& 1/4\\ 1/5& 1/2& 1& 1/2& 1/5\\ 1/8& 1/5& 1/4& 1/5& 1/8\end{array}\right],$ The calculation is performed and the obtained preliminary binary image with the best effect is shown in fig. 5.

Further, in step S23 executed by the re-processing module 23, the pre-processing of the preliminary binary image to obtain the target image is performed by morphological filtering. Image noise is caused by threshold segmentation, and the noise is mainly isolated bright noise points outside the human body or isolated dark noise points inside the human body. In order to remove the noise, the method used by the invention is to apply a mathematical morphology operation method to the preliminary binary image for filtering and transformation to obtain a binary image with clear and smooth contour, namely the target image, as shown in fig. 8, thereby being beneficial to subsequent processing.

The mathematical morphology operation method mainly comprises corrosion operation, expansion operation, opening operation and closing operation.

The erosion operation can weaken or even eliminate the bright areas smaller than the structural elements, and thus can be used to effectively remove the uneven protruding portions on the boundaries of isolated noise points.

The dilation operation is a process of incorporating all background points in contact with the target object into the object, filling up holes and forming connected domains and non-smooth concave portions on flat image boundaries.

The starting operation is to perform corrosion operation on the image and then perform expansion operation, so that isolated regions and burrs in the image can be removed, the noise points with shapes smaller than those of the structural elements can be eliminated, the target noise can be eliminated by selecting the appropriate structural elements according to the characteristics of the target noise, and the background is reserved.

The closed operation is to perform expansion operation on the image and then perform corrosion operation, and can fill tiny holes in the object and connect the adjacent object with the boundary of the smooth object.

The structural elements are basic operators of mathematical morphology operation, and the structural elements selected mainly consist of the shapes and the sizes of the structural elements.

Preferably, the morphological operation used in the method may be the following process: (1) carrying out corrosion operation on the image by using a square structural element with the side length of 5 to eliminate bright noise points outside a human body in the image; (2) performing open operation on the image by using a square structural element with the side length of 4, and eliminating isolated regions and burrs of human body edges while keeping the size of the image; (3) performing closed operation on the image by using a square structural element with the side length of 4, filling fine holes in the human body while keeping the size of the image, and smoothing the boundary of the human body; (4) and performing a dilation operation on the image by using a square structural element with the side length of 5 to restore the image to the original size. Through the process, the noise with the length and the width both less than 5 can be removed, the hollow hole with the length and the width both less than 5 on the human body is filled, and the target image formed after the processing comprises an approximately complete human body part, so that the human body characteristics are more obvious.

In addition, if the image has a large white interference area which is not eliminated, the area of each connected area in the image can be calculated to eliminate the area with a small area.

Further, as shown in fig. 11, the step S3 of segmenting and positioning the human body part according to the target image further includes the following sub-steps: s31, determining the vertical central line of the human body; s32, determining the coordinates of each key point of the human body of the target image and obtaining a horizontal dividing line between each part of the human body; and S33, determining the width and the slope of each part of the human body.

Accordingly, as shown in fig. 12, the segmentation positioning module 3 further includes the following sub-modules:

a vertical center line module 31 for performing step 31 and determining a vertical center line of the human body;

a coordinate horizontal line module 32, configured to execute step 32, determine coordinates of each key point of the human body in the target image, and obtain a horizontal dividing line between each part of the human body;

and a width slope module 33, configured to perform step 33 and determine the width and slope of each part of the human body.

When the vertical center line module 31 determines the position of the vertical center line of the person in step S31, since the human body region in the target image has left-right symmetry, the total pixel sum of the image of the human body region is calculated, for example, by S₀Indicating that the sum of the pixels of the partial image of the body is then calculated from left to right in columns of the image starting from the left edge of the body area, e.g. with S₁Indicates when S is₁Is S₀1/2, the current column is the vertical centerline of the human body.

In step S32, the coordinate level module 32 executes a step S32, where the coordinates of each key point of the human body are the position coordinates of each part of the human body, such as the edge point coordinates, the center point coordinates, and the like, and the parts of the human body include: top of the head, sole of the foot, neck, upper torso, lower torso (waist), crotch, knee, fingertip, and elbow. The coordinates of each key point of the human body and the horizontal parting line between each part are the process of mutual deduction, and the process is as follows:

and (3) downwards from the top of the image along the vertical center line of the human body, judging that if 1/10 same gray values which are continuous and have the length not less than the height of the image continuously downwards from the boundary point of the first human body region are continuously downwards from the boundary point, determining that the boundary point is the head top center point of the human body, and determining that the horizontal line of the boundary point is the head top horizontal line H2 of the human body.

And the position of the human foot is a fixed position in the image, so that the sole coordinates and the horizontal line H9 can be determined. Since the position where the person stands is fixed at the time of imaging, the height H of the person can be obtained by subtracting the ordinate of the sole from the ordinate of the head.

The human head occupies about 15% of the height according to human anatomy, and the coordinates of the upper end of the neck and the horizontal line H4 can be determined. The height of the neck is about 45% of the height of the head, so that the coordinates of the end points of the edges of the upper end of the torso and the horizontal line H5 can be determined.

Thinning the represented target image of fig. 8 may also result in a human skeleton map, as shown in fig. 9.

In the human body skeleton diagram of FIG. 9, the intersection of the trunk and the legs is the position of the lower end of the trunk, which is located on the horizontal line as the waist line H6, and the distance along this line from the horizontal line of the feet is defined as the leg length H_leg。

According to the judgment of human anatomy, the ratio of the shank to the thigh of the human body is about 1:1.2, so that the horizontal position of the knee can be determined to be from the foot to the upper H_legX 5/11, thus obtaining knee level H8.

On the object image shown in fig. 8, upward from the bottom of the image along the vertical center line, the first intersection position with the human body image is taken as the crotch, and a human body crotch horizontal line H7 is obtained.

The image of the human body area is divided into a left half and a right half by taking the vertical central line as a dividing line, the highest point of the left half is the position of the left fingertip, and the highest point of the human body on the right image is the position of the right fingertip, so that a fingertip horizontal line H0 is obtained. In the invention, the height difference between two finger tips is ignored.

Since the arms of the person to be examined are required to be opened outward before the person to be examined is scanned, the width between the elbows of the target image is the widest position of the human body, and therefore the leftmost position of the human body region is the left elbow of the body, and the rightmost position is the right elbow of the body, so that the elbow horizontal line H3 is obtained. The height difference between the two elbows is ignored in the present invention.

According to the human anatomy theory, the length ratio of the human hand to the upper arm is 7:9, so that the position of the wrist can be determined according to the positions of the hand tip and the elbow, and the horizontal line H1 of the wrist is obtained.

The human body segmentation effect map is shown in fig. 10.

In step S33 executed by the width slope module 33, the width of each part of the human body is determined based on the intersection of each part of the human body with each horizontal dividing line in the target image of fig. 8, and the slope of each part is calculated using the coordinates of the key points (e.g., the two central points at the upper and lower ends of each part) of each part of the human body.

Further, the step S4 executed by the human body model generating module 4 includes representing each part by a rectangle or a parallelogram according to the coordinates, the width and the slope of the key points of each part, respectively, and connecting all the parts together to obtain a human body model composed of rectangles and/or parallelograms, as shown in fig. 13. The human body can be proportionally corresponded to the human body region in the original image, and the effect is shown in fig. 14.

It should be noted that the human body model generated in the description of the method is illustrated by taking a strip model as an example. The rod-shaped model can be generated after the human body part is segmented and positioned according to the target image. Both the band model and the bar model are applied to the automatic detection and recognition method of a concealed article described below.

On the other hand, the invention also provides an automatic detection and identification method of the hidden object and an automatic detection and identification device of the hidden object, wherein the method comprises the steps of extracting and positioning human body characteristics and identifying non-human body targets. The process of extracting and positioning human body features is mainly based on the human body detection method and device for millimeter wave imaging, but is not limited to the human body detection method and device.

Thus, as shown in fig. 15, a method for automatic detection and identification of concealed items comprises the steps of: s1, millimeter wave scanning is carried out on the person to be detected to obtain an original image; s2, adjusting the original image to obtain a target image; s3, segmenting and positioning the human body part according to the target image; s5, generating a bar combination model; s6, detecting the non-human body target according to the original image to obtain a non-human body target distribution original image; s7, obtaining the position distribution information of the non-human body target distribution original image relative to the human body by utilizing the bar combination model; and S8, identifying the type of the non-human body target and displaying the position distribution information of the hidden article relative to the human body.

Accordingly, an apparatus for automatic detection and identification of concealed objects, comprising:

the scanning device 1 is used for executing the step S1, and performing millimeter wave scanning on the person to be detected to obtain an original image;

an adjusting module 2, configured to execute step S2, and adjust the original image to obtain a target image;

a segmentation and positioning module 3, configured to execute step S3, segment and position the human body part according to the target image;

a bar combination model generation module for executing step S5 to generate a bar combination model;

a non-human body target preliminary detection module, configured to execute step S6, detect a non-human body target according to the original image, and obtain a non-human body target distribution original image;

the non-human body target distribution module is used for executing the step S7, and obtaining the position distribution information of the non-human body target distribution original image relative to the human body by utilizing the bar combination model;

and the type identification module is used for executing the step S8 to identify the type of the non-human body target and display the position distribution information of the hidden article relative to the human body.

In the method for detecting a human body based on millimeter wave imaging, the scanning device 1, the adjusting module 2, and the dividing and positioning module 3 are already described in a human body detecting device based on millimeter wave imaging, and the steps S1, S2, and S3 are not described herein again.

According to fig. 16, step S5 further includes the following sub-steps: s51, generating a rod-shaped model for providing each key point of the human body; s52, generating a strip model providing human body contour information; and S53, combining the rod model and the strip model to generate a rod combination model.

Accordingly, the bar-binding model generation module performing step S5 includes the following sub-modules:

a bar model generating module for executing step S51 to generate a bar model providing each key point of the human body;

a strip model generating module for executing step S52 to generate a strip model providing the human body contour information;

a combining module, configured to perform step S53, and combine the rod model and the strip model.

The step S51 executed by the bar model generation module is a step S32 of the human body detection method based on millimeter wave imaging, which determines coordinates of each key point of the human body in the target image and obtains a horizontal dividing line between each part of the human body, and the bar model can be generated by obtaining each joint point for constructing the bar model by using each key point of the human body and connecting each joint point by a straight line.

The process of generating the strip model in step S52 performed by the strip model generation module is the same as the process of generating the human body model in S4 in a human body detection method based on millimeter wave imaging.

Step S53 executed by the combining module combines the bar model and the strip model to generate a bar combining model as shown in fig. 17, thereby completing the extraction and positioning of the human body features in the method, the circular nodes in fig. 17 represent joint points during human body feature extraction, and numbers can be added to the joint points to represent the extraction order of the joint points.

Further, the non-human target preliminary detection module of step S6 further includes: the device comprises an edge detection module, a highlight display module, a regularization module and a fusion module. Correspondingly, as shown in fig. 18, the step S6 of detecting the non-human body target according to the original image and obtaining the non-human body target distribution original image further includes the following sub-steps:

s61, performing edge detection on the original image by an edge detection module, for example, performing edge detection by using Sobel (Sobel) operator, and preliminarily identifying a non-human target, as shown in fig. 19;

s62, the highlighting module highlights the non-human body target distribution area through mathematical morphology operation, for example, the erosion operation is firstly carried out on the graph 19, and then the expansion operation is carried out. Wherein, the erosion and expansion operations respectively use square structural elements with the side length of 2 and 4, thereby highlighting a non-human body target distribution area map, as shown in fig. 20;

s63, selecting a minimum circumscribed rectangle by the regularization module according to the boundary of the non-human body target distribution area, so that the irregular area in the figure 20 is converted into the non-human body target regular area distribution map in the figure 21;

s64, fusing the non-human body target regular region distribution map and the original image map 3 by the module fusion map 21 to obtain the non-human body target original image of FIG. 22, wherein the image also shows the human body part of the position where the non-human body target is located.

Because the bar-shaped model only contains the human body outline information and no specific joint point information, the bar-shaped model only contains the joint point information, the position of the non-human body target on the human body outline is covered, further, therefore, the step S7 executed by the non-human object distribution module is to input the original non-human object distribution image to the bar-combining model, therefore, the human body contour information given by the strip model in the bar combination model can be used for obtaining the distribution of the non-human body target in the human body contour edge, the human body joint point position information given by the bar model in the bar combination model is further used for obtaining the relative position relation between the non-human body target and the joint point, therefore, the bar-combined model is used to make the non-human body target have more accurate positioning on the human body as the reference object, as shown in fig. 23, the distribution diagram of the non-human body target original image on the bar-combined model.

Further, the category identification module comprises the following sub-modules: the device comprises an exposed part positioning module, a classification module and a display module. The respective substeps for performing the risk category recognition of the non-human body target and displaying the position distribution information of the concealed article with respect to the human body in step S8 are respectively as shown in fig. 25:

s81, the exposed part positioning module positions the exposed part of the human body, such as the head, the wrist, the palm and the like, and can position the part with more pertinence in the segmentation and positioning of the human body part;

s82, the classification module classifies the non-human body targets, and determines the non-human body targets distributed on the exposed parts of the human body as non-hidden objects such as glasses, buttons, watches, rings and the like; due to the fact that clothes cover the objects, the non-human body objects outside the exposed parts of the human body cannot be directly checked by security personnel, and the non-human body objects distributed outside the exposed parts of the human body are determined to be hidden objects, so that important attention needs to be paid;

s83, the display module eliminates the original image of the non-concealed object and displays the distribution information of the original image of the concealed object on the bar combination model, as shown in fig. 24, which is an effect diagram of the wristwatch in fig. 23 with the original image of the wrist portion eliminated.

The automatic detection and identification method and device for the hidden articles can reduce the use requirements of personnel, reduce human errors and shorten the interpretation time of the inspection of the hidden dangerous articles of the human body.

On the other hand, after the millimeter wave imaging of the human body, because the original image is clearer, the human body and/or the hidden objects on the human body can be identified and displayed by the human body detection method based on the millimeter wave imaging and the automatic detection and identification method of the hidden objects, but the private parts of the human body are exposed at the same time.

In order to protect privacy, the present invention further provides a privacy protection method based on millimeter wave imaging, as shown in fig. 26, including: s1, millimeter wave scanning is carried out on the person to be detected to obtain an original image; A. detecting human bodies and hidden objects according to the original images; B. determining a private part of a human body; C. the private parts of the human body are shielded and the information of the hidden objects on the human body is marked.

Accordingly, as shown in fig. 27, the present invention further provides a privacy protecting apparatus based on millimeter wave imaging, including:

the scanning device 1 is used for performing millimeter wave scanning on a person to be detected to obtain an original image;

the detection device is used for executing the step A and carrying out human body detection and concealed article detection according to the original image;

a privacy part determining module for executing the step B and determining the privacy part of the human body;

and the privacy shielding module is used for executing the step C, shielding the privacy part of the human body and marking the information of the hidden articles on the human body.

The step a may be performed by the above-mentioned human body detection method based on millimeter wave imaging and automatic detection and identification method of concealed objects, and accordingly, the detection device may include a human body detection device based on millimeter wave imaging and an automatic detection and identification device of concealed objects.

And B, determining the privacy part of the human body by the privacy part determining module executing the step B, and determining and positioning the privacy part according to the division and positioning of the human body part and the human anatomy, wherein the privacy part comprises judging the sex of the detected person, when the detected person is male, the privacy part is determined by the region with the width 1/2 from the head area of the human body to the center of the waist of the human body downwards, and when the detected person is female, the privacy part is determined by the region with the width 1/2 from the head area of the human body to the center of the waist of the human body downwards and the region with the height 1/2 from the upper end of the trunk of the human body downwards. The present invention will be described taking millimeter wave imaging of a male subject person as an example.

The privacy shielding module for shielding the privacy part of the human body and marking the information of the hidden articles on the human body can adopt the following specific implementation modes to protect the privacy:

(1) the privacy shielding module comprises a partial fuzzification module and a first marking module, wherein the partial fuzzification module fuzzifies privacy parts on the original image to form a partial fuzzified original image, and the fuzzification can be realized by using morphological operation and using a mosaic area with a certain side length or directly using a rectangular block with a single color to cover the privacy parts; the first labeling module marks the concealed item with a labeling box, such as a highlighted border, over the partially obscured original image, as shown in fig. 28. This approach is suitable for the case of a combination of automatic and manual detection of concealed objects. Or,

(2) the privacy shielding module comprises a target image selecting module and a second marking module, and since a clear and complete binary image can cover up pixel information of a privacy part of a detected person, the target image is selected by the target image selecting module to represent a human body part, and the target image is obtained by the human body detection method based on millimeter wave imaging, as shown in fig. 8; before the information of the hidden article is displayed on the target image, the second marking module firstly judges whether the hidden article is positioned at the private part of the human body, if so, a color block with the color different from that of the human body is used for representing the hidden article and marking the hidden article on the target image, such as a gray rectangular block with a certain gray value, such as 128; and if not, directly displaying the original image of the concealed article on the destination image. Thus, the complete information of the human body in the original image is completely shielded, and the purpose of privacy protection is achieved, and the effect is shown in fig. 29. The method is only suitable for the automatic detection mode of the hidden objects, and does not need to display the complete human body image. Or,

(3) the privacy-mask module comprises a total fuzzification module and a third marking module. The total blurring module performs total blurring processing on the original image to form a total blurred original image, the blurring method can be the same as that of the specific embodiment (1), and the blurring method uses an operator $\left[\begin{array}{ccccc}0.2& 0.4& 0.6& 0.4& 0.2\\ 0.4& 0.6& 0.8& 0.6& 0.4\\ 0.6& 0.8& 1& 0.8& 0.6\\ 0.4& 0.6& 0.8& 0.6& 0.4\\ 0.2& 0.4& 0.6& 0.4& 0.2\end{array}\right];$ Before the information of the hidden objects is displayed on the all fuzzified original images, a third marking module judges whether the hidden objects are positioned at the private parts of the human body, if so, color blocks with different colors from the human body are used for representing the hidden objects and marking the hidden objects on the all fuzzified original images, and if so, a certain gray value, such as a 128 gray rectangular block, is used for marking; if not, the original image of the concealed item is displayed over the fully obscured original image, with the effect image as shown in fig. 30. The method is only suitable for the automatic detection mode of the hidden objects, and does not need to display the complete human body image. Or,

(4) the privacy-mask module includes a selection model module and a fourth labeling module. The selection model module selects a human body model in the human body detection, wherein the human body model can be a strip model or a strip-bar combination model; the fourth marking module represents and marks concealed objects on the mannequin with color blocks of a different color than the mannequin, such as red. This also completely shields the private information of the human body. The effect displayed on the stripe model is shown in fig. 31. This method is only suitable for situations where the complete information of concealed objects is not displayed and the complete human body image does not need to be displayed.

It should be understood that the above detailed description of the technical solution of the present invention with the help of preferred embodiments is illustrative and not restrictive. On the basis of reading the description of the invention, a person skilled in the art can modify the technical solutions described in the embodiments, or make equivalent substitutions for some technical features; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention. The scope of the invention is only limited by the appended claims.

Claims (3)

1. A privacy protection method based on millimeter wave imaging is characterized by comprising the following steps: millimeter wave scanning is carried out on the person to be detected to obtain an original image, and the original image is processed in a mode that

S1, performing millimeter wave scanning on the person to be detected to obtain an original image, wherein the person to be detected is required to enter a millimeter wave scanning detection area, and scanning and detecting are performed in a millimeter wave active/passive mode through a scanning device to obtain the original image;

s2, adjusting the original image to obtain a target image, wherein

S21, preprocessing the original image to obtain a primary de-noised image;

s22, binarizing the preliminary de-noised image to obtain a preliminary binary image, wherein the binarizing of the preliminary de-noised image to obtain the preliminary binary image is to select a binarized threshold value by utilizing a pulse coupled neural network PCNN algorithm according to the maximum entropy criterion, and convert a gray scale image of the preliminary de-noised image into a binarized image by utilizing the threshold value, so that the segmentation of a human body region and a background region in the image is realized, wherein an iteration formula of an iteration process for selecting the threshold value of the image is as follows:

<math> <mrow> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>F</mi> </msub> </mrow> </msup> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>F</mi> </msub> <munder> <mo>&Sigma;</mo> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </munder> <msub> <mi>M</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>I</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> </msub> </mrow> </math>

<math> <mrow> <msubsup> <mi>L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>L</mi> </msub> </mrow> </msup> <msubsup> <mi>L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>L</mi> </msub> <munder> <mo>&Sigma;</mo> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </munder> <mrow> <msub> <mi>W</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> </mrow> </mrow> </math>

<math> <mrow> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msubsup> <mi>F</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mrow> <mo>(</mo> <mn>1</mn> <mo>+</mo> <msubsup> <mi>&beta;L</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <msup> <mi>e</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;</mi> <mi>T</mi> </msub> </mrow> </msup> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <msub> <mi>V</mi> <mi>T</mi> </msub> <msubsup> <mi>Y</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> </mrow> </math>

<math> <mrow> <msubsup> <mi>Y</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>=</mo> <mfenced open = '{' close = ''> <mtable> <mtr> <mtd> <mn>1</mn> <mo>(</mo> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>&gt;</mo> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> <mo>(</mo> <msubsup> <mi>U</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>&le;</mo> <msubsup> <mi>T</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mrow> <mo>(</mo> <mi>n</mi> <mo>)</mo> </mrow> </msubsup> <mo>)</mo> </mtd> </mtr> </mtable> </mfenced> </mrow> </math>

f is the n-times feedback input F of the ith and j neurons_i,j ⁽ⁿ⁾，I_i,jFor inputting a stimulus signal, the stimulus signal is the gray value of the ith and jth pixels in a matrix formed by image pixels, beta is a connection coefficient, and L_i,j ⁽ⁿ⁾Is a connecting item, T_i,j ⁽ⁿ⁾Is a dynamic threshold, i.e. the threshold to be solved, Y_i,j ⁽ⁿ⁾Is the PCNN pulse output value, U_i,j ⁽ⁿ⁾Is an internal activity item, where M of the internal connection matrix M, W_i,j,k,l、W_i,j,k,lAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾Middle Y_i,j ⁽ⁿ⁾W ═ M, α_F、α_L、α_TAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾、T_i,j ⁽ⁿ⁾Decay time constant of V_F、V_L、V_TAre respectively F_i,j ⁽ⁿ⁾、L_i,j ⁽ⁿ⁾、T_i,j ⁽ⁿ⁾The inherent potential of (a) is,

the following parameters were taken: alpha is alpha_F＝0.2，α_L＝2，α_T＝0.1，V_F＝0.1，V_L＝0.5，V_T＝20，β＝0.5， $W=M=\left[\begin{array}{ccccc}1/8& 1/5& 1/4& 1/5& 1/8\\ 1/5& 1/2& 1& 1/2& 1/5\\ 1/4& 1& 1& 1& 1/4\\ 1/5& 1/2& 1& 1/2& 1/5\\ 1/8& 1/5& 1/4& 1/5& 1/8\end{array}\right],$ Calculating;

s23, filtering and transforming the preliminary binary image by applying a mathematical morphology operation method, wherein the used morphology operation is the following processing procedures: carrying out corrosion operation on the image by using a square structural element with the side length of 5 to eliminate bright noise points outside a human body in the image; performing open operation on the image by using a square structural element with the side length of 4, and eliminating isolated regions and burrs of human body edges while keeping the size of the image; performing closed operation on the image by using a square structural element with the side length of 4, filling fine holes in the human body while keeping the size of the image, and smoothing the boundary of the human body; performing expansion operation on the image by using a square structural element with the side length of 5 to restore the image to the original size; through the process, noise with the length and the width both smaller than 5 can be removed, the hollow with the length and the width both smaller than 5 on the human body is filled, a target image formed after the processing comprises an approximately complete human body part, and if a large-area white interference area which is not eliminated exists in the image, the area of each communication area in the image is calculated to remove the area with the smaller area;

s3, segmenting and positioning the human body part according to the target image, which comprises the following substeps:

s31, determining the vertical central line of the human body, calculating the total pixel sum of the image of the human body region because the human body region in the target image has bilateral symmetry when determining the position of the vertical central line of the human body, and using S₀Representing, then calculating the pixel sum of the partial image of the human body from left to right according to the columns of the image from the left edge of the human body area, and using S₁Indicates when S is₁Is S₀1/2, the current row is the vertical center line of the human body;

s32, determining the coordinates of each key point of the human body of the target image and obtaining a horizontal dividing line between each part of the human body, wherein the coordinates of each key point of the human body are the position coordinates of each part of the human body, the position coordinates comprise edge end point coordinates and center point coordinates, and the part of the human body comprises: top of the head, sole, neck, upper torso, lower torso, crotch, knee, fingertip, and elbow;

s33, determining the width and the slope of each part of the human body, determining the width of each part of the human body according to the intersection of each part of the human body in the target image and each horizontal dividing line, and calculating by using the key point coordinates of each part of the human body to obtain the slope of the part;

s4, generating a human body model, which comprises representing each part by a rectangle or a parallelogram according to the coordinate, the width and the slope of each part key point, connecting all the parts together to obtain a human body model composed of the rectangle and/or the parallelogram, namely a human body strip model, and segmenting and positioning the human body part according to the target image to generate a rod-shaped model;

the hidden object detection method comprises the following steps of firstly detecting a non-human body target according to the original image to obtain a non-human body target distribution original image:

s51, edge detection, namely performing edge detection on the original image, performing edge detection by using a Sobel operator, and preliminarily identifying a non-human body target;

s52, highlighting, namely highlighting a non-human body target distribution area through mathematical morphology operation, firstly carrying out corrosion operation, and then carrying out expansion operation; wherein, the corrosion and expansion operations respectively use square structural elements with the side length of 2 and 4, thereby highlighting the non-human body target distribution area diagram;

s53 regularization, namely selecting a minimum circumscribed rectangle according to the boundary of the non-human body target distribution area to convert the irregular area into a non-human body target regular area distribution map;

s54, fusing the images, namely fusing the non-human body target regular region distribution map and the original image to obtain a non-human body target distribution original image;

s6, obtaining the position distribution information of the non-human body target distribution original image relative to the human body by using the human body bar model, inputting the non-human body target distribution original image to the bar combination model, thus obtaining the distribution of the non-human body target in the human body contour edge by using the human body contour information given by the strip model in the bar combination model, and further obtaining the relative position relation between the non-human body target and the joint point by using the human body joint point position information given by the bar model in the bar combination model; the bar combination model comprises a bar model for providing each key point of the human body; generating a strip model providing human body contour information; combining the rod model and the strip model to generate a strip-rod combination model;

s7, identifying the type of the non-human body target and displaying the position distribution information of the hidden article relative to the human body, wherein the method comprises the following steps:

s71, positioning the exposed part of the human body, wherein the exposed part of the human body comprises a head, a wrist and a palm, and the more targeted part positioning is performed in the segmentation and positioning of the human body part;

s72, classifying, namely classifying the non-human body targets, and determining the non-human body targets distributed on the exposed parts of the human body as non-hidden objects, wherein the non-hidden objects comprise glasses, buttons, watches or rings; due to the fact that clothes cover the non-human body targets outside the exposed parts of the human body, the non-human body targets outside the exposed parts of the human body cannot be directly checked through security check personnel, the non-human body targets distributed outside the exposed parts of the human body are determined to be hidden objects, and important attention needs to be paid;

s73, displaying, namely, rejecting the original image of the non-concealed article and displaying the distribution information of the original image of the concealed article on the bar combination model;

detecting human bodies and hidden objects according to the original images;

determining a private part of a human body, the determining the private part of the human body comprising:

s8, judging the gender of the person to be detected, when the person to be detected is male, determining the head area and the area with the width 1/2 of the human waist downward trunk as privacy parts, and when the person to be detected is female, determining the head area, the area with the width 1/2 of the human waist downward trunk and the area with the height 1/2 of the human trunk downward from the upper end of the trunk as privacy parts;

the method is used for shielding the private parts of the human body and marking the information of the hidden objects on the human body, and comprises the following steps:

s9, blurring the private part on the original image to form a partially blurred original image;

s10, marking the concealed article with a mark frame on the partially blurred original image;

s11, selecting a target image in the human body detection, wherein the target image is the binary image;

s12, judging whether the hidden article is in the private part of the human body, if so, using a color block with different color from the human body to represent the hidden article and marking the hidden article on the target image; and if not, displaying the original image of the concealed article on the destination image.

2. The privacy protection method based on millimeter wave imaging according to claim 1, wherein the shielding of the privacy part of the human body and the marking of the information of the hidden objects on the human body comprises:

carrying out full blurring treatment on the original image to form a full blurring original image;

judging whether the hidden object is positioned at the private part of the human body, if so, using a color block with the color different from that of the human body to represent the hidden object and marking the hidden object on the completely blurred original image; if not, the original image of the concealed item is displayed over the fully obscured original image.

3. The privacy protection method based on millimeter wave imaging according to claim 1, wherein the shielding of the privacy part of the human body and the marking of the information of the hidden objects on the human body comprises:

selecting a human body model in the human body detection;

concealed items are represented and marked on the mannequin with color blocks that are different from the mannequin color.