CN114581944A

CN114581944A - Millimeter wave image processing method and device and electronic equipment

Info

Publication number: CN114581944A
Application number: CN202210153541.2A
Authority: CN
Inventors: 赵中玮; 陶毅
Original assignee: Hangzhou Ruiying Technology Co ltd
Current assignee: Hangzhou Ruiying Technology Co ltd
Priority date: 2022-02-18
Filing date: 2022-02-18
Publication date: 2022-06-03
Anticipated expiration: 2042-02-18
Also published as: CN114581944B

Abstract

The embodiment of the invention provides a millimeter wave image processing method and device and electronic equipment, and relates to the technical field of image processing. The method comprises the following steps: acquiring a millimeter wave image and a depth image of a detection object; identifying a human body part to be processed of a detection object from the depth image; wherein, the part to be treated of the human body comprises the contour edge and/or the contour internal area of the human body; determining enhancement coefficients respectively corresponding to characteristic points in the contour edge and/or the contour internal area; and performing display enhancement processing on the characteristic points of the part to be processed of the human body of the detection object on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image. Compared with the prior art, the scheme provided by the embodiment of the invention can improve the display quality of the millimeter wave image, particularly the image display quality of the edge part of the human body in the millimeter wave image, so that the edge display of the human body is clearer, the accuracy of image identification is improved, and the omission factor and the false detection rate are reduced.

Description

Millimeter wave image processing method and device and electronic equipment

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a millimeter wave image processing method and apparatus, and an electronic device.

Background

At present, along with the continuous increase of crowd intensive places, the security inspection equipment is arranged in the crowd intensive places to carry out security inspection, so that the security inspection becomes a normal state.

Because the millimeter waves can penetrate through conventional clothes, the millimeter wave images obtained by the millimeter wave imaging device can find forbidden articles hidden under the clothes, and have the characteristics of non-contact property, safety, civilization and the like, so that the millimeter wave imaging device becomes important equipment for security inspection in crowded places such as rail transit, airports, large-scale activity places and the like.

In the process of utilizing millimeter wave imaging equipment to carry out security inspection to the personnel who are detected, only the human trunk main part of the personnel who are detected is just to millimeter wave radar or millimeter wave antenna array face for echo signal is mostly collected by the radar, consequently, the image display quality of the human trunk main part of the personnel who are detected is higher relatively. However, for the human body trunk edge part of the detected person, for example, the arm edge, the trunk edge, the leg edge and the like, since the human body trunk edge part of the detected person is not over against the millimeter wave radar, the scattering angle of the echo signal is large, the signal loss is large, and the image contrast is low, so that the image display quality of the human body trunk edge part of the detected person is relatively poor, and further, the accuracy of image identification for the detected person is low, and the undetected rate and the false detection rate are high.

As shown in fig. 1, the millimeter wave image obtained by performing security check on the detected person by the millimeter wave imaging device is relatively good in image display quality of the main body part of the human body of the detected person, and relatively poor in image display quality of the edge part of the human body of the detected person.

Therefore, how to improve the display quality of the millimeter wave image, especially the image display quality of the human body edge part, so as to improve the accuracy of image identification, thereby reducing the omission factor and the false detection factor, and becoming the problem to be solved urgently at present.

Disclosure of Invention

Embodiments of the present invention provide a millimeter wave image processing method, a millimeter wave image processing apparatus, and an electronic device, so as to improve display quality of a millimeter wave image, especially display quality of an image at an edge of a human body in the millimeter wave image, thereby improving accuracy of image recognition, and reducing a missing rate and an erroneous detection rate. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a millimeter wave image processing method, where the method includes:

acquiring a millimeter wave image and a depth image of a detection object;

identifying and obtaining a human body part to be processed of the detection object from the depth image; wherein the part of the human body to be treated comprises the contour edge and/or the contour inner area of the human body;

determining enhancement coefficients corresponding to the characteristic points of the contour edge and/or the contour internal area respectively;

and performing display enhancement processing on the characteristic points of the part to be processed of the human body of the detection object on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image.

In a second aspect, an embodiment of the present invention provides a millimeter wave image processing apparatus, including:

the image acquisition module is used for acquiring a millimeter wave image and a depth image of the detection object;

the part identification module is used for identifying and obtaining a part to be processed of the human body of the detection object from the depth image; wherein the part of the human body to be treated comprises a contour edge and/or a contour inner area of the human body;

the coefficient determining module is used for determining enhancement coefficients corresponding to the characteristic points of the contour edge and/or the contour internal area respectively;

and the image enhancement module is used for carrying out display enhancement processing on the characteristic points of the part to be processed of the human body of the detection object on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the steps of any millimeter wave image processing method provided by the first aspect when executing the program stored in the memory.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when being executed by a processor, the computer program implements the steps of any of the millimeter wave image processing methods provided in the first aspect.

In a fifth aspect, an embodiment of the present invention provides a computer program product containing instructions, which when run on a computer, causes the computer to perform the steps of any of the millimeter wave image processing methods provided in the first aspect.

The embodiment of the invention has the following beneficial effects:

as can be seen from the above, with the solution provided by the embodiment of the present invention, in the process of processing the millimeter wave image of the detection object, firstly, the millimeter wave image and the depth image of the detection object are obtained, secondly, the to-be-processed part of the human body of the detection object is identified and obtained from the depth image, where the to-be-processed part of the human body includes the contour edge and/or the contour inner region of the human body, and then, the enhancement coefficients corresponding to the feature points located in the contour edge and/or the contour inner region are determined, and based on the correspondence between the enhancement coefficients and the feature points, the feature points of the to-be-processed part of the human body of the detection object on the millimeter wave image are subjected to display enhancement processing, so that the target image can be obtained.

Based on the above, by applying the scheme provided by the embodiment of the invention, in the process of processing the millimeter wave image of the detection object, the image enhancement of the millimeter wave image of the detection object can be realized by using the depth image of the detection object, so that the image display quality of the outline edge and/or the outline inner area of the detection object in the millimeter wave image is improved, the human body edge display can be clearer, the accuracy of image identification is further improved, and the omission ratio and the false detection ratio are reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.

FIG. 1 shows the result of image recognition of security check of a person to be detected by the millimeter wave imaging device;

fig. 2 is a schematic flowchart of a first millimeter wave image processing method according to an embodiment of the present invention;

fig. 3(a) is an installation diagram of a first millimeter wave radar and depth camera/visible light camera according to an embodiment of the present invention;

FIG. 3(b) is an installation diagram of a second millimeter wave radar and depth camera/visible light camera according to an embodiment of the present invention;

FIG. 3(c) is an integrated illustration of a multi-block small area millimeter wave radar and a depth camera/visible light camera;

FIG. 3(d) is an illustration of an installation of a third millimeter wave radar and depth camera/visible light camera according to the integrated manner shown in FIG. 3(c) provided in the present invention;

FIG. 3(e) is an illustration of an installation of a fourth millimeter wave radar and depth camera/visible light camera according to the integrated manner shown in FIG. 3(c) provided in the present invention;

FIG. 4 is a schematic flowchart of another millimeter wave image processing method according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a system to which a millimeter wave image processing method according to an embodiment of the present invention is applied;

FIG. 6 is a schematic flowchart of another millimeter wave image processing method according to an embodiment of the present invention;

FIG. 7 is a diagram of a normal on a human body contour region provided by an embodiment of the invention;

FIG. 8 is a schematic flowchart of another millimeter wave image processing method according to an embodiment of the present invention;

FIG. 9 is a flow chart illustrating an implementation of the present invention;

FIG. 10 is a flowchart illustrating another millimeter-wave image processing method according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a millimeter wave image processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.

In the process of utilizing millimeter wave imaging equipment to carry out security inspection to the personnel who are detected, only the human trunk main part of the personnel who are detected is just to millimeter wave radar or millimeter wave antenna array face for echo signal is mostly collected by the radar, consequently, the image display quality of the human trunk main part of the personnel who are detected is higher. However, for the human body trunk edge part of the detected person, for example, the arm edge, the trunk edge, the leg edge and the like, since the human body trunk edge part of the detected person is not over against the millimeter wave radar, the scattering angle of the echo signal is large, the signal loss is large, the image contrast is low, the image display quality of the human body trunk edge part of the detected person is poor, and further, the accuracy of image identification for the detected person is low, so that the missed detection rate and the false detection rate are high.

In order to at least solve the above technical problem, an embodiment of the present invention provides a millimeter wave image processing method.

The method can be applied to various application scenes for performing image recognition by using the millimeter wave image, such as security check on personnel in an airport, security check on personnel in a subway station, and the like.

Moreover, the method can be applied to various electronic devices which can acquire millimeter wave images and depth images and process the acquired images, such as millimeter wave imaging devices, computers, security check devices, background servers of security check systems and the like, and is hereinafter referred to as electronic devices for short.

For example, when the method is applied to a background server of a security inspection system, the background server may respectively acquire a millimeter wave image and a depth image of a detection object at a millimeter wave imaging device and a depth image acquisition device, and process the acquired images to complete image enhancement of the millimeter wave image of the detection object; for another example, the method may be applied to any electronic device independent of the security check system, and the electronic device may communicate with the millimeter wave imaging device and the depth image capturing device in the security check system to obtain the millimeter wave image and the depth image of the detection object, thereby completing image enhancement of the millimeter wave image of the detection object. This is all reasonable.

Based on this, the embodiment of the present invention does not limit the application scenario and the execution subject of the method.

The millimeter wave image processing method provided by the embodiment of the invention can comprise the following steps:

acquiring a millimeter wave image and a depth image of a detection object;

Based on the scheme provided by the embodiment of the invention, in the process of processing the millimeter wave image of the detection object, the image enhancement of the millimeter wave image of the detection object can be realized by utilizing the depth image of the detection object, so that the image display quality of the outline edge and/or the outline inner area of the detection object in the millimeter wave image is improved, the human body edge display is clearer, the image identification accuracy is further improved, and the omission ratio and the false detection ratio are reduced.

Hereinafter, a millimeter wave image processing method according to an embodiment of the present invention will be specifically described with reference to the drawings.

Fig. 2 is a millimeter wave image processing method according to an embodiment of the present invention, and as shown in fig. 2, the method includes the following steps S201 to S204.

S201: and acquiring a millimeter wave image and a depth image of the detection object.

In order to ensure the safety of the crowd in a plurality of crowded places such as airports, it is necessary to perform security check on each person in the place, and each person may be a detection target.

When security inspection is performed on a detection object, firstly, a millimeter wave image and a depth image of the detection object are acquired.

In this way, when the detection object is subjected to security inspection, the electronic device may acquire the millimeter wave image of the detection object acquired by the millimeter wave imaging device and acquire the depth image of the detection object acquired by the depth image acquisition device.

For example, the millimeter wave imaging device may be a millimeter wave radar, illustratively, a millimeter wave area radar, a millimeter wave line radar, or the like. The depth image acquisition device can be a depth camera, and can also be an image acquisition device integrated with a depth image acquisition module and a visible light image acquisition module.

The millimeter wave image of the detection object acquired by the millimeter wave imaging device is a three-dimensional depth image.

Further, the depth image is also called a range image, and refers to an image in which the distance (depth) from the image capturing device to each point in the scene is taken as a pixel value, and directly reflects the geometry of the visible surface of the captured object. The depth image can be calculated as point cloud data through coordinate conversion, and the point cloud data with regular and necessary information can also be inversely calculated as depth image data, and the depth image is a three-dimensional representation form of an object.

In practical application, above-mentioned millimeter wave imaging device and depth image acquisition equipment can install through multiple mode, and is preferred, and millimeter wave imaging device's scanning antenna array face is perpendicular with the axis direction of depth image acquisition equipment's camera lens, and furthest's assurance millimeter wave imaging device and depth image acquisition equipment carry out image acquisition to the same one side of detection object. Next, description will be made taking a millimeter wave radar and a depth camera as examples.

Optionally, as shown in fig. 3(a), the installation method is a millimeter wave radar and depth image installation method, where two depth cameras are installed on the upper and lower sides of the millimeter wave radar, and detect the detection object.

Wherein, millimeter wave radar's area is less, and the field of view scope of radar can't cover whole human body, needs realize whole human formation of image through the mode that reciprocates, and the depth camera that corresponds also can't cover whole human body, need follow the millimeter wave radar and reciprocate, accomplishes whole human scanning.

Alternatively, as shown in fig. 3(b), another installation manner of the millimeter wave radar and the depth image is adopted, in which three depth cameras are installed on any one of the left and right sides of the millimeter wave radar, and detect the detection object.

The area of the millimeter wave radar is large, the field of view range of the radar is enough to cover the whole human body, the field of view of the depth camera is small, the depth camera cannot cover the whole human body, and a plurality of depth cameras and visible light cameras are used at intervals to achieve imaging of the whole human body.

Optionally, as shown in fig. 3(c), the method is another installation method of a millimeter wave radar and a depth image, where the millimeter wave radar is spliced according to a plurality of small-area modules, transceiver chips in the small-area radar are arranged in a manner shown in fig. 3(c), a square frame with stripes in the figure represents a receiver chip, a blank square frame represents a transmitter chip, and at this time, a central portion 800 of the module is not arranged with the transceiver chips, and a depth camera may be embedded in the central portion 800, so as to integrate multiple detectors.

Optionally, as shown in fig. 3(d), in order to adopt an installation manner of the millimeter wave radar and the depth image in the integration manner of the millimeter wave radar and the depth camera shown in fig. 3(c), the depth camera is installed at a central portion of the millimeter wave radar, and the detection object is detected.

Wherein, millimeter wave radar's area is less, and radar field of view scope can't cover whole human body, needs realize whole human formation of image through the mode that reciprocates, and the depth camera that corresponds also can't cover whole human body, need follow millimeter wave radar and reciprocate, accomplishes whole human scanning.

Optionally, as shown in fig. 3(e), in order to adopt another installation manner of the millimeter wave radar and the depth image in the integration manner of the millimeter wave radar and the depth camera shown in fig. 3(c), three depth cameras are installed in the middle area of the millimeter wave radar at the same interval, and the detection object is detected.

The area of the millimeter wave radar is large, the radar view field range is enough to cover the whole human body, the depth camera view field is small, the whole human body cannot be covered, and a plurality of depth cameras and visible light cameras are used at intervals to achieve imaging of the whole human body.

Fig. 3(a) to 3(e) are merely illustrative and non-limiting of the installation manners of the millimeter wave imaging device and the depth image capturing device, and any installation manner of the millimeter wave imaging device and the depth image capturing device that can acquire the depth image and the millimeter wave image of the detection object falls within the protection scope of the present invention.

Optionally, the millimeter-wave radar used above may be an area array radar, and may also be a linear array radar.

Optionally, the depth camera used above may be an area-array depth camera, and may be a line-array depth camera.

Optionally, when the millimeter wave radar is an area array radar, the millimeter wave radar may be used in combination with an area array depth camera, or in combination with a linear array depth camera. Correspondingly, when the used millimeter wave radar is a linear array radar, the millimeter wave radar can be used in combination with an area array depth camera and also can be used in combination with the linear array depth camera.

S202: and identifying the human body to-be-processed part of the detection object from the depth image.

The part to be processed of the human body comprises the contour edge and/or the contour internal area of the human body, namely the part needing image display enhancement processing.

And after the depth image of the detection object is acquired, recognizing the human body contour edge and/or the contour internal area of the detection object from the depth image. Namely, the following three cases are included:

in an example, a human body contour edge of the detection object may be identified from the depth image.

In the second example, the human body contour internal region of the detection object can be identified from the depth image.

In the third example, the human body contour edge and the contour inner region of the detection object can be identified from the depth image.

In the embodiment of the application, the part of the detection object, which needs to be subjected to image display enhancement, can be determined according to the requirement.

Optionally, in a specific implementation manner, the step S202 may include the following step 2021:

step 2021: and smoothing the region where the human body is located on the depth image, and identifying and obtaining the part to be processed of the human body of the detection object from the depth image after smoothing.

In this specific implementation manner, since wrinkles exist in human clothing, the image effect may be affected by the wrinkles in the depth image, and therefore, after the depth image of the detection object is obtained, the region where the human body is located on the depth image may be smoothed, and the influence of the wrinkles in the clothing on the image is eliminated, so that the to-be-processed part of the human body of the detection object may be identified from the smoothed depth image.

For example, the entire region of the region where the human body is located on the depth image may be smoothed by using an algorithm such as a linear smoothing algorithm or a nonlinear smoothing algorithm, and then the to-be-processed portion of the human body of the detection object may be identified and obtained from the depth image after the smoothing processing.

Optionally, the step 2021 may include the following step 2021A:

step 2021A: and identifying a sub-region of the region where the human body is located on the depth image and performing smoothing processing on the sub-region.

In this specific implementation manner, when smoothing the region where the human body is located on the depth image, the sub-region where the human body is located on the depth image has wrinkles may be first identified, and then the identified sub-region where the wrinkles exist may be smoothed, so that the to-be-processed part of the human body of the detection object may be identified and obtained from the smoothed depth image.

The smoothing processing may be performed on the identified sub-region with the wrinkle by using an algorithm such as a linear smoothing algorithm, a nonlinear smoothing algorithm, or the like.

S203: and determining the enhancement coefficients corresponding to the characteristic points of the edge and/or the inner area of the contour respectively.

The feature points refer to any position points of the contour edge and/or the contour inner region of the human body.

Optionally, the enhancement coefficients corresponding to the feature points at the edge of the contour may be different from the enhancement coefficients corresponding to the feature points at the inner region of the contour, for example, the enhancement coefficients have a larger value at the edge of the human body and a smaller value at the center of the trunk, and further, the enhancement coefficient values corresponding to the feature points closer to the edge of the human body may be larger. At this time, enhancement of the targeted image display effect of the human body contour edge and the human body contour inner region can be achieved.

Optionally, the enhancement coefficients corresponding to the feature points at the edge of the contour may be the same as the enhancement coefficients corresponding to the feature points in the inner region of the contour. At this moment, after the image display effect of the internal area of the human body contour is enhanced, the display difference between the corresponding human body contour edge and the image background is more obvious, so that the effect of enabling the human body edge part to be displayed more clearly can be achieved.

Specifically, the part of the human body to be processed, which needs to be subjected to image display enhancement, may be a main body part of a human body, an extremity part of the human body, an edge part of the human body, or the like, and the embodiment of the present invention is not particularly limited.

S204: and performing display enhancement processing on the characteristic points of the part to be processed of the human body of the detection object on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image.

After determining the enhancement coefficients corresponding to each feature point, the corresponding relationship between the enhancement coefficients and the feature points can be established.

Since the millimeter wave image and the depth image are images acquired for the same detection object, points indicating the same part of the detection object in the millimeter wave image and the depth image correspond to the same point in the space where the detection object is located, respectively, and therefore, for each feature point determined in the depth image, a point corresponding to the feature point can be determined in the millimeter wave image as a feature point of the part to be processed of the human body of the detection object on the millimeter wave image. In other words, a specific feature point of the to-be-processed part of the human body in the embodiment of the present application refers to the same position point on the detection object in the millimeter wave image and the depth image. When the depth image and the millimeter wave image use the same coordinate system, the feature point at a certain position on the detection object in the depth image and the millimeter wave image can be represented by the same coordinate; when the depth image and the millimeter wave image use different coordinate systems, the coordinate representation of the feature point at the same position on the detection object in the depth image and the millimeter wave image can be obtained by transformation according to the transformation relation between the two coordinate systems.

Therefore, the characteristic points of the human body part to be processed of the detection object on the millimeter wave image can be subjected to display enhancement processing by utilizing the established corresponding relation between the enhancement coefficient and the characteristic points. That is, for each feature point specified in the depth image, the display enhancement processing is performed on the feature point corresponding to the feature point in the millimeter wave image using the enhancement coefficient corresponding to the feature point.

For example, if the enhancement coefficient corresponding to the feature point a determined in the depth image is p, and the feature point corresponding to the feature point a on the human body part to be processed of the detection object on the millimeter wave image is B, the feature point B in the millimeter wave image may be subjected to display enhancement processing by using the enhancement coefficient p, and the feature point a and the feature point B actually correspond to the same position point on the detection object.

After all the characteristic points of the part to be processed of the human body of the detection object on the millimeter wave image are displayed and enhanced, the millimeter wave image can be enhanced to obtain a target image.

Optionally, in a specific implementation manner, as shown in fig. 4, the step S204 may include the following step S401:

s401: and adjusting the pixel value corresponding to the characteristic point of the part to be processed of the human body on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic point to obtain a target image.

In this specific implementation manner, after determining the enhancement coefficients corresponding to the feature points located in the edge and/or the inner region of the contour, the pixel values corresponding to the feature points of the to-be-processed part of the human body on the millimeter wave image may be adjusted based on the correspondence between the enhancement coefficients and the feature points, so as to obtain the target image.

In some cases, the target image obtained above may be displayed, so that the user may further operate the target image to achieve various application purposes.

Based on this, alternatively, as shown in fig. 5, the millimeter wave radar 501, the depth camera 502, the data processing device 500, and the display device 503 may constitute a system in which the data processing device 500 corresponds to an electronic device that performs a millimeter wave image processing method provided by an embodiment of the present invention.

In the implementation of the present embodiment, the millimeter wave radar 501 and the depth camera 502 respectively acquire a millimeter wave image and a depth image of a detection object, and send the acquired millimeter wave image and depth image to the data processing device 500. The data processing device 500 receives the millimeter wave image and the depth image of the detection object, executes a millimeter wave image processing method provided by the embodiment of the present invention to obtain an enhanced target image, and then sends the target image to the display device 503. The display device 503 receives the target image and outputs the target image. Thus, the millimeter wave image of the detection object is enhanced.

The data processing device and the display device may be the same device or different devices.

For example, if the data processing device and the display device are the same device, the data processing device may be an independent computer, and the display device is a display of the computer; if the data processing device and the display device are different devices, the data processing device may be a background server of the security check system, and the display device is a separate computer.

As can be seen from the above, by applying the scheme provided by the embodiment of the present invention, in the process of processing the millimeter wave image of the detection object, the image enhancement of the millimeter wave image of the detection object can be realized by using the depth image of the detection object, so as to improve the image display quality of the contour edge and/or the contour internal region of the detection object in the millimeter wave image, further improve the accuracy of image identification, and reduce the omission factor and the false detection rate.

Optionally, in a specific implementation manner, as shown in fig. 6, the step S203 of determining the enhancement coefficients corresponding to the feature points located at the edge and/or the inner region of the contour respectively may include the following steps S601 to S603:

s601: respectively determining the corresponding normal lines of the characteristic points at the contour edge and/or the contour inner area on the human body contour surface;

s602: determining an included angle between a normal corresponding to each characteristic point and a reference line;

s603: and determining the respective corresponding enhancement coefficients of the characteristic points positioned at the edge and/or the inner area of the contour according to the included angle between the normal corresponding to each characteristic point and the reference line.

The enhancement coefficient is positively correlated with the included angle, namely the value of the enhancement coefficient is increased along with the increase of the included angle, and the reference line is vertical to the millimeter wave antenna array surface.

In this specific implementation manner, after obtaining the feature points located in the human body contour edge and/or the contour internal region of the detection object, the normal lines corresponding to the feature points on the human body contour surface may be respectively determined, so as to determine the included angle between the normal line corresponding to each feature point and the reference line, and further determine the enhancement coefficients corresponding to the feature points located in the contour edge and/or the contour internal region according to the included angle between the normal line corresponding to each feature point and the reference line.

Optionally, in a specific implementation manner, the step S603 may include the following step 6031:

step 6031: calculating the enhancement coefficient omega corresponding to each characteristic point at the edge and/or the inner region of the contour according to the following formula_α：

ω_α＝m-n·φ(θ_α)

Wherein, ω is_αEnhancement factor corresponding to alpha-th characteristic point at contour edge and/or contour inner region, theta_αIs the angle between the normal corresponding to the alpha-th characteristic point at the edge and/or the inner area of the contour and the reference line, phi (theta)_α) Is the included angle theta corresponding to the alpha-th characteristic point at the edge and/or the inner area of the contour_αAnd m and n are respectively preset constants of the related functions.

It should be emphasized that the above calculation formula of the enhancement coefficients is an example of the determination manner of the enhancement coefficients corresponding to the feature points at the edge and/or the inner region of the contour. Other functions which are used for calculating the corresponding enhancement coefficients of the specific points and ensure that the calculated enhancement coefficients are positively correlated with the included angles corresponding to the feature points belong to the protection scope of the embodiment of the invention, wherein the included angles corresponding to the feature points are as follows: and the included angle between the normal line corresponding to the characteristic point and the reference line.

Alternatively, in one embodiment, phi (theta)_α) The method comprises the following steps: angle theta corresponding to alpha-th characteristic point at contour edge and/or contour inner region_αThe associated cosine function, expressed as follows:

wherein the content of the first and second substances,

is the normal corresponding to the alpha-th characteristic point at the edge and/or the inner area of the contour,

is a reference line.

Optionally, for an α -th feature point of a human body contour edge and/or a contour internal region of the detected object on the depth image, a formula for determining a normal line corresponding to the feature point is as follows:

and G_α(x_α，y_α，z_α)＝0

Wherein the content of the first and second substances,

for the normal line corresponding to said alpha characteristic point, G_α(x_α，y_α，z_α) 0 is used to denote: the alpha-th feature point is located on the human body contour surface G of the detection object.

For a detection object, a depth camera can acquire a human body three-dimensional surface image of the detection object, and the output result of the depth camera comprises the distance from a position point (namely a characteristic point) on the surface of the detection object at the view range of the camera to a lens of the camera; where a position point on the surface of the detection object may be represented as (x, y), and a distance between the position point on the surface of the detection object and the camera lens may be represented as z, the output result of the depth camera may be represented as:

z＝g(x，y)

in the three-dimensional space, the output result is a curved surface, and the curved surface is: the human body contour surface G can be expressed as:

G(x，y，z)＝g(x，y)-z＝0

alternatively, in a specific coordinate system in which the millimeter wave image is located, for example, the coordinate system as shown in fig. 3(a), 3(b), 3(d), and 3(e), the reference line may be expressed as

The embodiment of the present invention is not particularly limited to the specific coordinate system, as long as the embodiment of the present invention is applicable.

For example, as shown in fig. 7, a diagram of a normal line on a human body contour surface is provided for an embodiment of the present invention, where a millimeter wave radar 700 transmits a plurality of transmission signals 701 to a detected person, the plurality of transmission signals 701 reflect a plurality of corresponding echo signals 702 after contacting the detected person, a normal line corresponding to a feature point on the human body contour surface is 703, and θ is an angle between the normal line 703 corresponding to the feature point on the human body contour surface and a reference line. The included angle between the normal line corresponding to the characteristic point on the human body contour surface and the reference line can be used for distinguishing the human body edge part from the trunk central part, the corresponding characteristic point when the theta value is smaller is the characteristic point on the trunk central part, and the corresponding characteristic point when the theta value is larger is the characteristic point on the human body edge part. The value of theta is generally [0, pi/2 ]. Also, as shown in fig. 7, the millimetric-wave image of the detected person includes a trunk body 705 and a plurality of trunk edges 704.

Note that, as shown in fig. 7, the reference line coincides with the moving track of the transmission signal 701. Of course, in other specific application scenarios, the moving tracks of the reference line and the transmission signal may not coincide.

Optionally, on the basis of the foregoing embodiment, identifying the to-be-processed part of the human body of the detection object from the depth image may include: constructing a target surface function G (x, y, z) by using the spatial position information of the characteristic points on the detection object carried in the depth image, and determining a first profile surface omega to which the characteristic points (x, y) of the detection object belong₁(otherwise known as human body boundary surface); and identifying and obtaining the human body to-be-processed part of the detection object from the depth image based on the target curved function and the first contour surface. Wherein, the target surface function can be expressed as: g (x, y, z) — G (x, y) -z ═ 0, and the coordinates of the feature points of the detection object satisfy (x, y) ∈ Ω₁The target surface function is solved as a constraint condition, and a human body to-be-processed part of the detection object, namely the contour edge and/or the contour inner area of the human body can be obtained from the depth image. It should be understood that the contour surface or the human body boundary surface of the detection object mentioned in the embodiments of the present application refers to a surface region formed with the human body contour edge as a boundary.

Because background noise exists in the millimeter wave image and the background noise affects the identification of the image, in order to improve the image display quality of the millimeter wave image, the accuracy of image identification is further improved, the missing detection rate and the false detection rate are reduced, and the background noise existing in the millimeter wave image can be suppressed.

Based on this, optionally, in a specific implementation manner, as shown in fig. 8, the millimeter wave image processing method provided in the embodiment of the present invention may further include the following step S801:

s801: and carrying out image registration on the millimeter wave image and the depth image to obtain a registered millimeter wave image and a registered depth image.

Accordingly, in this specific implementation manner, the step S202 of identifying the human body to-be-processed portion of the detection object from the depth image may include the following steps S802:

s802: and identifying and obtaining the human body to-be-processed part of the detection object from the registered depth image.

Illustratively, step S802 may include: constructing a target surface function G (x, y, z) by using the space position information of the feature points on the detection object carried in the registered depth image, and determining a first profile surface omega to which the feature points of the detection object belong₁(otherwise known as a body boundary surface); and identifying and obtaining the human body to-be-processed part of the detection object from the registered depth image based on the target surface function and the first contour surface. Wherein, the target surface function can be expressed as: g (x, y, z) ═ G (x, y) -z ═ 0, and the coordinates of the feature points of the detection object satisfy (x, y) ∈ Ω₁The target surface function is solved as a constraint condition, and a human body to-be-processed part of the detection object, namely the contour edge and/or the contour internal area of the human body can be obtained from the depth image after registration.

In the present embodiment, before the human body part to be processed of the detection object is identified from the depth image, since the above-described millimetric-wave image and the depth image are images captured for the same detection object, therefore, in the millimeter wave image and the depth image, points indicating the same portion of the detection object, respectively, corresponding to the same point in space where the detection object is located, and therefore, the millimeter wave image and the depth image can be subjected to image registration, so that points in the millimetric-wave image and the depth image respectively indicating the same position in the space where the detection object is located coincide at the same position, thereby, the registered millimeter wave image and the registered depth image of the detection object can be obtained, furthermore, the human body part to be processed of the detection object can be identified from the registered depth image.

Optionally, if the coordinate systems of the millimeter wave image and the depth image are the same, the millimeter wave image and the depth image may be overlapped, so that points respectively indicating the same position in the space where the detection object is located in the millimeter wave image and the depth image are overlapped at the same position, and thus, the millimeter wave image after registration and the depth image after registration of the detection object are obtained.

Optionally, if the coordinate systems of the millimeter wave image and the depth image are different, the millimeter wave image and the depth image may be converted into the same coordinate system, so that points in the millimeter wave image and the depth image, which respectively indicate the same position in the space where the detection object is located, are converted into the same point in the same coordinate system, thereby obtaining the millimeter wave image after registration and the depth image after registration of the detection object.

Optionally, in an embodiment, the millimeter wave image may be translated so that points in the millimeter wave image that respectively indicate the same position in the space where the detection object is located in the depth image coincide at the same position, so as to obtain a registered millimeter wave image of the detection object. At this time, the depth image before the image registration operation is performed and the depth image after the image registration operation is performed are the same image.

Accordingly, in this specific implementation manner, the step S204 of performing display enhancement processing on the feature point of the to-be-processed part of the human body of the detection object on the millimeter wave image based on the corresponding relationship between the enhancement coefficient and the feature point to obtain the target image may include the following steps S804:

s804: and performing display enhancement processing on the characteristic points of the to-be-processed part of the human body of the detection object on the registered millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image.

In this specific implementation manner, after the registered millimeter wave image is obtained, the enhancement coefficient corresponding to each feature point may be used to perform display enhancement processing on each feature point on the registered millimeter wave image based on the corresponding relationship between the enhancement coefficient and the feature point, so that the obtained enhanced millimeter wave image is used as the target image.

The specific implementation manner of step S804 is the same as the specific implementation manner of step S204, and is not described herein again.

For example, as shown in fig. 9, the person in fig. 9 is a detection object in the embodiment of the present invention, the millimeter wave radar is a millimeter wave imaging device in the embodiment of the present invention, the depth camera is a depth image collecting device in the embodiment of the present invention, the three-dimensional depth image is a millimeter wave image in the embodiment of the present invention, and the three-dimensional surface image is a depth image in the embodiment of the present invention; the three-dimensional image smoothing is a depth image acquired by the depth image acquisition equipment in the embodiment of the invention, the region of the human body of the detection object is determined, and smoothing is carried out; the normal image is calculated as a normal corresponding to the feature point located at the contour edge and/or the contour internal area in the embodiment of the present invention; the image registration is to perform image registration processing on a depth image acquired by the depth image acquisition device and a millimeter wave image acquired by the millimeter wave imaging device in the embodiment of the invention.

When a person enters a scanning area, scanning the person by using a millimeter wave radar and a depth camera to respectively obtain a three-dimensional depth image and a three-dimensional surface image of the person, then performing three-dimensional image smoothing processing on the three-dimensional surface image obtained by the depth camera, performing normal image calculation, further performing image registration on the obtained three-dimensional depth image and the three-dimensional surface image of the person, and further performing recognition of a part to be processed of a human body, such as human body contour recognition, and enhancing the images.

The embodiment of the present invention is not particularly limited to the mode of image registration of the millimeter wave image and the depth image.

Thus, after the image registration processing, some parts to be processed of the human body (for example, the parts to be processed of the human body which are regarded as the detection objects due to some reasons) in the millimeter wave image can be suppressed, but the parts actually belong to background noise points, so that the background noise of the millimeter wave image can be suppressed, the image display quality of the millimeter wave image can be improved, the accuracy of image identification can be further improved, and the omission ratio and the false detection ratio can be reduced.

In order to further enhance the effect of suppressing background noise, a visible light image may be added on the basis of obtaining a millimeter wave image and a depth image of the detection object.

Optionally, in a specific implementation manner, as shown in fig. 10, the millimeter wave image processing method provided in the embodiment of the present invention may further include the following step S1001:

s1001: acquiring a visible light image of a detection object;

correspondingly, in this specific implementation manner, in the step S801, performing image registration on the millimeter wave image and the depth image to obtain a registered millimeter wave image and a registered depth image, may include the following step S1002:

s1002: and carrying out image registration on the visible light image, the millimeter wave image and the depth image to obtain a registered millimeter wave image, a registered depth image and a registered visible light image.

In this specific implementation manner, it is further required to obtain a visible light image of the detection object, and because the millimeter wave image, the depth image, and the visible light image are images collected for the same detection object, points in the millimeter wave image, the depth image, and the visible light image that respectively indicate the same portion of the detection object correspond to the same point in the space where the detection object is located, the millimeter wave image, the depth image, and the visible light image may be subjected to image registration, so that the points in the millimeter wave image, the depth image, and the visible light image that respectively indicate the same position in the space where the detection object is located coincide at the same position, and thus, the registered millimeter wave image, the registered depth image, and the registered visible light image of the detection object may be obtained.

Optionally, in an embodiment, the depth image capturing device and the visible light image capturing device may be integrated, so that the depth image and the visible light image of the detected object are integrated, and the registration is automatically performed during the image capturing process without performing an additional image registration process on the two. Therefore, in this case, it is only necessary to perform image registration of the millimeter wave image and the visible light image or the depth image.

For example, in the case that the depth image capture device and the visible light image capture device are integrated, in practical applications, the visible light image capture device, the depth image capture device and the millimeter wave imaging device may be installed in various manners, such as a visible light camera, a depth camera and a millimeter wave radar, which may be installed as shown in fig. 3(a) -3 (e).

Optionally, as shown in fig. 3(a), the method is an installation manner of a visible light camera, a depth camera and a millimeter wave radar, in which the two depth cameras and the visible light camera are installed on the upper and lower sides of the millimeter wave radar, and detect the detection object.

Wherein, the area of millimeter wave radar is less, and the field of view scope of radar can't cover whole human body, needs realize whole human formation of image through the mode that reciprocates, and the depth camera that corresponds and visible light camera also can't cover whole human body, need follow the millimeter wave radar and reciprocate, accomplish whole human scanning.

Optionally, as shown in fig. 3(b), another installation manner of the visible light camera, the depth camera, and the millimeter wave radar is adopted, in which the three depth cameras and the visible light camera are installed on any one of the left and right sides of the millimeter wave radar, and detect the detection object.

The area of the millimeter wave radar is large, the field of view range of the radar is enough to cover the whole human body, the field of view of the depth camera and the field of view of the visible light camera are small, the depth camera and the visible light camera cannot cover the whole human body, and the depth camera and the visible light camera need to be used at intervals to achieve imaging of the whole human body.

Optionally, as shown in fig. 3(c), the method is another installation method of a visible light camera, a depth camera and a millimeter wave radar, in which the millimeter wave radar is spliced according to a plurality of small-area modules, transceiver chips in the small-area modules are arranged in a manner shown in fig. 3(c), a frame with stripes in the figure represents a receiving chip, a blank frame represents a transmitting chip, at this time, a central portion 800 of the module is not arranged with the transceiver chips, and the depth camera can be embedded in the central portion 800, so that integration of multiple detectors is achieved.

Optionally, as shown in fig. 3(d), in order to adopt an installation manner of the visible light camera, the depth camera, and the millimeter wave radar in the integration manner of the visible light camera, the depth camera, and the millimeter wave radar shown in fig. 3(c), the depth camera and the visible light camera are installed at a central portion of the millimeter wave radar, and the detection object is detected.

Wherein, millimeter wave radar's area is less, and radar field of view scope can't cover whole human body, needs realize whole human formation of image through the mode that reciprocates, and the depth camera that corresponds and visible light camera also can't cover whole human body, need follow millimeter wave radar and reciprocate, accomplish whole human scanning.

Optionally, as shown in fig. 3(e), in order to adopt another installation manner of the visible light camera, the depth camera, and the millimeter wave radar in the integration manner of the visible light camera, the depth camera, and the millimeter wave radar shown in fig. 3(c), the three depth cameras and the visible light camera are installed in the middle area of the millimeter wave radar at the same intervals, and the detection object is detected. The area of the millimeter wave radar is large, the range of the radar field of view is enough to cover the whole human body, the fields of view of the depth camera and the visible light camera are small, the whole human body cannot be covered, and multiple depth cameras and visible light cameras are used at intervals to achieve imaging of the whole human body.

The above fig. 3(a) -3 (e) are merely examples of the installation manners of the millimeter wave imaging device, the depth image capturing device, and the visible light image capturing device, but are not limited thereto, and any installation manner of the millimeter wave imaging device, the depth image capturing device, and the visible light image capturing device that can acquire the depth image, the millimeter wave image, and the visible light image of the detection object belongs to the protection scope of the present invention.

Furthermore, the method for identifying and obtaining the human body to-be-processed part of the detection object from the depth image further comprises the following steps: determining a second profile surface omega to which the characteristic point of the detection object belongs by using the registered visible light image₂So that the first profile surface omega is based on the target surface function₁And a second profile surface omega₂And identifying and obtaining the human body to-be-processed part of the detection object from the registered depth image. Wherein, the target surface function can be expressed as: g (x, y, z) ═ G (x, y) -z ═ 0, and the coordinates of the feature points of the detection object satisfy (x, y) ∈ Ω₁And (x, y) is epsilon to omega₂The target surface function is solved as a constraint condition (that is, the required feature points are located on the first contour surface and the second contour surface at the same time), and the human body to-be-processed part of the detection object, that is, the contour edge and/or the contour internal region of the human body can be obtained from the registered depth image.

Optionally, the visible light camera used above may be an area-array visible light camera, and may be a linear-array visible light camera.

Optionally, when the millimeter wave radar is an area array radar, the millimeter wave radar may be used in combination with an area array depth camera and an area array visible light camera, or in combination with a linear array depth camera and a linear array visible light camera. Correspondingly, when the millimeter wave radar is a linear array radar, the millimeter wave radar can be used in combination with an area array depth camera and an area array visible light camera, and also can be used in combination with the linear array depth camera and the linear array visible light camera.

Optionally, when the millimeter wave radar is an area array radar, the millimeter wave radar may be used in combination with an area array depth camera and a linear array visible light camera, or may be used in combination with a linear array depth camera and an area array visible light camera. Correspondingly, when the millimeter wave radar is a linear array radar, the millimeter wave radar can be used in combination with an area array depth camera and a linear array visible light camera, and also can be used in combination with the linear array depth camera and the area array visible light camera.

Therefore, the image registration method of the visible light image, the depth image and the millimeter wave image may be the same as or different from the image registration method of the millimeter wave image and the depth image, and the embodiment of the present invention is not particularly limited.

For the convenience of understanding the millimeter wave image processing method provided by the above embodiment of the present invention, a specific example is described below. As can be seen from the foregoing embodiments, in the three-dimensional space, the output result of the depth camera is a curved surface, i.e. a human body contour surface, which can be expressed as the following target curved surface function:

G(x，y，z)＝g(x，y)-z＝0

since the three-dimensional surface image of the human body is interfered by clothes, a first contour plane to which the feature points of the detection object on the depth image belong, namely a boundary plane of the human body (including clothes) in a three-dimensional space, which is expressed as omega, can be obtained through human body contour recognition₁；

Further, the human body part to be processed of the detection object can be identified in the depth image, and the human body part to be processed of the detection object is expressed as:

g (x, y) -z is less than or equal to 0 and (x, y) belongs to omega₁

Wherein g (x, y) -z is less than 0 and (x, y) is equal to omega₁The points of (1) are points in the inner area of the human body contour of the detection object;

satisfy g (x, y)) -z is 0 and (x, y) e Ω₁The point of (2) is a point at the edge of the human body contour of the detection object.

Therefore, the characteristic point can be satisfied with g (x, y) -z < 0 and (x, y) ∈ Ω₁The point (c) may be such that (x, y) ∈ Ω and g (x, y) -z is 0₁The point (c) may be such that g (x, y) -z is not more than 0 and (x, y) ∈ Ω₁Point (2) of (c).

When the feature points include a feature point at an inner region of a human body contour of the detection object in the depth image and a feature point at an edge of the human body contour of the detection object in the depth image, (x) is satisfied for each feature point α at the surface of the human body_α，y_α)∈Ω₁And g (x)_α，y_α)-z_α≤0。

Further, the above formula ω is used_α＝m-n·φ(θ_α) When calculating the enhancement coefficient corresponding to each feature point, for example, if m is 2 and n is 1, the enhancement coefficient for the first α feature point is:

wherein, ω is_αThe enhancement coefficient of the alpha characteristic point in the part to be processed of the human body is obtained; x is the number of_αThe abscissa of the alpha-th characteristic point in the part to be treated of the human body is taken as the coordinate; y is_αIs the ordinate of the alpha characteristic point in the part to be processed of the human body; z is a radical of formula_αThe vertical coordinate of the alpha characteristic point in the part to be processed of the human body is taken as the coordinate; theta_αIs the included angle between the normal corresponding to the alpha-th characteristic point in the part to be processed of the human body and the reference line. Illustratively, regarding the establishment of the coordinate system, as shown in fig. 3(a), the plane in which the millimeter wave antenna array is located is an x-y plane, and the direction perpendicular to the millimeter wave antenna array and directed toward the detection object is a z-axis direction. The vertical coordinate is the coordinate value of the characteristic point in the z-axis direction. Phi (theta)_α) This may be implemented in the form of a cosine function, for example.

Of course, since there may be wrinkles in the clothes of the detection object, the human body contour surface may be disturbed by the clothes, and thus, there may be wrinkles, so that the human body contour surface may be smoothed to obtain a smooth curved surface.

Optionally, the surface of the human body contour is smoothed, and the obtained smooth curved surface is:

G^*(x，y，z)＝g^*(x，y)-z＝0

the boundary surface of the human body in the depth image of the detection object in the three-dimensional space is Ω₁。

Further, the human body part to be processed of the detection object can be identified in the smooth curved surface, and the human body part to be processed of the detection object is expressed as:

g^*(x, y) -z is less than or equal to 0, and (x, y) belongs to omega₁

Wherein, g is satisfied^*(x, y) -z < 0 and (x, y) e.g.. omega₁The points of (1) are points in the inner area of the human body contour of the detection object;

satisfies g^*(x, y) -z is 0 and (x, y) is ∈ Ω₁That is, the points at the edge of the human body contour of the detection object.

Therefore, the feature point may be that g is satisfied^*(x, y) -z < 0 and (x, y) e.g.. omega₁May satisfy g^*(x, y) -z is 0 and (x, y) is ∈ Ω₁It is also possible to satisfy g^*(x, y) -z is less than or equal to 0 and (x, y) belongs to omega₁Point (2) of (c).

When the characteristic points comprise characteristic points which are positioned on the smooth curved surface of the human body and are positioned in the inner area of the human body outline and characteristic points which are positioned on the smooth curved surface of the human body and are positioned on the edge of the human body outline, then (x) is satisfied for each characteristic point alpha_α，y_α)∈Ω₁And g is^*(x_α，y_α)-z_α≤0。

the meaning of the individual parameters in the expression function of the enhancement coefficients can be found, inter alia, in the preceding explanations.

In order to further enhance the effect of suppressing background noise, a visible light image may be added on the basis of obtaining a millimeter wave image and a depth image of the detection object. After the registration processing of the three images, the human body contour surface obtained based on the registered depth image can be assumed to be still represented by the following function:

G(x，y，z)＝g(x，y)-z＝0

the first contour surface to which the feature point of the detection object belongs on the registered depth image is represented as omega₁And the second contour surface to which the characteristic point of the detection object on the registered visible light image belongs is expressed as omega₂In the process of identifying and obtaining the human body to-be-processed part of the detection object from the registered depth image, the feature points (x, y) of the human body to-be-processed part belong to the first profile surface and the second profile surface at the same time, namely the feature points of the human body part are determined according to the following formula:

g (x, y) -z ≦ 0 and (x, y) ∈ Ψ, Ψ ∈ (Ω)₁∩Ω₂)

Wherein: the point satisfying g (x, y) -z is less than 0 and (x, y) belongs to psi, namely the point in the inner region of the human body contour of the detected object;

the point satisfying g (x, y) -z ═ 0 and (x, y) ∈ Ψ is the point at the edge of the human body contour of the detection object.

In each of the above examples, the human body contour divides the depth image into two regions, i.e., a human body region and a background region, g (x, y) -z ≦ 0 indicating the human body region, and g (x, y) -z > 0 indicating the background region. By adopting the technical scheme of the embodiment of the application, the human body area and the background area on the millimeter wave image can be clearly distinguished and displayed, and the display effect of the human body outline edge is greatly enhanced.

Corresponding to the millimeter wave image processing method provided by the embodiment of the invention, the embodiment of the invention provides a millimeter wave image processing device.

Fig. 11 is a schematic structural diagram of a millimeter wave image processing apparatus according to an embodiment of the present invention, and as shown in fig. 11, the apparatus may include the following modules:

an image obtaining module 1110, configured to obtain a millimeter wave image and a depth image of a detection object;

a part identification module 1120, configured to identify a to-be-processed part of the human body of the detection object from the depth image; wherein the part of the human body to be treated comprises the contour edge and/or the contour inner area of the human body;

a coefficient determining module 1130, configured to determine enhancement coefficients corresponding to feature points located in the edge and/or the inner region of the contour, respectively;

the image enhancement module 1140 is configured to perform display enhancement processing on the feature points of the to-be-processed part of the human body of the detection object on the millimeter wave image based on the corresponding relationship between the enhancement coefficient and the feature points to obtain a target image.

Based on the scheme provided by the embodiment of the invention, in the process of processing the millimeter wave image of the detection object, the image enhancement of the millimeter wave image of the detection object can be realized by utilizing the depth image of the detection object, so that the image display quality of the outline edge and/or the outline inner area of the detection object in the millimeter wave image is improved, the human body edge display can be clearer, the image identification accuracy is further improved, and the omission ratio and the false detection ratio are reduced.

Optionally, in a specific implementation manner, the coefficient determining module 1130 includes:

the normal determination submodule is used for respectively determining the corresponding normal of the characteristic points of the contour edge and/or the contour internal area on the surface of the human body contour;

the included angle determining submodule is used for determining the included angle between the normal corresponding to each characteristic point and the reference line; the reference line is vertical to the millimeter wave antenna array surface;

the coefficient determining submodule is used for determining respective corresponding enhancement coefficients of the feature points in the contour edge and the contour internal area according to the included angle between the normal corresponding to each feature point and the reference line; wherein the enhancement factor is positively correlated with the included angle.

Optionally, in a specific implementation manner, the coefficient determining submodule is specifically configured to:

calculating the enhancement coefficient omega corresponding to each characteristic point of the contour edge and/or the contour inner region according to the following formula_α：

ω_α＝m-n·φ(θ_α)

Wherein, ω is_αAn enhancement factor, theta, corresponding to an alpha-th feature point at the edge of the contour and/or the inner region of the contour_αIs the included angle between the normal corresponding to the alpha characteristic point at the edge of the contour and/or the inner area of the contour and the reference line, phi (theta)_α) Is an included angle theta corresponding to the alpha-th characteristic point at the edge of the contour and/or the inner area of the contour_αAnd m and n are respectively preset constants of the related functions.

Alternatively, in one embodiment, phi (theta)_α) Including an angle theta corresponding to an alpha-th feature point at an edge and/or an inner region of the contour_αThe associated cosine function, expressed as follows:

wherein the content of the first and second substances,

is a normal corresponding to the alpha-th characteristic point at the edge of the contour and/or the inner area of the contour,

is a reference line.

Optionally, in a specific implementation manner, the image enhancement module 1140 is specifically configured to:

and adjusting the pixel value corresponding to the characteristic point of the part to be processed of the human body on the millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic point to obtain a target image.

Optionally, in a specific implementation manner, the part identification module 1120 further includes:

the smoothing sub-module is used for smoothing the region where the human body is located on the depth image;

and the part identification submodule is used for identifying and obtaining the human body part to be processed of the detection object from the depth image after the smoothing processing.

Optionally, in a specific implementation manner, the smoothing sub-module is specifically configured to:

and identifying a sub-region of the human body on the depth image, wherein the region has wrinkles, and smoothing the sub-region.

Optionally, in a specific implementation manner, the apparatus further includes:

the image registration module is used for carrying out image registration on the millimeter wave image and the depth image to obtain a registered millimeter wave image and a registered depth image;

the part identification module 1120 is specifically configured to:

identifying and obtaining a human body part to be processed of the detection object from the registered depth image;

further, the part identification module 1120 is specifically configured to: constructing a target surface function by using the spatial position information of the feature points on the detection object carried in the registered depth image, and determining a first profile surface to which the feature points of the detection object belong; and identifying and obtaining the human body to-be-processed part of the detection object from the registered depth image based on the target surface function and the first contour surface.

The image enhancement module 1140 is specifically configured to:

and performing display enhancement processing on the characteristic points of the part to be processed of the human body of the detection object on the registered millimeter wave image based on the corresponding relation between the enhancement coefficient and the characteristic points to obtain a target image.

the visible light image acquisition module is used for acquiring a visible light image of the detection object;

the image registration module is specifically configured to: carrying out image registration on the visible light image, the millimeter wave image and the depth image to obtain a registered millimeter wave image, a registered depth image and a registered visible light image;

the site identification module 1120 is further configured to: and determining a second contour surface to which the feature points of the detection object belong by using the registered visible light image, so that the human body to-be-processed part of the detection object is identified and obtained from the registered depth image based on the target surface function, the first contour surface and the second contour surface.

The millimeter wave image processing device and the millimeter wave image processing method provided by the embodiment of the present application belong to an inventive concept, and the contents that are not explained in detail in the embodiment of the device can be referred to the explanations in the embodiment of the method.

Corresponding to the millimeter wave image processing apparatus provided by the above embodiment of the present invention, the embodiment of the present invention further provides an electronic device, as shown in fig. 12, including a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204, where the processor 1201, the communication interface 1202, and the memory 1203 complete mutual communication through the communication bus 1204,

a memory 1203 for storing a computer program;

the processor 1201 is configured to implement the steps of any millimeter wave image processing method provided in the embodiment of the present invention when executing the program stored in the memory 1203.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In still another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above millimeter wave image processing methods.

In yet another embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of any of the millimeter wave image processing methods of the above embodiments.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are described with relative simplicity as they are substantially similar to method embodiments, where relevant only as described in portions of the method embodiments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A millimeter wave image processing method, characterized in that the method comprises:

acquiring a millimeter wave image and a depth image of a detection object;

2. The method according to claim 1, wherein the determining the enhancement coefficients corresponding to the feature points at the edge of the contour and/or the inner region of the contour respectively comprises:

respectively determining the corresponding normal lines of the feature points at the contour edge and/or the contour inner area on the human body contour surface;

determining an included angle between a normal corresponding to each characteristic point and a reference line; the reference line is vertical to the millimeter wave antenna array surface;

determining respective corresponding enhancement coefficients of the feature points in the contour edge and/or the contour internal area according to the included angle between the normal corresponding to each feature point and the reference line; wherein the enhancement factor is positively correlated with the included angle.

3. The method according to claim 2, wherein the determining the enhancement coefficients corresponding to the feature points at the edge and/or the inner region of the contour according to the included angle between the normal corresponding to each feature point and the reference line comprises:

ω_α＝m-n·φ(θ_α)

Wherein, ω is_αAn enhancement factor, theta, corresponding to an alpha-th feature point at the edge of the contour and/or the inner region of the contour_αIs the included angle between the normal corresponding to the alpha characteristic point at the edge of the contour and/or the inner area of the contour and the reference line, phi (theta)_α) Is an included angle theta corresponding to the alpha-th characteristic point at the edge and/or the inner area of the contour_αAnd m and n are respectively preset constants of the related functions.

4. A method according to claim 3, characterized in that (θ)_α) Including an angle theta corresponding to an alpha-th feature point at an edge and/or an inner region of the contour_αThe associated cosine function, expressed as follows:

wherein the content of the first and second substances,

is the reference line.

5. The method according to claim 1, wherein the performing of display enhancement processing on the feature points of the to-be-processed part of the human body on the millimeter wave image based on the corresponding relationship between the enhancement coefficient and the feature points to obtain a target image comprises:

6. The method according to claim 1, wherein the identifying the human body part to be processed of the detection object from the depth image comprises:

and smoothing the region of the human body on the depth image, and identifying and obtaining the part to be processed of the human body of the detection object from the depth image after smoothing.

7. The method according to claim 6, wherein the smoothing of the region of the depth image where the human body is located comprises:

8. The method according to claim 1, wherein before said identifying a human body part to be processed of the detection object from the depth image, the method further comprises:

carrying out image registration on the millimeter wave image and the depth image to obtain a registered millimeter wave image and a registered depth image;

the identifying of the human body to-be-processed part of the detection object from the depth image comprises the following steps:

constructing a target surface function by using the spatial position information of the feature points on the detection object carried in the registered depth image, and determining a first contour surface to which the feature points of the detection object belong;

identifying and obtaining a human body to-be-processed part of the detection object from the registered depth image based on the target surface function and the first contour surface;

the displaying and enhancing processing of the feature points of the to-be-processed part of the human body of the detection object on the millimeter wave image based on the corresponding relationship between the enhancement coefficient and the feature points to obtain a target image comprises the following steps:

9. The method of claim 8, wherein prior to said image registering said millimetre wave image and said depth image resulting in a registered millimetre wave image and a registered depth image, said method further comprises:

acquiring a visible light image of the detection object;

the image registration of the millimeter wave image and the depth image to obtain the registered millimeter wave image and the registered depth image includes:

carrying out image registration on the visible light image, the millimeter wave image and the depth image to obtain a registered millimeter wave image, a registered depth image and a registered visible light image;

the identifying of the human body to-be-processed part of the detection object from the depth image further comprises:

and determining a second contour surface to which the characteristic points of the detection object belong by using the registered visible light image, so that the human body to-be-processed part of the detection object is identified and obtained from the registered depth image based on the target curved surface function, the first contour surface and the second contour surface.

10. A millimeter-wave image processing apparatus, characterized in that the apparatus comprises:

the part identification module is used for identifying and obtaining a part to be processed of the human body of the detection object from the depth image; wherein the part of the human body to be treated comprises the contour edge and/or the contour inner area of the human body;

11. The apparatus of claim 10, wherein the coefficient determination module comprises:

the included angle determining submodule is used for determining an included angle between the normal line corresponding to each characteristic point and the reference line; the reference line is vertical to the millimeter wave antenna array surface;

the coefficient determining submodule is used for determining respective corresponding enhancement coefficients of the feature points in the contour edge and the contour internal area according to the included angle between the normal corresponding to each feature point and the reference line; wherein the enhancement factor is positively correlated with the included angle;

the coefficient determination submodule is specifically configured to: calculating the enhancement coefficient omega corresponding to each characteristic point of the contour edge and/or the contour inner region according to the following formula_α：

ω_α＝m-n·φ(θ_α)

Wherein, ω is_αAn enhancement factor, theta, corresponding to an alpha-th feature point at the edge of the contour and/or the inner region of the contour_αIs the included angle between the normal corresponding to the alpha characteristic point at the edge of the contour and/or the inner area of the contour and the reference line, phi (theta)_α) Is an included angle theta corresponding to the alpha-th characteristic point at the edge and/or the inner area of the contour_αRelated functions, m and n are respectively preset constants;

φ(θ_α) Including an angle theta corresponding to an alpha-th feature point at an edge and/or an inner region of the contour_αThe associated cosine function, expressed as follows:

wherein the content of the first and second substances,

is the reference line;

and/or the presence of a gas in the gas,

the image enhancement module is specifically configured to: based on the corresponding relation between the enhancement coefficient and the characteristic point, adjusting the pixel value corresponding to the characteristic point of the part to be processed of the human body on the millimeter wave image to obtain a target image;

and/or the presence of a gas in the gas,

the part recognition module further includes:

the smoothing sub-module is used for smoothing the region of the human body on the depth image;

the part identification submodule is used for identifying and obtaining a human body part to be processed of the detection object from the depth image after the smoothing processing;

the smoothing sub-module is specifically configured to: identifying a sub-region of the depth image where the human body is located and with wrinkles, and smoothing the sub-region;

and/or the presence of a gas in the gas,

the device further comprises: the image registration module is used for carrying out image registration on the millimeter wave image and the depth image to obtain a registered millimeter wave image and a registered depth image;

the part identification module is specifically configured to: constructing a target surface function by using the space position information of the characteristic points on the detection object carried in the registered depth image, and determining a first profile surface to which the characteristic points of the detection object belong; identifying and obtaining a human body to-be-processed part of the detection object from the registered depth image based on the target surface function and the first contour surface;

the image enhancement module is specifically configured to: performing display enhancement processing on the feature points of the to-be-processed part of the human body of the detection object on the registered millimeter wave image based on the corresponding relation between the enhancement coefficient and the feature points to obtain a target image;

and/or the presence of a gas in the gas,

the device further comprises: the visible light image acquisition module is used for acquiring a visible light image of the detection object;

the site recognition module is further configured to: and determining a second contour surface to which the feature points of the detection object belong by using the registered visible light image, so that the human body to-be-processed part of the detection object is identified and obtained from the registered depth image based on the target surface function, the first contour surface and the second contour surface.

12. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-9 when executing a program stored in the memory.

13. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-9.