WO2023179011A1 - Image generation method and device - Google Patents
Image generation method and device Download PDFInfo
- Publication number
- WO2023179011A1 WO2023179011A1 PCT/CN2022/127355 CN2022127355W WO2023179011A1 WO 2023179011 A1 WO2023179011 A1 WO 2023179011A1 CN 2022127355 W CN2022127355 W CN 2022127355W WO 2023179011 A1 WO2023179011 A1 WO 2023179011A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- preset
- projection
- dimensional point
- dimensional
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 89
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims description 50
- 238000007619 statistical method Methods 0.000 claims description 24
- 230000009466 transformation Effects 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 19
- 238000007499 fusion processing Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 230000004927 fusion Effects 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 19
- 238000010586 diagram Methods 0.000 description 21
- 230000006870 function Effects 0.000 description 17
- 238000007689 inspection Methods 0.000 description 15
- 239000006185 dispersion Substances 0.000 description 7
- 238000006243 chemical reaction Methods 0.000 description 5
- 210000002414 leg Anatomy 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/06—Topological mapping of higher dimensional structures onto lower dimensional surfaces
- G06T3/067—Reshaping or unfolding 3D tree structures onto 2D planes
Definitions
- the present application relates to the field of image processing technology, and in particular to an image generation method and device.
- three-dimensional scanning equipment such as lidar, millimeter-wave radar, or microwave radar can be used to obtain information on several points in space, including coordinates and corresponding amplitudes, to obtain a three-dimensional point cloud map.
- the amplitude of each three-dimensional point in the three-dimensional point cloud image is determined based on the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point in space.
- the data volume of the 3D point cloud image is large, and directly performing business processing (such as target detection, target recognition, etc.) on the 3D point cloud image is highly complex and costly. Therefore, a method is urgently needed to generate a two-dimensional image corresponding to the three-dimensional point cloud image, and then business processing can be performed based on the two-dimensional image.
- an embodiment of the present application discloses an image generation method, which method includes:
- the three-dimensional point Determine the statistical characteristics corresponding to the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional point based on the statistical characteristics; wherein, the three-dimensional point
- the amplitude of is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
- the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane are determined; wherein the preset straight line direction is the same as the preset straight line direction.
- the default projection plane is vertical;
- a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
- an image generation device which includes:
- the three-dimensional point cloud image acquisition module is used to obtain the three-dimensional point cloud image
- the image feature acquisition module is used to determine the statistical features corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image features of the preset three-dimensional points based on the statistical features.
- the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
- the projection feature acquisition module is used to determine the projection features of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image features of the preset three-dimensional point in the preset straight line direction; wherein, the The preset straight line direction is perpendicular to the preset projection plane;
- a two-dimensional projection image acquisition module is used to determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
- an electronic device in another aspect of the application, includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory pass through The above communication bus completes mutual communication;
- the memory is used to store computer programs
- the processor is configured to implement any of the above image generation methods when executing a program stored on the memory.
- a non-transitory computer-readable storage medium is also provided.
- a computer program is stored in the non-transitory computer-readable storage medium.
- the computer program is executed by a processor, the following is implemented: Any of the image generation methods described above.
- a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any one of the above image generation methods.
- the image generation method provided by the embodiment of the present application obtains a three-dimensional point cloud image; determines the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determines the amplitude of the preset three-dimensional point based on the statistical characteristics.
- Image features among them, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point; according to the preset straight line direction
- the image characteristics of the preset three-dimensional point determine the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane; the preset straight line direction is perpendicular to the preset projection plane; according to the projection characteristics on the preset projection plane , determine the two-dimensional projection of the three-dimensional point cloud image on the preset projection plane.
- the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the image characteristics of the preset three-dimensional points can be determined, and based on the preset three-dimensional points. Assume the image characteristics of the preset three-dimensional points in the straight line direction and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated. Moreover, different statistical features can reflect different image features.
- Figure 1 is a flow chart of an image generation method provided by an embodiment of the present application.
- Figure 2 is a schematic diagram of a three-dimensional space provided by an embodiment of the present application.
- Figure 3 is a schematic diagram of another three-dimensional space provided by an embodiment of the present application.
- Figure 4 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 5 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 6 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 7a is a first projection sub-image provided by an embodiment of the present application.
- Figure 7b is an enlarged view of the partial image area of the first projection sub-image shown in Figure 7a;
- Figure 8a is a second projection sub-image provided by the embodiment of the present application.
- Figure 8b is an enlarged view of the partial image area of the second projection sub-image shown in Figure 8a;
- Figure 9a is a third projection sub-image provided by the embodiment of the present application.
- Figure 9b is an enlarged view of the partial image area of the third projection sub-image shown in Figure 9a;
- Figure 10 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 11 is a schematic diagram of the principle of an image generation method provided by an embodiment of the present application.
- Figure 12 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 13 is a flow chart of another image generation method provided by an embodiment of the present application.
- Figure 14 is a comparison diagram of a two-dimensional projection image provided by an embodiment of the present application.
- Figure 15 is a comparison of enlarged views of a partial image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
- Figure 16 is a comparison diagram of another enlarged view of the local image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
- Figure 17 is a comparison diagram of another enlarged view of the local image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
- Figure 18 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application.
- Figure 19 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application.
- Figure 20 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application.
- Figure 21 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application.
- Figure 22 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application.
- Figure 23 is a structural diagram of an image generation device provided by an embodiment of the present application.
- Figure 24 is a structural diagram of an electronic device provided by an embodiment of the present application.
- Embodiments of the present application provide an image generation method, which can be applied to electronic devices.
- the electronic device can process the obtained three-dimensional point cloud image based on the method provided by the embodiments of the present application to obtain a corresponding two-dimensional image.
- image recognition can be performed on the two-dimensional image.
- the category of the object for example, a person, an object, etc.
- the category to which the object in the two-dimensional image belongs can also be identified.
- the image area or it can also identify whether the specified object exists in the two-dimensional image, but is not limited to this.
- Figure 1 is a flow chart of an image generation method provided by an embodiment of the present application. The method may include the following steps:
- S102 Determine the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional points based on the statistical characteristics.
- the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point
- the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point.
- the size and shape of the preset point cloud area can be determined according to actual needs.
- the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the above neighborhood refer to the characteristic quantities obtained by statistically processing each amplitude using statistical rules. Compared with the situation where only the amplitude of the preset three-dimensional point itself is considered, the use of statistical features helps to better characterize the corresponding amplitude characteristics of the preset three-dimensional point as a whole, thereby obtaining high-quality two-dimensional projections in the future.
- the obtained statistical features can be directly determined as image features of preset three-dimensional points, or the obtained statistical features can be processed, for example, according to preset operation rules (which can be determined according to different processing requirements).
- the features are operated or transformed, and the processing results are determined as image features of preset three-dimensional points.
- image features may refer to features used to characterize the display characteristics or display effects of the display image corresponding to the three-dimensional point.
- S103 Determine the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image characteristics of the preset three-dimensional point in the preset straight line direction.
- the preset straight line direction is perpendicular to the preset projection plane. If there are multiple preset three-dimensional points in each preset straight line direction, then there are multiple corresponding image features. Then, based on these multiple image features, the projection features corresponding to the preset straight line direction on the preset projection plane can be determined. .
- Projection features are the direct basis for forming a two-dimensional projection image. The selection of projection features is related to the quality of the two-dimensional projection image. In other words, in the embodiment of the present application, the projection features may refer to display features used to characterize the two-dimensional projection image. or the characteristics of the display effect. For example, the pixel value of the preset three-dimensional point corresponding to the projection feature in the three-dimensional point cloud map can be used to determine the pixel value at the position corresponding to the preset three-dimensional point on the two-dimensional projection map.
- S104 Determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
- the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the preset three-dimensional points can be determined.
- Image characteristics, and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the image characteristics of the preset three-dimensional point in the preset straight line direction, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated.
- different statistical features can reflect different image features.
- the electronic device can obtain the three-dimensional point cloud image through the radar device.
- the three-dimensional point cloud image in the embodiment of this application may be a SAR (Synthetic Aperture Radar) image.
- Radar equipment can be lidar, millimeter wave radar, microwave radar, etc.
- step S101 may include the following steps: obtaining a three-dimensional point cloud image of the scanned object.
- the modulated signal can be emitted through the millimeter wave radar in the active millimeter wave security inspection instrument (the modulated signal is a millimeter wave signal), and the modulated signal is scanned according to the imaging algorithm (for example, The echo signals reflected by people, objects, etc.) are processed to obtain a three-dimensional point cloud image of the scanned object. Then, the electronic device can generate a corresponding two-dimensional image based on the obtained three-dimensional point cloud image based on the method provided by the embodiment of the present application. Subsequently, image recognition can be performed on the two-dimensional images to detect, identify and locate dangerous items.
- the imaging algorithm for example, The echo signals reflected by people, objects, etc.
- the echo signal reflected by the millimeter wave signal at the scanning object is evenly sampled to generate a corresponding three-dimensional point cloud image. Therefore, the obtained three-dimensional point cloud image contains multiple three-dimensional points evenly distributed in the three-dimensional space. The interval between each three-dimensional point is determined based on the sampling interval when the three-dimensional point cloud image is generated.
- the amplitude of a three-dimensional point in the three-dimensional point cloud image represents The electromagnetic scattering characteristics at the corresponding position of the three-dimensional point in space.
- the respective amplitudes of each three-dimensional point in the three-dimensional point cloud image can be represented by a three-dimensional matrix.
- the projection plane where the two-dimensional projection image to be generated is located ie, the preset projection plane in the embodiment of the present application
- the three-dimensional point cloud image can be projected to the preset projection plane,
- the corresponding two-dimensional projection map can be obtained.
- the preset projection plane may include: a projection plane parallel to the front and/or back of the scanned object.
- the scanning object is a person
- the preset projection plane can be a projection plane parallel to the front and/or back of the person
- the front of the person is the plane toward which the person's face faces.
- the two-dimensional projection of the three-dimensional point cloud image on the preset projection plane contains the complete image information of the person and the image information of the items carried by the person, which can facilitate the security inspection of the person.
- the character carries a wrench.
- the generated two-dimensional projection image contains complete image information of the wrench. Based on the two-dimensional projection image, the wrench can be detected, identified and positioned. .
- the preset projection plane includes projection planes parallel to the front and back of the character
- multiple two-dimensional projection images in different directions can be obtained, and the image information of the character can be enriched from different directions and combined with the image information in different directions.
- Recognition can improve the accuracy of image recognition.
- a two-dimensional image under a preset projection perspective can be generated based on the three-dimensional point cloud image.
- a two-dimensional image of the scanned object under the main perspective (that is, the main view of the scanned object) can be generated based on the three-dimensional point cloud image.
- a two-dimensional image of the scanned object from a side view (ie, a side view of the scanned object) can also be generated based on the three-dimensional point cloud image.
- the electronic device can determine that the projection plane under the preset projection perspective is the preset projection plane, and determine whether the original three-dimensional point cloud image obtained by the radar device matches the preset projection perspective. If the original three-dimensional point cloud image matches the preset projection perspective, for example, the preset projection plane is parallel to a coordinate plane in the three-dimensional coordinate system of the original three-dimensional point cloud image, the electronic device can directly calculate the preset three-dimensional angle in the original three-dimensional point cloud image. The statistical characteristics corresponding to the amplitude of three-dimensional points in the neighborhood of the point.
- FIG. 2 is a schematic diagram of a three-dimensional space provided by an embodiment of the present application.
- the scanning object is a person.
- the three-dimensional range shown by the cuboid in Figure 2 represents the original three-dimensional point cloud image of the person.
- the coordinate planes of the three-dimensional coordinate system shown in Figure 2 include: XOY plane, XOZ plane and YOZ plane.
- the preset projection angle is the main view angle
- the preset projection plane under the main view angle is a projection plane parallel to the front of the character
- the preset projection plane and the three-dimensional coordinates The XOY plane in the system is parallel, that is, the original three-dimensional point cloud image matches the preset projection perspective.
- the electronic device obtains the original three-dimensional point cloud image shown in Figure 2, it can directly calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image that are in the neighborhood of the preset three-dimensional points.
- step S101 may include the following steps: rotating the original three-dimensional point cloud image based on a preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective.
- the preset projection plane is the projection plane under the preset projection angle.
- the preset projection plane is not parallel to all coordinate planes in the three-dimensional coordinate system of the three-dimensional point cloud image, in order to generate the three-dimensional point cloud image, in the preset projection A two-dimensional projection image under a viewing angle.
- the electronic device can rotate the original three-dimensional point cloud image based on the positional relationship between the preset projection angle of view and the original three-dimensional point cloud image to obtain the same as the preset projection angle of view. Matching 3D point cloud images.
- the electronic device can perform coordinate conversion on the original three-dimensional point cloud image based on the two-dimensional coordinate system of the preset projection plane and the coordinate mapping relationship between the two-dimensional coordinate system of the preset projection plane and the three-dimensional coordinate system of the original three-dimensional point cloud image, so as to After coordinate conversion, one coordinate plane in the three-dimensional coordinate system of the obtained three-dimensional point cloud image is parallel to the preset projection plane, and the three-dimensional point cloud image obtained by the coordinate conversion is a three-dimensional point cloud image that matches the preset projection perspective.
- the electronic device can then calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image.
- FIG. 3 is a schematic diagram of another three-dimensional space provided by an embodiment of the present application.
- the scanning object is a person.
- the three-dimensional range shown by the cuboid in Figure 3 represents the original three-dimensional point cloud image of the person.
- the coordinate planes of the three-dimensional coordinate system shown in Figure 3 include: XOY plane, XOZ plane and YOZ plane.
- the default projection view is the side view
- the preset projection plane in the side view is the X 1 O 1 Y 1 plane, X 1 O
- the 1 Y 1 plane is not parallel to any coordinate plane in the three-dimensional coordinate system.
- the electronic device can be based on the two-dimensional coordinate system of the X 1 O 1 Y 1 plane and the coordinate mapping relationship between the two-dimensional coordinate system of the X 1 O 1 Y 1 plane and the three-dimensional coordinate system of the original three-dimensional point cloud image.
- the original three-dimensional point cloud image undergoes coordinate transformation so that the XOY plane in the three-dimensional coordinate system of the three-dimensional point cloud image is parallel to the X 1 O 1 Y 1 plane after the coordinate transformation.
- the electronic device can then calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image.
- the preset three-dimensional point can be any three-dimensional point in the three-dimensional point cloud image.
- the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point.
- Preset point cloud areas can be determined based on requirements.
- the preset point cloud area may be a one-dimensional area, for example, in a three-dimensional point cloud image, each three-dimensional point is located on a straight line perpendicular to the preset projection plane and including the preset three-dimensional point.
- the preset point cloud area can also be a two-dimensional area.
- each three-dimensional point in the three-dimensional point cloud image is located in a plane area parallel to the preset projection plane and containing the preset three-dimensional point.
- the shape of the plane area can be a rectangle, Round etc.
- the preset point cloud area may also be a three-dimensional area, for example, each three-dimensional point in a cuboid or sphere containing the preset three-dimensional points in the three-dimensional point cloud image, which is not specifically limited in this embodiment.
- Different image features of three-dimensional points can be determined based on different statistical features.
- different feature statistical methods can be selected based on different needs.
- the image features of the preset three-dimensional point may include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining smooth areas of the image, and a second image feature used to participate in characterizing image details.
- the third image feature of image background noise may include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining smooth areas of the image, and a second image feature used to participate in characterizing image details.
- the electronic device Based on the first image feature, the second image feature, and the third image feature, the electronic device generates a two-dimensional projection image that can reflect image details and image smooth areas, and can reduce image background noise, and can improve the generated two-dimensional projection image.
- the quality of the projection image to improve the accuracy of identifying scanned objects in the image.
- the two-dimensional projection image when the two-dimensional projection image is applied to a security inspection scene, the two-dimensional projection image can reflect the details of different scanned objects.
- the obtained two-dimensional projection image can reflect the image area where the person is located, and then security inspection based on the two-dimensional projection image can accurately distinguish the persons in the two-dimensional projection image and the items carried by the person. , to determine whether the items carried by the person are dangerous goods, which can improve the accuracy of dangerous goods identification.
- step S102 may include at least one of the following operations, where Figure 4 is an example, showing all the operations included in step S102:
- S1021 Determine the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among variance calculation, gradient calculation, and Laplacian operator. Statistical features, as the first image features of preset three-dimensional points.
- the plane where the first neighborhood is located is parallel to the preset projection plane.
- S1022 Determine the amplitude correspondence of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among entropy value calculation, integrated side-lobe ratio calculation, and peak side-lobe ratio calculation.
- the second statistical feature is used as the second image feature of the preset three-dimensional point.
- S1023 Determine the third value corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through variance calculation, gradient calculation, or at least one characteristic statistical method of the Laplacian operator. Statistical features are used as third image features of preset three-dimensional points.
- the plane of the second neighborhood is perpendicular to the preset projection plane.
- the statistical characteristics of the preset three-dimensional points in different dimensions can be obtained based on different feature statistics methods, and the image characteristics of the preset three-dimensional points in different dimensions can be determined based on the statistical characteristics of different dimensions.
- the electronic device in order to make the generated two-dimensional projection map reflect image features of different dimensions and facilitate subsequent image recognition of the two-dimensional projection map, the electronic device can use a variety of feature statistics methods to calculate the number of points in the neighborhood of the preset three-dimensional point. Statistical characteristics corresponding to the amplitude of three-dimensional points. Then, the image characteristics of the preset three-dimensional points can be determined based on the obtained statistical characteristics.
- At least one characteristic statistical method such as variance calculation, gradient calculation, and Laplacian operator to determine the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image (i.e., the first statistical characteristics ).
- characteristic statistical method such as variance calculation, gradient calculation, and Laplacian operator
- gradient calculation and Laplacian operator can be used to extract the edge features of the scanned object, and the variance value obtained by variance calculation can reflect the dispersion of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points.
- the dispersion represents The difference in amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional point.
- the first statistical feature can reflect the edge features of the scanned object corresponding to the three-dimensional point cloud image and the image details of different areas in the three-dimensional point cloud image.
- Using the first image feature that participates in characterizing image details as an evaluation index in the process of determining the two-dimensional projection map can avoid the loss of details in the image caused by background clutter covering dark areas in strongly scattering targets.
- three-dimensional point cloud images can be obtained through active millimeter wave security inspection devices.
- the target area with strong electromagnetic scattering contains fewer three-dimensional points in the distance direction, and the distance direction is the direction perpendicular to the preset projection plane. That is, only the three-dimensional points with a smaller distance upward contain the focus information of the target area.
- the remaining three-dimensional points with a distance upward correspond to other areas, for example, background areas or transmission areas.
- the amplitudes of the corresponding three-dimensional points in other areas in the three-dimensional point cloud map are lower than the amplitudes of the corresponding three-dimensional points in the three-dimensional point cloud map of the target area. That is, the target area with strong electromagnetic scattering has a larger amplitude of the corresponding three-dimensional point in the three-dimensional point cloud diagram. Therefore, the dispersion of the three-dimensional points in the target area in the three-dimensional point cloud diagram is also larger.
- At least one characteristic statistical method among variance calculation, gradient calculation and Laplacian operator can be used. , calculate the first statistical feature corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, as the first image feature of the preset three-dimensional point cloud image. Since the first image feature is used to participate in representing image details, the two-dimensional projection map obtained based on the first image feature can reflect the details of the scanned object, that is, the quality of the generated two-dimensional projection map can be improved, and subsequent processing of the two-dimensional projection map can be facilitated.
- Image Identification the first image feature corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image
- the statistical characteristics corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image are determined ( That is, the second statistical characteristic), which can reflect the degree of order and focus of the amplitudes of the three-dimensional points in the first neighborhood of the preset three-dimensional points.
- the degree of order represents the amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional points.
- the changing trend for example, smooth increase or smooth decrease, etc.
- the degree of focus represents the degree of concentration of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional point.
- the amplitudes of the three-dimensional points corresponding to the smooth areas in the scanned object have a high degree of concentration and a smooth changing trend. Therefore, the second statistical feature can reflect the smoothness of different areas in the scanned object corresponding to the three-dimensional point cloud image. .
- the scanning object can be a person. Since the surface of the human body is relatively smooth, the amplitude of the corresponding three-dimensional points on the human body surface in the three-dimensional point cloud image has a high degree of order. Therefore, in order to make the generated two-dimensional projection image reflect the smooth area of the scanned object, at least one of the characteristic statistical methods of entropy value calculation, integral side-lobe ratio calculation, and peak side-lobe ratio calculation can be used to calculate the three-dimensional point cloud image.
- the second statistical feature corresponding to the amplitude of the three-dimensional point located in the first neighborhood of the preset three-dimensional point is used as the second image feature of the preset three-dimensional point.
- the two-dimensional projection map obtained based on the second image feature can reflect the smooth area of the scanned object, that is, it can improve the quality of the generated two-dimensional projection map and facilitate subsequent two-dimensional Projection images for image recognition.
- the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can reflect the dispersion of the amplitudes of the three-dimensional points in the second neighborhood of the preset three-dimensional point, and the dispersion represents the difference in the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points.
- the amplitude of each three-dimensional point in the background area in the three-dimensional point cloud image is greatly different from the amplitude of each three-dimensional point in the target area containing the scanned object in the three-dimensional point cloud image. Therefore, the third statistical feature can be used to distinguish the background area and the target area containing the scanned object.
- the target area with strong electromagnetic scattering in the 3D point cloud image obtained by the active millimeter wave security inspection instrument contains fewer 3D points in the distance direction, and the amplitude of the contained 3D points changes greatly.
- the background region has smaller changes in the magnitude of the three-dimensional points it contains in the distance direction. Therefore, the difference in amplitude of the three-dimensional points in the target area and the background area in the three-dimensional point cloud image is large.
- At least one feature statistics method in the method calculates the third statistical feature corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image as the third image feature of the preset three-dimensional point. Since the third image feature is used to participate in characterizing the image background noise, the two-dimensional projection map obtained based on the third image feature can reduce the image background noise, and can also distinguish the background area from the target area containing the scanned object, that is, the generated image can be improved. The quality of the two-dimensional projection image facilitates image recognition of the two-dimensional projection image.
- the electronic device can calculate the power sum of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image based on the power function, as the image feature of the preset three-dimensional point.
- the obtained image features can characterize the difference between the amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional point and the average amplitude of the image background noise, thereby retaining image details in the generated two-dimensional projection map.
- step S103 may include at least one of the following operations, where Figure 5 is an example, showing all the operations included in step S103:
- S1031 Determine the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
- S1032 Determine the feature with the largest value among the second image features of the preset three-dimensional point in the preset straight line direction as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
- S1033 Determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
- the above-mentioned preset straight line direction is perpendicular to the preset projection plane, and the plane of the second neighborhood considered in the process of determining the third image characteristics is perpendicular to the preset projection plane. Therefore, when the second neighborhood is a one-dimensional neighborhood, for the same For a preset three-dimensional point, its second neighborhood is parallel to or coincides with the preset straight line. For a completely coincident situation (each preset three-dimensional point in the second neighborhood is each preset three-dimensional point on the preset straight line). ), the calculated statistical feature or third image feature is the same value.
- the third statistical features corresponding to the amplitudes of the three-dimensional points in the second neighborhood of each preset three-dimensional point are the same, and correspondingly, the third image features of each preset three-dimensional point are also the same. Therefore, there is no need to determine the maximum value in the third image feature, and the third image feature of the preset three-dimensional point can be directly determined as the third projection feature.
- the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane can be determined. That is, in an optional embodiment, determining the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the projection characteristics on the preset projection plane includes: based on the first projection characteristics on the preset projection plane, At least one of the second projection feature and the third projection feature determines the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane. Specifically, the projection characteristics involved in forming the two-dimensional projection image can be determined based on the generation requirements for the image characteristics or image effects of the two-dimensional projection image.
- step S104 may include the following steps:
- S1041 Based on the first projection feature, the second projection feature, and the third projection feature on the preset projection plane, determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
- the feature with the largest value among the first image features may be the feature corresponding to the maximum amplitude dispersion.
- the dispersion reflects the difference between the amplitudes of the three-dimensional points, and the edges of the scanned objects are adjacent to them. The difference in amplitude of the position is large. Therefore, the preset three-dimensional point corresponding to the feature with the largest value among the first image features is also the three-dimensional point corresponding to the edge of the scanned object. Furthermore, based on the first image feature, the edge of the scanned object in the three-dimensional point cloud image can be effectively determined to enrich the image details of the generated two-dimensional projection image.
- the electronic device can determine the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the feature in the preset straight line direction.
- the first projection feature of the preset three-dimensional point on the preset projection plane can determine the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the feature in the preset straight line direction.
- the feature with the largest value among the second image features may be the feature corresponding to the maximum amplitude order degree.
- the order degree can reflect the smoothness of different areas in the scanned object corresponding to the three-dimensional point cloud image.
- the area where the preset three-dimensional point corresponding to the image feature is located is the image area with the greatest smoothness.
- the electronic device can place the preset three-dimensional point in the preset straight line direction.
- the feature with the largest value among the second image features is determined as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
- the third image feature of the preset three-dimensional point is used to participate in determining the image background noise.
- the generated two-dimensional projection map can distinguish the background area from the target area containing the scanned object.
- the electronic device can determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
- the electronic device can use different methods to generate a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the first projection feature, the second projection feature, and the third projection feature on the preset projection plane.
- the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined based on at least one of the first projection feature, the second projection feature, and the third projection feature on the preset projection plane.
- Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- the projection characteristics participating in the formation of the two-dimensional projection image can be determined, and then based on the determined projection characteristics, it is determined that the projection sub-image needs to be generated, and finally through image fusion processing Generate a two-dimensional projection of the requirements.
- step S1041 may include the following steps:
- S10411 Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate the first projection sub-image based on the first pixel value.
- S10412 Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value.
- S10413 Use the third projection feature on the preset projection plane as the third pixel value, and generate the third projection sub-image based on the third pixel value.
- S10414 Fusion process the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- a preset straight line corresponds to a pixel point in the preset projection plane.
- the preset three-dimensional point on the preset straight line is reversely determined according to the projection characteristics.
- the pixel value corresponding to the preset three-dimensional point is also the pixel value in the preset projection plane.
- the pixel value of this pixel corresponds to determining the pixel values corresponding to the preset three-dimensional points in each preset straight line direction, that is, determining the pixel values of each pixel point in the preset projection plane, the corresponding projection sub-image can be obtained.
- the pixel value corresponding to a preset three-dimensional point is the amplitude of the preset three-dimensional point in the three-dimensional point cloud image.
- the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character
- the preset projection plane is the XOY plane.
- the default straight line direction is perpendicular to the XOY plane. It is preset that the plane where the first neighborhood of the three-dimensional point is located is parallel to the XOY plane.
- the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z). Since each three-dimensional point is located in the preset straight line direction perpendicular to the XOY plane, the x and y of each three-dimensional point are the same.
- the preset three-dimensional point corresponding to the first projection feature can be recorded as (x, y, z 1 ).
- the first pixel value of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image is: the amplitude of the preset three-dimensional point in the three-dimensional point cloud image, then the first pixel value corresponding to the preset three-dimensional point corresponding to the first projection feature
- a pixel value ⁇ 1 can be expressed as:
- ⁇ 1 ⁇ (x,y,z 1 ) (1)
- ⁇ (x,y,z 1 ) represents the amplitude of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image.
- Figure 7a is a first projection sub-image provided by an embodiment of the present application.
- the electronic device may generate a first projection sub-image as shown in Figure 7a based on the first projection feature.
- the image on the left side in Figure 7b is an enlarged view of the background area (areas other than the image area where the character is located) in the first projection sub-image shown in Figure 7a.
- the middle image of Figure 7b is an enlarged view of the area where the character's back is located in the first projection sub-image shown in Figure 7a.
- the image on the right side in Figure 7b is an enlarged view of the area where the character's legs are located in the first projection sub-image shown in Figure 7a.
- both Figure 7a and Figure 7b contain more image details.
- the image in the middle of Figure 7b can reflect the different positions of the character's back area, and the image on the right side of Figure 7b can reflect that a wrench is placed on the character's leg.
- the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character
- the preset projection plane is the XOY plane.
- the default straight line direction is perpendicular to the XOY plane. It is preset that the plane where the first neighborhood of the three-dimensional point is located is parallel to the XOY plane.
- the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z). Since each three-dimensional point is located in the preset straight line direction perpendicular to the XOY plane, the x and y of each three-dimensional point are the same.
- the preset three-dimensional point corresponding to the second projection feature can be recorded as (x, y, z 2 ).
- the second pixel value of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is: the amplitude of the preset three-dimensional point in the three-dimensional point cloud image, then the second pixel value corresponding to the preset three-dimensional point corresponding to the second projection feature
- the two-pixel value ⁇ 2 can be expressed as:
- ⁇ (x,y,z 2 ) represents the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image.
- Figure 8a is a second projection sub-image provided by an embodiment of the present application.
- the electronic device may generate the second projection sub-image shown in FIG. 8a based on the second projection feature.
- the image on the left in Figure 8b is an enlarged view of the background area in the second projected sub-image shown in Figure 8a.
- the middle image of Figure 8b is an enlarged view of the area where the character's back is located in the second projection sub-image shown in Figure 8a.
- the image on the right side in Figure 8b is an enlarged view of the area where the character's legs are located in the second projection sub-image shown in Figure 8a.
- both Figure 8a and Figure 8b can reflect the smooth area of the image, that is, they can reflect the smoothness of the scanned object.
- the area containing the scanned object in Figure 8a has a different brightness than the background area.
- the brightness difference at each position is small.
- the brightness of the area where the character's legs are located and the area where the wrench carried by the character is located are different.
- the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character
- the preset projection plane is the XOY plane.
- the default straight line direction is perpendicular to the XOY plane.
- the plane where the second neighborhood of the preset three-dimensional point is located is perpendicular to the XOY plane.
- the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z).
- the third pixel value ⁇ 3 corresponding to the preset three-dimensional point corresponding to the third projection feature can be expressed as:
- f() represents the objective function, which is used to calculate the third image feature of the preset three-dimensional point.
- ⁇ (x,y,z) represents the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image.
- Figure 9a is a third projection sub-image provided by an embodiment of the present application.
- the electronic device may generate the third projection sub-image shown in FIG. 9a based on the third projection feature.
- the image on the left in Figure 9b is an enlarged view of the background area in the third projection sub-image shown in Figure 9a.
- the middle image of Figure 9b is an enlarged view of the area where the character's back is located in the third projection sub-image shown in Figure 9a.
- the image on the right side in Figure 9b is an enlarged view of the area where the character's legs are located in the third projection sub-image shown in Figure 9a.
- Figures 9a and 9b can clearly show the boundary between the target area where the scanned object is located and the background area, as well as the complete outline of the scanned object, and there is less noise in the background area.
- the electronic device can fuse at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image. After processing, the two-dimensional projection image corresponding to the three-dimensional point cloud image is obtained.
- the electronic device will be in a preset three-dimensional direction in the preset straight line direction.
- the feature with the largest value among the first image features of the point is determined as the first projection feature, and the amplitude of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image is used as the pixel value to obtain the first projection sub-image.
- the feature with the largest value in the second image feature of the preset three-dimensional point in the preset straight line direction is determined as the second projection feature, and the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is determined As pixel values, the second projected sub-image is obtained.
- the feature with the largest median value of the third image feature of the preset three-dimensional point in the preset straight line direction is determined as the third projection feature, and the amplitude of the preset three-dimensional point corresponding to the third projection feature in the three-dimensional point cloud image is determined As pixel values, the third projected sub-image is obtained.
- the first projection sub-image, the second projection sub-image and the third projection sub-image are all data magnitudes corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image.
- the electronic device can directly perform fusion processing on the first projection sub-image, the second projection sub-image and the third projection sub-image based on the preset fusion algorithm to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- the electronic device can perform fusion processing on each projection sub-image according to the following formula.
- H(x,y) g(h 1 (x, y), h 2 (x, y), h 3 (x, y)) (4)
- H(x,y) represents the pixel value of the pixel point with coordinates (x,y) in the two-dimensional projection image.
- g() represents the fusion function.
- h 1 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the first projection sub-image.
- h 2 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the second projection sub-image.
- h 3 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the third projection sub-image.
- the electronic device can perform fusion processing on two projection sub-images with the same data magnitude among the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain an intermediate image.
- the electronic device can pre-process (for example, normalize) another projection sub-image, and fuse the intermediate image with the pre-processing result to obtain a corresponding two-dimensional projection image.
- the electronic device determines the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature, and adds the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image.
- the amplitude in is used as the pixel value to obtain the first projected sub-image.
- the feature with the largest value in the second image feature of the preset three-dimensional point in the preset straight line direction is determined as the second projection feature, and the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is determined As pixel values, the second projected sub-image is obtained.
- the first projection sub-image and the second projection sub-image are both data magnitudes corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image.
- the electronic device determines the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature, and uses the third projection feature as the pixel value to obtain the third projection sub-image.
- the third projection sub-image is the data magnitude corresponding to the statistical characteristics of the amplitude of the three-dimensional points in the three-dimensional point cloud image.
- the electronic device can perform fusion processing on the first projection sub-image and the second projection sub-image to obtain an intermediate image.
- the electronic device can pre-process the third projection sub-image, and fuse the intermediate image with the pre-processing result to obtain a corresponding two-dimensional projection image.
- step S10414 may include the following steps:
- S104141 Use the preset fusion algorithm to process the first projection sub-image and the second projection sub-image to obtain the first intermediate image.
- S104142 Perform normalization processing and grayscale transformation processing on the third projection sub-image to obtain the second intermediate image.
- the grayscale transformation process is used to adjust the difference in pixel values between the background area and the target area on the third projection sub-image.
- S104143 Determine the product of the pixel values corresponding to the same position on the first intermediate image and the second intermediate image, and generate a two-dimensional projection map corresponding to the three-dimensional point cloud image based on the product.
- the fusion algorithm involved may include but is not limited to any one of the weighted fusion algorithm, the pyramid fusion algorithm, and the maximum mapping algorithm.
- the above-mentioned preset fusion algorithm is a weighted fusion algorithm.
- the pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image can correspond to different weights.
- the electronic device can calculate the third projection according to the corresponding weights.
- the first intermediate image can be obtained by taking the weighted sum of the pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image, and using the calculated weighted sum as the pixel value.
- the preset fusion algorithm is a maximum mapping algorithm, and the electronic device can select the largest pixel value from the pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image to obtain the first intermediate image.
- the electronic device can also normalize the third projection sub-image.
- the pixel value of the third projection sub-image can be normalized to [0, 1] to adjust the data amount of the third projection sub-image. level, so that the data magnitude of the third projection sub-image is smaller than the data magnitude of the first projection sub-image and the second projection sub-image.
- a grayscale transformation is performed on the normalized third projection subimage.
- the electronic device can perform a linear grayscale transformation on the normalized third projection subimage, or it can also be based on a nonlinear transformation.
- the function (for example, exponential function, logarithmic function, power function, gamma function, etc.) performs nonlinear grayscale transformation on the normalized third projection sub-image to adjust the normalized third projection sub-image.
- the contrast of different areas in the image can avoid the loss of image detail information in the subsequent image fusion process, or ensure a better noise reduction effect.
- the difference in pixel values between the background area and the target area on the third projection sub-image can be reduced (that is, the contrast of different image areas is reduced), or the difference between the background area and the target area on the third projection sub-image can be improved.
- the difference in pixel values between areas i.e., improving the contrast of different image areas
- an exponential function can be used in the grayscale transformation process to improve the contrast of different areas in the projected sub-image; or, for example, in order to retain more noise during the image fusion process
- a logarithmic function can be used to reduce the contrast of different areas in the projected sub-image.
- the echo at the edge of the human body is weak due to the scattering angle. If the contrast of the projected sub-image is too high, part of the human body edge information may be lost in the subsequent image fusion process. Therefore, through grayscale Transformation reduces image contrast, ensuring that as much useful detail information is retained as possible while reducing background noise. If the contrast of the projected sub-image is low, the noise reduction effect during the image fusion process will be average. Therefore, the contrast can be improved through grayscale transformation, thereby achieving better noise reduction effect during the image fusion process.
- Performing grayscale transformation processing on the normalized third projection sub-image can adjust the difference in pixel values between the background area on the normalized third projection sub-image and the target area containing the scanned object, so that the final result is
- the two-dimensional projection image can clearly distinguish the background area from the target area containing the scanned object.
- the electronic device can calculate a product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image, and generate a two-dimensional projection image corresponding to the three-dimensional point cloud image based on the calculated product.
- the electronic device can normalize the calculated product according to the preset normalization interval (for example, 0 to 255), and use the normalized result as a pixel value to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image. .
- the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object.
- the background noise is low, which can improve the quality of the generated two-dimensional projection image.
- FIG 11 is a schematic diagram of the principle of an image generation method provided by an embodiment of the present application.
- the larger cube in Figure 11 represents the 3D point cloud, which is the three-dimensional point cloud image in the embodiment of this application.
- Different statistical features represent different image features of the preset three-dimensional points in the embodiment of the present application.
- the three smaller cubes in Figure 11 respectively represent the respective neighborhoods of the three preset three-dimensional points in the preset straight line direction.
- the preset projection plane is the front of the larger cube in Figure 11, and the line connecting the three preset three-dimensional points (ie, the preset straight line) is perpendicular to the preset projection plane.
- the electronic device can obtain a three-dimensional point cloud image, and based on different feature statistics methods, calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points. Based on the calculated statistical characteristics, it is determined that the preset three-dimensional points are different.
- the image features of the preset three-dimensional points can be obtained in different dimensions.
- the electronic device can determine the projection characteristics of the preset three-dimensional point corresponding to the image feature on the preset projection plane based on each image feature of the preset three-dimensional point.
- the electronic device can obtain the projection sub-image corresponding to the projection feature based on each projection feature of the preset three-dimensional point on the preset projection plane.
- the electronic device can fuse each projection sub-image to obtain a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
- different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object.
- the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object.
- the background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
- Figure 12 is a flow chart of an image generation method provided by an embodiment of the present application.
- the electronic device can obtain the 3D point cloud, which is the three-dimensional point cloud image in the embodiment of the present application, and determine the plane under the preset projection perspective as the preset projection plane. If the three-dimensional point cloud image matches the preset projection perspective, for example, the preset projection plane is parallel to a coordinate plane in the three-dimensional coordinate system of the three-dimensional point cloud image, the electronic device can directly calculate the neighborhood statistical features, that is, according to different features Statistical method, calculate different image features of preset three-dimensional points respectively.
- the electronic device can perform three-dimensional rotation, that is, the electronic device is based on the preset projection perspective Coordinate conversion is performed on the three-dimensional point cloud image so that after the coordinate conversion, a coordinate plane in the three-dimensional coordinate system of the three-dimensional point cloud image is parallel to the preset projection plane, thereby obtaining a three-dimensional point cloud image that matches the preset projection perspective.
- the electronic device can then perform neighborhood statistical feature calculations, that is, calculate different image features of preset three-dimensional points according to different feature statistical methods.
- the electronic device can perform projection, that is, based on the different image characteristics of the preset three-dimensional point, determine the different projection characteristics of the preset three-dimensional point on the preset projection plane, and based on the different projection characteristics, generate the corresponding projection characteristics. Projected subgraph. Furthermore, the electronic device can perform fusion processing on each projection sub-image to obtain a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
- different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object.
- the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object.
- the background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
- Figure 13 is a flow chart of an image generation method provided by an embodiment of the present application.
- the electronic device can obtain a 3D point cloud, which is a three-dimensional point cloud image in the embodiment of the present application. Then, the electronic device can calculate different image features of the preset three-dimensional points in the three-dimensional point cloud image according to different feature statistics methods.
- the image characteristics of a preset three-dimensional point are: According to different feature statistics methods, the calculated image characteristics of the preset three-dimensional point are: Assume that the three-dimensional point is the statistical characteristic of the amplitude of each three-dimensional point in the neighborhood of the central point.
- the statistical characteristics of the preset three-dimensional point include: statistical characteristics 1, statistical characteristics 2,..., statistical characteristics n. Each statistical characteristic corresponds to a characteristic statistical method. n represents the number of characteristic statistical methods, and n is greater than or equal to 1. .
- Feature statistical methods can include: mode calculation, maximum value calculation, variance calculation, signal to noise ratio calculation, entropy value calculation, convolution kernel operator and custom function, etc.
- the electronic device determines the projection feature of the preset three-dimensional point corresponding to the image feature on the preset projection plane based on the image feature, and generates the projection feature based on each projection feature.
- the generated projection sub-images include: projection image 1, projection image 2,..., projection image n.
- the electronic device can fuse each projection sub-image (i.e., projection image 1, projection image 2,..., projection image n) to obtain a front view of the scanned object.
- the front view of the scanned object is also a three-dimensional point cloud image in the main viewing direction. 2D projection diagram.
- different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object.
- the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object.
- the background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
- the image generation method in the related art is the maximum projection method.
- the maximum projection method uses the following formula to determine the projection characteristics of a preset three-dimensional point in the preset straight line direction projected on the preset projection plane:
- ⁇ 4 represents the projection characteristics of the preset three-dimensional point in the preset straight line direction determined based on the maximum projection method on the preset projection plane, and ⁇ (x, y, x) represents the preset three-dimensional point in the preset straight line direction.
- max() represents the maximum value function.
- the image on the left in Figures 14 to 22 is a two-dimensional projection image obtained based on the image generation method in the related art
- the image on the right is a two-dimensional projection image obtained based on the image generation method provided by the embodiment of the present application.
- Figures 15, 16 and 17 are multiple sets of enlarged views of local image areas. Each set of enlarged views is an enlarged view of the local image area corresponding to the same position in the two two-dimensional projections in Figure 14.
- the two-dimensional projection on the right in Figure 14 has clearer edges in different areas than the two-dimensional projection on the left, and the two-dimensional projection on the right in Figure 14 contains more edges than the two-dimensional projection on the left. Image details.
- the image in Figure 15 is an enlarged view of the background area in the two-dimensional projection image. It can be seen that the background area in the image on the right in Figure 15 is darker than the background area in the image on the left, that is, the background area in the image on the right has a lower amplitude than the background area in the image on the left.
- the image in Figure 16 is an enlarged view of the area where the character's knees are located in the two-dimensional projection. It can be seen that the edge of the knee pad in the image on the right side of Figure 16 is clearer than that in the image on the left side.
- the scanning object in Figures 17 and 18 is a wrench.
- the image on the right in Figures 17 and 18 has a clearer edge of the wrench than the image on the left.
- the image on the right side of Figure 17 contains more image details than the image on the left side. For example, in the image on the right side of Figure 17, the details of the head of the wrench, the details of the handle, etc. can be clearly observed.
- the scanning objects in Figures 19 and 20 are mobile phones and headphone boxes.
- the edges of the mobile phone and headphone box are clearer in the images on the right than in the images on the left.
- the image on the right side of Figure 20 contains more image details than the image on the left side. For example, it can be clearly observed in the image on the right side of Figure 20 that the scanned object includes a mobile phone.
- the scanned object in Figure 21 is plasticine.
- the echo signal of the millimeter wave signal reflected from the plasticine is weaker than the echo signal of the millimeter wave signal reflected from the human body.
- the image on the right in Figure 21 has a clearer outline of the scanned object than the image on the left, and the contrast between light and dark is more obvious.
- the scanning object in Figure 22 is the tool.
- the edge of the tool in the image on the right side of Figure 22 is clearer than the image on the left side, and the image on the right side of Figure 22 contains more image details than the image on the left side.
- the edges of different areas in the two-dimensional projection image generated based on the image generation method provided by the embodiment of the present application are clearer, that is, the outline of the scanned object is clearer. And it contains more image details, and can clearly distinguish different areas, and the background noise is low, which can improve the quality of the generated two-dimensional projection map.
- the subsequent two-dimensional projection map is applied to different scenes, the application effect of the image is improved. .
- Figure 23 is a structural diagram of an image generation device provided by an embodiment of the present application. Contents not explained in detail in the following embodiments can refer to the description of the above embodiments.
- the image generation device includes:
- the three-dimensional point cloud image acquisition module 2301 is used to obtain the three-dimensional point cloud image
- the image feature acquisition module 2302 is used to determine the statistical features corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image of the preset three-dimensional points based on the statistical features.
- the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
- the projection feature acquisition module 2303 is used to determine the projection features of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image features of the preset three-dimensional point in the preset straight line direction; wherein, the The preset straight line direction is perpendicular to the preset projection plane;
- the two-dimensional projection image acquisition module 2304 is used to determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
- the image features of the preset three-dimensional point include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining smooth areas of the image, and The third image feature participates in characterizing the image background noise.
- the image feature acquisition module 2302 is specifically used for at least one of the following:
- the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image is determined.
- Statistical features as the first image features of the preset three-dimensional point; wherein the plane where the first neighborhood is located is parallel to the preset projection plane;
- the second statistical feature corresponding to the value is used as the second image feature of the preset three-dimensional point;
- the projection feature acquisition module 2303 is specifically used for at least one of the following:
- the two-dimensional projection image acquisition module 2304 is specifically configured to determine where the three-dimensional point cloud image is based on at least one of the first projection feature, the second projection feature, and the third projection feature on the preset projection plane.
- the two-dimensional projection image on the preset projection plane is specifically configured to determine where the three-dimensional point cloud image is based on at least one of the first projection feature, the second projection feature, and the third projection feature on the preset projection plane. The two-dimensional projection image on the preset projection plane.
- the two-dimensional projection image acquisition module 2304 is specifically used to:
- Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- the two-dimensional projection image acquisition module 2304 is specifically used to:
- a product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image is determined, and a two-dimensional projection image corresponding to the three-dimensional point cloud image is generated based on the product.
- the three-dimensional point cloud image acquisition module 2301 is specifically used to acquire a three-dimensional point cloud image of the scanned object
- the preset projection plane includes: a projection plane parallel to the front and/or back of the scanned object.
- the three-dimensional point cloud image acquisition module 2301 is specifically configured to rotate the original three-dimensional point cloud image based on a preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective;
- the preset projection plane is the projection plane under the preset projection viewing angle.
- the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the preset three-dimensional points can be determined.
- Image characteristics, and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the image characteristics of the preset three-dimensional point in the preset straight line direction, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated.
- different statistical features can reflect different image features.
- the embodiment of the present application also provides an electronic device, as shown in Figure 24, including a processor 2401, a communication interface 2402, a memory 2403, and a communication bus 2404.
- the processor 2401, the communication interface 2402, and the memory 2403 communicate through the communication bus 2404. complete mutual communication,
- Memory 2403 used to store computer programs
- the processor 2401 is used to implement the following steps when executing the program stored in the memory 2403:
- the three-dimensional point Determine the statistical characteristics corresponding to the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional point based on the statistical characteristics; wherein, the three-dimensional point
- the amplitude of is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
- the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane are determined; wherein the preset straight line direction is the same as the preset straight line direction.
- the default projection plane is vertical;
- a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
- the communication bus mentioned in the above-mentioned electronic equipment can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
- PCI Peripheral Component Interconnect
- EISA Extended Industry Standard Architecture
- the communication bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
- the communication interface is used for communication between the above-mentioned electronic devices and other devices.
- the memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory.
- RAM Random Access Memory
- NVM Non-Volatile Memory
- the memory may also be at least one storage device located remotely from the aforementioned processor.
- the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processor, DSP), special integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
- CPU central processing unit
- NP Network Processor
- DSP Digital Signal Processor
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- a non-transitory computer-readable storage medium stores a computer program, and the computer program is implemented when executed by a processor. The steps for any of the above image generation methods.
- a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image generation methods in the above embodiments.
- the computer program product includes one or more computer instructions.
- the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
- the computer instructions may be stored in a non-transitory computer-readable storage medium or transmitted from one non-transitory computer-readable storage medium to another. For example, the computer instructions may be transferred from a non-transitory computer-readable storage medium to another.
- a website, computer, server or data center transmits data to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) Make the transfer.
- the non-transitory computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
- the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), etc.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
An image generation method and device, relating to the technical field of image processing. The method comprises: obtaining a three-dimensional point cloud image; determining statistical characteristics corresponding to the amplitude of a three-dimensional point in a neighborhood of a preset three-dimensional point in the three-dimensional point cloud image, and determining the image characteristics of the preset three-dimensional point according to the statistical characteristics, wherein the amplitude of the three-dimensional point is used for representing the electromagnetic scattering characteristics at a corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point comprises a preset point cloud area of the preset three-dimensional point; according to the image characteristics of the preset three-dimensional point in a preset linear direction, determining the projection characteristics of the preset three-dimensional point on a preset projection plane in the preset linear direction perpendicular to the preset projection plane; and determining a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane. Therefore, a two-dimensional image corresponding to the three-dimensional point cloud image can be generated, and the generated two-dimensional image can adapt to different application scenes, so that the application effect of the image is improved.
Description
本申请要求于2022年03月25日提交中国专利局、申请号为202210305072.1发明名称为“一种图像生成方法和装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority to the Chinese patent application with application number 202210305072.1 and the invention title is "an image generation method and device" submitted to the China Patent Office on March 25, 2022, the entire content of which is incorporated into this application by reference.
本申请涉及图像处理技术领域,特别是涉及一种图像生成方法和装置。The present application relates to the field of image processing technology, and in particular to an image generation method and device.
相关技术中,可以通过激光雷达、毫米波雷达、或微波雷达等三维扫描设备获取空间中若干点的信息,包括坐标和对应的幅值,得到三维点云图。其中,三维点云图中每一三维点的幅值根据空间中该三维点对应位置处的电磁散射特性确定。In related technologies, three-dimensional scanning equipment such as lidar, millimeter-wave radar, or microwave radar can be used to obtain information on several points in space, including coordinates and corresponding amplitudes, to obtain a three-dimensional point cloud map. Among them, the amplitude of each three-dimensional point in the three-dimensional point cloud image is determined based on the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point in space.
然而,三维点云图的数据量较大,直接对三维点云图进行业务处理(例如,目标检测、目标识别等)的复杂度较大,且成本较高。因此,亟需一种方法,以生成三维点云图对应的二维图像,进而,则可以基于该二维图像进行业务处理。However, the data volume of the 3D point cloud image is large, and directly performing business processing (such as target detection, target recognition, etc.) on the 3D point cloud image is highly complex and costly. Therefore, a method is urgently needed to generate a two-dimensional image corresponding to the three-dimensional point cloud image, and then business processing can be performed based on the two-dimensional image.
发明内容Contents of the invention
第一方面,本申请实施例公开了一种图像生成方法,所述方法包括:In a first aspect, an embodiment of the present application discloses an image generation method, which method includes:
获取三维点云图;Obtain a three-dimensional point cloud image;
确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;Determine the statistical characteristics corresponding to the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional point based on the statistical characteristics; wherein, the three-dimensional point The amplitude of is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;According to the image characteristics of the preset three-dimensional point in the preset straight line direction, the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane are determined; wherein the preset straight line direction is the same as the preset straight line direction. The default projection plane is vertical;
根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。According to the projection characteristics on the preset projection plane, a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
第二方面,本申请实施例公开了一种图像生成装置,所述装置包括:In a second aspect, an embodiment of the present application discloses an image generation device, which includes:
三维点云图获取模块,用于获取三维点云图;The three-dimensional point cloud image acquisition module is used to obtain the three-dimensional point cloud image;
图像特征获取模块,用于确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;The image feature acquisition module is used to determine the statistical features corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image features of the preset three-dimensional points based on the statistical features. ; Wherein, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
投影特征获取模块,用于根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;The projection feature acquisition module is used to determine the projection features of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image features of the preset three-dimensional point in the preset straight line direction; wherein, the The preset straight line direction is perpendicular to the preset projection plane;
二维投影图获取模块,用于根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。A two-dimensional projection image acquisition module is used to determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
在本申请实施的另一方面,还公开了一种电子设备,所述电子设备包括处理器、通信接口、存储器和通信总线,其中,所述处理器,所述通信接口,所述存储器通过所述通信总线完成相互间的通信;In another aspect of the application, an electronic device is also disclosed. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory pass through The above communication bus completes mutual communication;
所述存储器,用于存放计算机程序;The memory is used to store computer programs;
所述处理器,用于执行所述存储器上所存放的程序时,实现如上述任一所述的图像生成方法。The processor is configured to implement any of the above image generation methods when executing a program stored on the memory.
在本申请实施的又一方面,还提供了一种非临时性计算机可读存储介质,所述非临时性计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现如上述任一所述的图像生成方法。In yet another aspect of the implementation of the present application, a non-transitory computer-readable storage medium is also provided. A computer program is stored in the non-transitory computer-readable storage medium. When the computer program is executed by a processor, the following is implemented: Any of the image generation methods described above.
在本申请实施的又一方面,还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使 得计算机执行上述任一所述的图像生成方法。In yet another aspect of the implementation of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any one of the above image generation methods.
本申请实施例有益效果:Beneficial effects of the embodiments of this application:
本申请实施例提供的图像生成方法,获取三维点云图;确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据统计特征,确定预设三维点的图像特征;其中,三维点的幅值用于表征三维点对应位置处的电磁散射特性,预设三维点的邻域指包括预设三维点的预设点云区域;根据处于预设直线方向上的预设三维点的图像特征,确定预设直线方向上的预设三维点在预设投影平面上的投影特征;预设直线方向与预设投影平面垂直;根据预设投影平面上的投影特征,确定三维点云图在预设投影平面上的二维投影图。The image generation method provided by the embodiment of the present application obtains a three-dimensional point cloud image; determines the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determines the amplitude of the preset three-dimensional point based on the statistical characteristics. Image features; among them, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point; according to the preset straight line direction The image characteristics of the preset three-dimensional point determine the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane; the preset straight line direction is perpendicular to the preset projection plane; according to the projection characteristics on the preset projection plane , determine the two-dimensional projection of the three-dimensional point cloud image on the preset projection plane.
基于上述处理,可以确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据确定出的统计特征,确定预设三维点的图像特征,并根据处于预设直线方向上的预设三维点的图像特征,确定三维点云图在预设投影平面上的二维投影图,即能够生成三维点云图对应的二维图像。并且,不同的统计特征可以体现不同的图像特征,相应的,针对不同的应用场景,可以基于实际需求采用不同的统计特征,则生成的二维图像能够体现对应的图像特征,以适应于当前的应用场景,即本申请实施例可以提高投影得到的二维图像的质量,进而可以提高图像的应用效果。Based on the above processing, the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the image characteristics of the preset three-dimensional points can be determined, and based on the preset three-dimensional points. Assume the image characteristics of the preset three-dimensional points in the straight line direction and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated. Moreover, different statistical features can reflect different image features. Correspondingly, for different application scenarios, different statistical features can be used based on actual needs, and the generated two-dimensional image can reflect the corresponding image features to adapt to the current situation. Application scenarios, that is, embodiments of the present application can improve the quality of the projected two-dimensional image, thereby improving the application effect of the image.
当然,实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。Of course, implementing any product or method of the present application does not necessarily require achieving all the above-mentioned advantages simultaneously.
此处所说明的附图用来提供对本申请的进一步理解,构成本申请的一部分,本申请的示意性实施例及其说明用于解释本申请,并不构成对本申请的不当限定。The drawings described here are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application.
图1为本申请实施例提供的一种图像生成方法的流程图;Figure 1 is a flow chart of an image generation method provided by an embodiment of the present application;
图2为本申请实施例提供的一种三维空间的示意图;Figure 2 is a schematic diagram of a three-dimensional space provided by an embodiment of the present application;
图3为本申请实施例提供的另一种三维空间的示意图;Figure 3 is a schematic diagram of another three-dimensional space provided by an embodiment of the present application;
图4为本申请实施例提供的另一种图像生成方法的流程图;Figure 4 is a flow chart of another image generation method provided by an embodiment of the present application;
图5为本申请实施例提供的另一种图像生成方法的流程图;Figure 5 is a flow chart of another image generation method provided by an embodiment of the present application;
图6为本申请实施例提供的另一种图像生成方法的流程图;Figure 6 is a flow chart of another image generation method provided by an embodiment of the present application;
图7a为本申请实施例提供的一种第一投影子图;Figure 7a is a first projection sub-image provided by an embodiment of the present application;
图7b为图7a所示的第一投影子图的局部图像区域的放大图;Figure 7b is an enlarged view of the partial image area of the first projection sub-image shown in Figure 7a;
图8a为本申请实施例提供的一种第二投影子图;Figure 8a is a second projection sub-image provided by the embodiment of the present application;
图8b为图8a所示的第二投影子图的局部图像区域的放大图;Figure 8b is an enlarged view of the partial image area of the second projection sub-image shown in Figure 8a;
图9a为本申请实施例提供的一种第三投影子图;Figure 9a is a third projection sub-image provided by the embodiment of the present application;
图9b为图9a所示的第三投影子图的局部图像区域的放大图;Figure 9b is an enlarged view of the partial image area of the third projection sub-image shown in Figure 9a;
图10为本申请实施例提供的另一种图像生成方法的流程图;Figure 10 is a flow chart of another image generation method provided by an embodiment of the present application;
图11为本申请实施例提供的一种图像生成方法的原理示意图;Figure 11 is a schematic diagram of the principle of an image generation method provided by an embodiment of the present application;
图12为本申请实施例提供的另一种图像生成方法的流程图;Figure 12 is a flow chart of another image generation method provided by an embodiment of the present application;
图13为本申请实施例提供的另一种图像生成方法的流程图;Figure 13 is a flow chart of another image generation method provided by an embodiment of the present application;
图14为本申请实施例提供的一种二维投影图的对比图;Figure 14 is a comparison diagram of a two-dimensional projection image provided by an embodiment of the present application;
图15为图14所示的各二维投影图中对应相同位置的一种局部图像区域的放大图的对比图;Figure 15 is a comparison of enlarged views of a partial image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
图16为图14所示的各二维投影图中对应相同位置的另一种局部图像区域的放大图的对比图;Figure 16 is a comparison diagram of another enlarged view of the local image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
图17为图14所示的各二维投影图中对应相同位置的另一种局部图像区域的放大图的对比图;Figure 17 is a comparison diagram of another enlarged view of the local image area corresponding to the same position in each of the two-dimensional projections shown in Figure 14;
图18为本申请实施例提供的另一种二维投影图的对比图;Figure 18 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application;
图19为本申请实施例提供的另一种二维投影图的对比图;Figure 19 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application;
图20为本申请实施例提供的另一种二维投影图的对比图;Figure 20 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application;
图21为本申请实施例提供的另一种二维投影图的对比图;Figure 21 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application;
图22为本申请实施例提供的另一种二维投影图的对比图;Figure 22 is a comparison diagram of another two-dimensional projection image provided by the embodiment of the present application;
图23为本申请实施例提供的一种图像生成装置的结构图;Figure 23 is a structural diagram of an image generation device provided by an embodiment of the present application;
图24为本申请实施例提供的一种电子设备的结构图。Figure 24 is a structural diagram of an electronic device provided by an embodiment of the present application.
为使本申请的目的、技术方案、及优点更加清楚明白,以下参照附图并举实施例,对本申请进一步详细说明。显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to make the purpose, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.
本申请实施例提供了一种图像生成方法,该方法可以应用于电子设备,该电子设备可以基于本申请实施例提供的方法,对获取到的三维点云图进行处理,得到对应的二维图像。相应的,在获得二维图像之后,可以对二维图像进行图像识别,例如,可以识别二维图像中对象(例如,人物、物品等)的类别,或者,也可以识别二维图像中对象所属的图像区域,或者,也可以识别二维图像中是否存在指定的对象,但并不限于此。Embodiments of the present application provide an image generation method, which can be applied to electronic devices. The electronic device can process the obtained three-dimensional point cloud image based on the method provided by the embodiments of the present application to obtain a corresponding two-dimensional image. Correspondingly, after obtaining the two-dimensional image, image recognition can be performed on the two-dimensional image. For example, the category of the object (for example, a person, an object, etc.) in the two-dimensional image can be identified, or the category to which the object in the two-dimensional image belongs can also be identified. The image area, or it can also identify whether the specified object exists in the two-dimensional image, but is not limited to this.
参见图1,图1为本申请实施例提供的一种图像生成方法的流程图,该方法可以包括以下步骤:Referring to Figure 1, Figure 1 is a flow chart of an image generation method provided by an embodiment of the present application. The method may include the following steps:
S101:获取三维点云图。S101: Obtain a three-dimensional point cloud image.
S102:确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据统计特征,确定预设三维点的图像特征。S102: Determine the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional points based on the statistical characteristics.
其中,三维点的幅值用于表征三维点对应位置处的电磁散射特性,预设三维点的邻域指包括预设三维点的预设点云区域。预设点云区域的大小、形状可以根据实际需求而定。上述邻域内的三维点的幅值对应的统计特征是指利用统计学规律对各个幅值进行统计处理后得到的特征量。相比于只考虑预设三维点自身的幅值的情况,通过利用统计特征,有助于从整体上更好的表征该预设三维点对应幅值特性,进而为后续得到优质的二维投影图像奠定基础。示例性的,可以将得到的统计特征,直接确定为预设三维点的图像特征,也可以对得到的统计特征进行处理,例如按照预设运算规则(可以根据不同的处理需求而定)对统计特征进行运算或变换等,将处理结果确定为预设三维点的图像特征。在本申请实施例中,图像特征可以是指用于表征三维点对应的显示图像的显示特点或显示效果的特征。Among them, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point. The size and shape of the preset point cloud area can be determined according to actual needs. The statistical characteristics corresponding to the amplitudes of the three-dimensional points in the above neighborhood refer to the characteristic quantities obtained by statistically processing each amplitude using statistical rules. Compared with the situation where only the amplitude of the preset three-dimensional point itself is considered, the use of statistical features helps to better characterize the corresponding amplitude characteristics of the preset three-dimensional point as a whole, thereby obtaining high-quality two-dimensional projections in the future. Images lay the foundation. For example, the obtained statistical features can be directly determined as image features of preset three-dimensional points, or the obtained statistical features can be processed, for example, according to preset operation rules (which can be determined according to different processing requirements). The features are operated or transformed, and the processing results are determined as image features of preset three-dimensional points. In the embodiment of the present application, image features may refer to features used to characterize the display characteristics or display effects of the display image corresponding to the three-dimensional point.
S103:根据处于预设直线方向上的预设三维点的图像特征,确定预设直线方向上的预设三维点在预设投影平面上的投影特征。S103: Determine the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image characteristics of the preset three-dimensional point in the preset straight line direction.
其中,预设直线方向与预设投影平面垂直。每个预设直线方向上的预设三维点为多个,则对应的图像特征为多个,进而可以依据这多个图像特征,确定该预设直线方向在预设投影平面上对应的投影特征。投影特征是形成二维投影图的直接依据,投影特征的选取与二维投影图的质量相关联,换言之,在本申请实施例中,投影特征可以是指用于表征二维投影图的显示特征或显示效果的特征。例如,可以利用投影特征对应的预设三维点在三维点云图中的像素值,确定二维投影图上该预设三维点对应位置处的像素值。Wherein, the preset straight line direction is perpendicular to the preset projection plane. If there are multiple preset three-dimensional points in each preset straight line direction, then there are multiple corresponding image features. Then, based on these multiple image features, the projection features corresponding to the preset straight line direction on the preset projection plane can be determined. . Projection features are the direct basis for forming a two-dimensional projection image. The selection of projection features is related to the quality of the two-dimensional projection image. In other words, in the embodiment of the present application, the projection features may refer to display features used to characterize the two-dimensional projection image. or the characteristics of the display effect. For example, the pixel value of the preset three-dimensional point corresponding to the projection feature in the three-dimensional point cloud map can be used to determine the pixel value at the position corresponding to the preset three-dimensional point on the two-dimensional projection map.
S104:根据预设投影平面上的投影特征,确定三维点云图在预设投影平面上的二维投影图。S104: Determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
基于本申请实施例提供的图像生成方法,可以确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据确定出的统计特征,确定预设三维点的图像特征,并根据处于预设直线方向上的预设三维点的图像特征,确定三维点云图在预设投影平面上的二维投影图,即能够生成三维点云图对应的二维图像。并且,不同的统计特征可以体现不同的图像特征,相应的,针对不同的应用场景,可以基于实际需求采用不同的统计特征,则生成的二维图像能够体现对应的图像特征,以适应于当前的应用场景,即本申请实施例可以提高投影得到的二维图像的质量,进而可以提高图像的应用效果。Based on the image generation method provided by the embodiment of the present application, the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the preset three-dimensional points can be determined. Image characteristics, and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the image characteristics of the preset three-dimensional point in the preset straight line direction, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated. Moreover, different statistical features can reflect different image features. Correspondingly, for different application scenarios, different statistical features can be used based on actual needs, and the generated two-dimensional image can reflect the corresponding image features to adapt to the current situation. Application scenarios, that is, embodiments of the present application can improve the quality of the projected two-dimensional image, thereby improving the application effect of the image.
针对步骤S101,电子设备可以通过雷达设备,获取三维点云图。本申请实施例中的三维点云图可以为SAR(Synthetic Aperture Radar,合成孔径雷达)图像。雷达设备可以为激光雷达,或者毫米波雷达,或者微波雷达等。For step S101, the electronic device can obtain the three-dimensional point cloud image through the radar device. The three-dimensional point cloud image in the embodiment of this application may be a SAR (Synthetic Aperture Radar) image. Radar equipment can be lidar, millimeter wave radar, microwave radar, etc.
在一个实施例中,步骤S101可以包括以下步骤:获取扫描对象的三维点云图。In one embodiment, step S101 may include the following steps: obtaining a three-dimensional point cloud image of the scanned object.
例如,在机场安检、地铁安检等安检场景中,可以通过主动式毫米波安检仪中的毫米波雷达发射调制信号(调制信号为毫米波信号),根据成像算法对调制信号在扫描对象(例如,人物,物品等)处反射的回波信号进行处理,得到扫描对象的三维点云图。然后,电子设备可以基于本申请实施例提供的方法,基于获取到的三维点云图,生成对应的二维图像。后续,可以对二维图像进行图像识别,以对危险物品进行检出、识别和定位。For example, in security inspection scenarios such as airport security inspection and subway security inspection, the modulated signal can be emitted through the millimeter wave radar in the active millimeter wave security inspection instrument (the modulated signal is a millimeter wave signal), and the modulated signal is scanned according to the imaging algorithm (for example, The echo signals reflected by people, objects, etc.) are processed to obtain a three-dimensional point cloud image of the scanned object. Then, the electronic device can generate a corresponding two-dimensional image based on the obtained three-dimensional point cloud image based on the method provided by the embodiment of the present application. Subsequently, image recognition can be performed on the two-dimensional images to detect, identify and locate dangerous items.
通过主动式毫米波安检仪获取三维点云图时,基于成像机制与实际情况,对毫米波信号在扫描对象处反射的回波信号进行均匀采样,以生成相应的三维点云图。因此,获取的三维点云图中包含在三维空间中均匀分布的多个三维点,各三维点之间的间隔基于生成三维点云图时的采样间隔确定,三维点云图中一个三维点的幅值表示空间中该三维点对应位置处的电磁散射特性。三维点云图中各三维点各自的幅值可以用一个三维矩阵表示。When obtaining a three-dimensional point cloud image through an active millimeter wave security detector, based on the imaging mechanism and actual conditions, the echo signal reflected by the millimeter wave signal at the scanning object is evenly sampled to generate a corresponding three-dimensional point cloud image. Therefore, the obtained three-dimensional point cloud image contains multiple three-dimensional points evenly distributed in the three-dimensional space. The interval between each three-dimensional point is determined based on the sampling interval when the three-dimensional point cloud image is generated. The amplitude of a three-dimensional point in the three-dimensional point cloud image represents The electromagnetic scattering characteristics at the corresponding position of the three-dimensional point in space. The respective amplitudes of each three-dimensional point in the three-dimensional point cloud image can be represented by a three-dimensional matrix.
对于通过其他方式(例如,激光雷达)获取的三维点云图中的多个三维点稀疏分布在三维空间中。For the multiple 3D points in the 3D point cloud obtained by other means (for example, lidar), they are sparsely distributed in the 3D space.
在基于三维点云图生成二维投影图时,可以确定待生成的二维投影图所在的投影平面(即本申请实施例中的预设投影平面),将三维点云图投影至预设投影平面,可以得到对应的二维投影图。When generating a two-dimensional projection image based on a three-dimensional point cloud image, the projection plane where the two-dimensional projection image to be generated is located (ie, the preset projection plane in the embodiment of the present application) can be determined, and the three-dimensional point cloud image can be projected to the preset projection plane, The corresponding two-dimensional projection map can be obtained.
预设投影平面可以包括:与扫描对象的正面和/或背面平行的投影平面。在安检场景中,扫描对象为人物,预设投影平面可以为与人物的正面和/或背面平行的投影平面,人物的正面为人物面部朝向的平面。相应的,三维点云图在预设投影平面上的二维投影图包含人物的完整的图像信息,以及该人物携带的物品的图像信息,可以方便对人物进行安检。例如,该人物携带了扳手,按照与人物的正面平行的预设投影平面,生成的二维投影图中包含扳手的完整的图像信息,基于二维投影图能够对扳手进行检出、识别和定位。The preset projection plane may include: a projection plane parallel to the front and/or back of the scanned object. In a security inspection scenario, the scanning object is a person, and the preset projection plane can be a projection plane parallel to the front and/or back of the person, and the front of the person is the plane toward which the person's face faces. Correspondingly, the two-dimensional projection of the three-dimensional point cloud image on the preset projection plane contains the complete image information of the person and the image information of the items carried by the person, which can facilitate the security inspection of the person. For example, the character carries a wrench. According to the preset projection plane parallel to the front of the character, the generated two-dimensional projection image contains complete image information of the wrench. Based on the two-dimensional projection image, the wrench can be detected, identified and positioned. .
预设投影平面包含与人物的正面和背面平行的投影平面时,可以得到多张不同方向上的二维投影图,则可以从不同方向上丰富人物的图像信息,结合不同方向上的图像信息进行识别,可以提高图像识别的准确性。When the preset projection plane includes projection planes parallel to the front and back of the character, multiple two-dimensional projection images in different directions can be obtained, and the image information of the character can be enriched from different directions and combined with the image information in different directions. Recognition can improve the accuracy of image recognition.
在安检场景中,可以基于三维点云图生成预设投影视角下的二维图像,例如,可以基于三维点云图生成扫描对象在主视视角下的二维图像(即扫描对象的主视图)。也可以基于三维点云图生成扫描对象在侧视视角下的二维图像(即扫描对象的侧视图)。In the security inspection scene, a two-dimensional image under a preset projection perspective can be generated based on the three-dimensional point cloud image. For example, a two-dimensional image of the scanned object under the main perspective (that is, the main view of the scanned object) can be generated based on the three-dimensional point cloud image. A two-dimensional image of the scanned object from a side view (ie, a side view of the scanned object) can also be generated based on the three-dimensional point cloud image.
一种实现方式中,电子设备可以确定预设投影视角下的投影平面为预设投影平面,并判断雷达设备获取的原始三维点云图与预设投影视角是否匹配。如果原始三维点云图与预设投影视角相匹配,例如,预设投影平面与原始三维点云图的三维坐标系中的一个坐标平面相平行,电子设备可以直接计算原始三 维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征。In one implementation, the electronic device can determine that the projection plane under the preset projection perspective is the preset projection plane, and determine whether the original three-dimensional point cloud image obtained by the radar device matches the preset projection perspective. If the original three-dimensional point cloud image matches the preset projection perspective, for example, the preset projection plane is parallel to a coordinate plane in the three-dimensional coordinate system of the original three-dimensional point cloud image, the electronic device can directly calculate the preset three-dimensional angle in the original three-dimensional point cloud image. The statistical characteristics corresponding to the amplitude of three-dimensional points in the neighborhood of the point.
示例性的,参见图2,图2为本申请实施例提供的一种三维空间的示意图。扫描对象为人物,图2中长方体所示的三维范围表示人物的原始三维点云图,图2所示的三维坐标系的坐标平面包括:XOY平面、XOZ平面和YOZ平面。当需要生成图2所示的人物的主视图时,预设投影视角为主视视角,主视视角下的预设投影平面为与人物的正面平行的投影平面,则预设投影平面与三维坐标系中的XOY平面平行,即,原始三维点云图与预设投影视角相匹配。相应的,电子设备获取图2中所示原始三维点云图之后,可以直接计算三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征。For example, see FIG. 2 , which is a schematic diagram of a three-dimensional space provided by an embodiment of the present application. The scanning object is a person. The three-dimensional range shown by the cuboid in Figure 2 represents the original three-dimensional point cloud image of the person. The coordinate planes of the three-dimensional coordinate system shown in Figure 2 include: XOY plane, XOZ plane and YOZ plane. When it is necessary to generate the front view of the character as shown in Figure 2, the preset projection angle is the main view angle, and the preset projection plane under the main view angle is a projection plane parallel to the front of the character, then the preset projection plane and the three-dimensional coordinates The XOY plane in the system is parallel, that is, the original three-dimensional point cloud image matches the preset projection perspective. Correspondingly, after the electronic device obtains the original three-dimensional point cloud image shown in Figure 2, it can directly calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image that are in the neighborhood of the preset three-dimensional points.
另一种实现方式中,步骤S101可以包括以下步骤:基于预设投影视角对原始三维点云图进行旋转,得到与预设投影视角匹配的三维点云图。其中,预设投影平面为预设投影视角下的投影平面。In another implementation, step S101 may include the following steps: rotating the original three-dimensional point cloud image based on a preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective. Wherein, the preset projection plane is the projection plane under the preset projection angle.
如果通过雷达设备获取的原始三维点云图与预设投影视角不匹配,例如,预设投影平面与三维点云图的三维坐标系中的所有坐标平面均不平行,为了生成三维点云图在预设投影视角下的二维投影图像,在获取到原始三维点云图后,电子设备可以基于预设投影视角与原始三维点云图之间的位置关系,对原始三维点云图进行旋转,得到与预设投影视角相匹配的三维点云图。If the original three-dimensional point cloud image obtained through the radar device does not match the preset projection perspective, for example, the preset projection plane is not parallel to all coordinate planes in the three-dimensional coordinate system of the three-dimensional point cloud image, in order to generate the three-dimensional point cloud image, in the preset projection A two-dimensional projection image under a viewing angle. After acquiring the original three-dimensional point cloud image, the electronic device can rotate the original three-dimensional point cloud image based on the positional relationship between the preset projection angle of view and the original three-dimensional point cloud image to obtain the same as the preset projection angle of view. Matching 3D point cloud images.
电子设备可以基于预设投影平面的二维坐标系,以及预设投影平面的二维坐标系与原始三维点云图的三维坐标系之间的坐标映射关系,对原始三维点云图进行坐标转换,以使得进行坐标转换之后,得到的三维点云图的三维坐标系中的一个坐标平面与预设投影平面相平行,坐标转换得到的三维点云图为与预设投影视角相匹配的三维点云图。然后电子设备可以计算三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征。The electronic device can perform coordinate conversion on the original three-dimensional point cloud image based on the two-dimensional coordinate system of the preset projection plane and the coordinate mapping relationship between the two-dimensional coordinate system of the preset projection plane and the three-dimensional coordinate system of the original three-dimensional point cloud image, so as to After coordinate conversion, one coordinate plane in the three-dimensional coordinate system of the obtained three-dimensional point cloud image is parallel to the preset projection plane, and the three-dimensional point cloud image obtained by the coordinate conversion is a three-dimensional point cloud image that matches the preset projection perspective. The electronic device can then calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image.
示例性的,参见图3,图3为本申请实施例提供的另一种三维空间的示意图。扫描对象为人物,图3中长方体所示的三维范围表示人物的原始三维点云图,图3所示的三维坐标系的坐标平面包括:XOY平面、XOZ平面和YOZ平面。当需要生成图3所示的人物在侧视视角的二维投影图时,预设投影视角为侧视视角,侧视视角下的预设投影平面为X
1O
1Y
1平面,X
1O
1Y
1平面与三维坐标系中的所有坐标平面均不平行。
For example, see FIG. 3 , which is a schematic diagram of another three-dimensional space provided by an embodiment of the present application. The scanning object is a person. The three-dimensional range shown by the cuboid in Figure 3 represents the original three-dimensional point cloud image of the person. The coordinate planes of the three-dimensional coordinate system shown in Figure 3 include: XOY plane, XOZ plane and YOZ plane. When it is necessary to generate a two-dimensional projection of the character from the side view as shown in Figure 3, the default projection view is the side view, and the preset projection plane in the side view is the X 1 O 1 Y 1 plane, X 1 O The 1 Y 1 plane is not parallel to any coordinate plane in the three-dimensional coordinate system.
相应的,电子设备可以基于X
1O
1Y
1平面的二维坐标系,以及X
1O
1Y
1平面的二维坐标系与原始三维点云图的三维坐标系之间的坐标映射关系,对原始三维点云图进行坐标转换,使得进行坐标转换之后三维点云图的三维坐标系中的XOY平面与X
1O
1Y
1平面相平行。然后电子设备可以计算三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征。
Correspondingly, the electronic device can be based on the two-dimensional coordinate system of the X 1 O 1 Y 1 plane and the coordinate mapping relationship between the two-dimensional coordinate system of the X 1 O 1 Y 1 plane and the three-dimensional coordinate system of the original three-dimensional point cloud image. The original three-dimensional point cloud image undergoes coordinate transformation so that the XOY plane in the three-dimensional coordinate system of the three-dimensional point cloud image is parallel to the X 1 O 1 Y 1 plane after the coordinate transformation. The electronic device can then calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image.
针对步骤S102,预设三维点可以为三维点云图中的任意一个三维点。预设三维点的邻域指包括预设三维点的预设点云区域。Regarding step S102, the preset three-dimensional point can be any three-dimensional point in the three-dimensional point cloud image. The neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point.
预设点云区域可以基于需求确定。预设点云区域可以为一维区域,例如,三维点云图中,位于垂直于预设投影平面且包含预设三维点的直线上的各三维点。或者,预设点云区域也可以为二维区域,例如,三维点云图中位于平行于预设投影平面且包含预设三维点的平面区域中的各三维点,平面区域的形状可以为矩形、圆形等。或者,预设点云区域也可以为三维区域,例如,三维点云图中位于包含预设三维点的长方体、球体内的各三维点等,本实施例中不做具体限定。Preset point cloud areas can be determined based on requirements. The preset point cloud area may be a one-dimensional area, for example, in a three-dimensional point cloud image, each three-dimensional point is located on a straight line perpendicular to the preset projection plane and including the preset three-dimensional point. Alternatively, the preset point cloud area can also be a two-dimensional area. For example, each three-dimensional point in the three-dimensional point cloud image is located in a plane area parallel to the preset projection plane and containing the preset three-dimensional point. The shape of the plane area can be a rectangle, Round etc. Alternatively, the preset point cloud area may also be a three-dimensional area, for example, each three-dimensional point in a cuboid or sphere containing the preset three-dimensional points in the three-dimensional point cloud image, which is not specifically limited in this embodiment.
基于不同的统计特征可以确定三维点不同的图像特征,在确定预设三维点的图像特征时,可以基于不同的需求,选择不同的特征统计方式。Different image features of three-dimensional points can be determined based on different statistical features. When determining the image features of preset three-dimensional points, different feature statistical methods can be selected based on different needs.
在一个实施例中,预设三维点的图像特征可以包括以下至少之一:用于参与表征图像细节的第一图像特征、用于参与确定图像光滑区域的第二图像特征、以及用于参与表征图像背景噪声的第三图像特征。In one embodiment, the image features of the preset three-dimensional point may include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining smooth areas of the image, and a second image feature used to participate in characterizing image details. The third image feature of image background noise.
示例性地,电子设备基于第一图像特征、第二图像特征和第三图像特征,生成的二维投影图可以体现图像细节和图像光滑区域,并且可以降低图像背景噪声,可以提高生成的二维投影图的质量,以提高对图像中的扫描对象进行识别的准确性。例如,当二维投影图应用于安检场景时,二维投影图能够体现不同的扫描对象的细节。另外,由于人体表面相对比较光滑,因此,得到的二维投影图能够体现人物所在图像区域,进而基于二维投影图进行安检,可以准确的区分二维投影图中的人物,以及人物携带的物品,以确定人物携带的物品是否为危险品,可以提高危险品识别的准确性。For example, based on the first image feature, the second image feature, and the third image feature, the electronic device generates a two-dimensional projection image that can reflect image details and image smooth areas, and can reduce image background noise, and can improve the generated two-dimensional projection image. The quality of the projection image to improve the accuracy of identifying scanned objects in the image. For example, when the two-dimensional projection image is applied to a security inspection scene, the two-dimensional projection image can reflect the details of different scanned objects. In addition, since the surface of the human body is relatively smooth, the obtained two-dimensional projection image can reflect the image area where the person is located, and then security inspection based on the two-dimensional projection image can accurately distinguish the persons in the two-dimensional projection image and the items carried by the person. , to determine whether the items carried by the person are dangerous goods, which can improve the accuracy of dangerous goods identification.
在一个实施例中,在图1的基础上,步骤S102可以包括以下操作中的至少之一,其中,图4作为一种示例,将步骤S102包括的操作均进行示出:In one embodiment, based on Figure 1, step S102 may include at least one of the following operations, where Figure 4 is an example, showing all the operations included in step S102:
S1021:通过方差计算、梯度计算、和拉普拉斯算子中的至少一种特征统计方式,确定三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第一统计特征,作为预设三维点的第一图像特征。S1021: Determine the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among variance calculation, gradient calculation, and Laplacian operator. Statistical features, as the first image features of preset three-dimensional points.
其中,第一邻域所处平面与预设投影平面平行。The plane where the first neighborhood is located is parallel to the preset projection plane.
S1022:通过熵值计算、积分旁瓣比计算、和峰值旁瓣比计算中的至少一种特征统计方式,确定三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第二统计特征,作为预设三维点的第二图像特征。S1022: Determine the amplitude correspondence of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among entropy value calculation, integrated side-lobe ratio calculation, and peak side-lobe ratio calculation. The second statistical feature is used as the second image feature of the preset three-dimensional point.
S1023:通过方差计算、梯度计算、或者拉普拉斯算子中的至少一种特征统计方式,确定三维点云图中处于预设三维点的第二邻域内的三维点的幅值对应的第三统计特征,作为预设三维点的第三图像特征。S1023: Determine the third value corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through variance calculation, gradient calculation, or at least one characteristic statistical method of the Laplacian operator. Statistical features are used as third image features of preset three-dimensional points.
其中,第二邻域所处平面与预设投影平面垂直。Wherein, the plane of the second neighborhood is perpendicular to the preset projection plane.
基于不同的特征统计方式可以得到预设三维点在不同维度的统计特征,基于不同维度的统计特征可以确定预设三维点在不同维度的图像特征。相应的,为了使生成的二维投影图能够体现不同维度的图像特征,便于后续对二维投影图进行图像识别,电子设备可以采用多种特征统计方式,计算处于预设三维点的邻域内的三维点的幅值对应的统计特征。进而可以根据得到的统计特征,确定预设三维点的图像特征。The statistical characteristics of the preset three-dimensional points in different dimensions can be obtained based on different feature statistics methods, and the image characteristics of the preset three-dimensional points in different dimensions can be determined based on the statistical characteristics of different dimensions. Correspondingly, in order to make the generated two-dimensional projection map reflect image features of different dimensions and facilitate subsequent image recognition of the two-dimensional projection map, the electronic device can use a variety of feature statistics methods to calculate the number of points in the neighborhood of the preset three-dimensional point. Statistical characteristics corresponding to the amplitude of three-dimensional points. Then, the image characteristics of the preset three-dimensional points can be determined based on the obtained statistical characteristics.
采用方差计算、梯度计算和拉普拉斯算子等至少一种特征统计方式,确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征(即第一统计特征)。例如利用梯度计算、拉普拉斯算子可以提取扫描对象的边缘特征,利用方差计算得到的方差值能够体现处于预设三维点的邻域内的三维点的幅值的离散度,离散度表示处于预设三维点的邻域内的三维点的幅值的差异性,三维点云图中对应扫描对象的边缘处的三维点的幅值与其他三维点的幅值的差异较大。因此,第一统计特征能够体现三维点云图对应的扫描对象的边缘特征和三维点云图中不同区域的图像细节。将用于参与表征图像细节的第一图像特征作为确定二维投影图过程中的一种评价指标,可以避免因背景杂波掩盖强散射目标中的暗区而引起的图像上的细节损失。Using at least one characteristic statistical method such as variance calculation, gradient calculation, and Laplacian operator to determine the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image (i.e., the first statistical characteristics ). For example, gradient calculation and Laplacian operator can be used to extract the edge features of the scanned object, and the variance value obtained by variance calculation can reflect the dispersion of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points. The dispersion represents The difference in amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional point. The amplitude of the three-dimensional point corresponding to the edge of the scanned object in the three-dimensional point cloud image is greatly different from the amplitude of other three-dimensional points. Therefore, the first statistical feature can reflect the edge features of the scanned object corresponding to the three-dimensional point cloud image and the image details of different areas in the three-dimensional point cloud image. Using the first image feature that participates in characterizing image details as an evaluation index in the process of determining the two-dimensional projection map can avoid the loss of details in the image caused by background clutter covering dark areas in strongly scattering targets.
针对安检场景,可以通过主动式毫米波安检仪获取三维点云图。根据三维点云图的聚焦特性和扫描对象的电磁散射特性可知,在三维点云图中电磁散射较强的目标区域在距离向上包含较少的三维点,距离向为垂直于预设投影平面的方向,即只有距离向上较少的三维点包含目标区域的聚焦信息。距离向上的其余三维点对应其他区域,例如,背景区域或透射区域,其他区域在三维点云图中对应的三维点的幅值,低于目标区域在三维点云图中对应的三维点的幅值,即电磁散射较强的目标区域在三维点云图中对应的三维点的幅值较大,因此,三维点云图中目标区域内的三维点的离散度也较大。For security inspection scenarios, three-dimensional point cloud images can be obtained through active millimeter wave security inspection devices. According to the focusing characteristics of the three-dimensional point cloud image and the electromagnetic scattering characteristics of the scanning object, it can be known that in the three-dimensional point cloud image, the target area with strong electromagnetic scattering contains fewer three-dimensional points in the distance direction, and the distance direction is the direction perpendicular to the preset projection plane. That is, only the three-dimensional points with a smaller distance upward contain the focus information of the target area. The remaining three-dimensional points with a distance upward correspond to other areas, for example, background areas or transmission areas. The amplitudes of the corresponding three-dimensional points in other areas in the three-dimensional point cloud map are lower than the amplitudes of the corresponding three-dimensional points in the three-dimensional point cloud map of the target area. That is, the target area with strong electromagnetic scattering has a larger amplitude of the corresponding three-dimensional point in the three-dimensional point cloud diagram. Therefore, the dispersion of the three-dimensional points in the target area in the three-dimensional point cloud diagram is also larger.
因此,为了避免由于背景区域掩盖目标区域中较暗的区域,导致生成的二维投影图的图像细节损失, 可以采用方差计算、梯度计算和拉普拉斯算子中的至少一种特征统计方式,计算三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第一统计特征,作为预设三维点云图的第一图像特征。由于第一图像特征用于参与表征图像细节,基于第一图像特征得到的二维投影图能够体现扫描对象的细节,即可以提高生成的二维投影图的质量,方便后续对二维投影图进行图像识别。Therefore, in order to avoid the loss of image details in the generated two-dimensional projection map due to the background area covering up the darker areas in the target area, at least one characteristic statistical method among variance calculation, gradient calculation and Laplacian operator can be used. , calculate the first statistical feature corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, as the first image feature of the preset three-dimensional point cloud image. Since the first image feature is used to participate in representing image details, the two-dimensional projection map obtained based on the first image feature can reflect the details of the scanned object, that is, the quality of the generated two-dimensional projection map can be improved, and subsequent processing of the two-dimensional projection map can be facilitated. Image Identification.
采用熵值计算、峰值旁瓣比计算和积分旁瓣比计算等至少一种特征统计方式,确定三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的统计特征(即第二统计特征),能够体现处于预设三维点的第一邻域内的三维点的幅值的有序度和聚焦度,有序度表示预设三维点的邻域内的三维点的幅值的变化趋势,例如,平滑递增或平滑递减等。聚焦度表示预设三维点的邻域内的三维点的幅值的集中程度。三维点云图中扫描对象中光滑区域对应的三维点的幅值的集中程度较高,且为平滑的变化趋势,因此,第二统计特征能够体现三维点云图对应的扫描对象中不同区域的光滑程度。Using at least one characteristic statistical method such as entropy value calculation, peak side lobe ratio calculation, and integral side lobe ratio calculation, the statistical characteristics corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image are determined ( That is, the second statistical characteristic), which can reflect the degree of order and focus of the amplitudes of the three-dimensional points in the first neighborhood of the preset three-dimensional points. The degree of order represents the amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional points. The changing trend, for example, smooth increase or smooth decrease, etc. The degree of focus represents the degree of concentration of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional point. In the three-dimensional point cloud image, the amplitudes of the three-dimensional points corresponding to the smooth areas in the scanned object have a high degree of concentration and a smooth changing trend. Therefore, the second statistical feature can reflect the smoothness of different areas in the scanned object corresponding to the three-dimensional point cloud image. .
针对安检场景,扫描对象可以为人物,由于人体表面较为光滑,人体表面在三维点云图中对应的各三维点的幅值的有序度较高。因此,为了使得生成的二维投影图可以体现扫描对象的光滑区域,可以采用熵值计算、积分旁瓣比计算、和峰值旁瓣比计算中的至少一种特征统计方式,计算三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第二统计特征,作为预设三维点的第二图像特征。由于第二图像特征用于参与确定图像光滑区域,基于第二图像特征得到的二维投影图,能够体现扫描对象的光滑区域,即可以提高生成的二维投影图的质量,方便后续对二维投影图进行图像识别。For security inspection scenarios, the scanning object can be a person. Since the surface of the human body is relatively smooth, the amplitude of the corresponding three-dimensional points on the human body surface in the three-dimensional point cloud image has a high degree of order. Therefore, in order to make the generated two-dimensional projection image reflect the smooth area of the scanned object, at least one of the characteristic statistical methods of entropy value calculation, integral side-lobe ratio calculation, and peak side-lobe ratio calculation can be used to calculate the three-dimensional point cloud image. The second statistical feature corresponding to the amplitude of the three-dimensional point located in the first neighborhood of the preset three-dimensional point is used as the second image feature of the preset three-dimensional point. Since the second image feature is used to determine the smooth area of the image, the two-dimensional projection map obtained based on the second image feature can reflect the smooth area of the scanned object, that is, it can improve the quality of the generated two-dimensional projection map and facilitate subsequent two-dimensional Projection images for image recognition.
采用方差计算、梯度计算和拉普拉斯算子等至少一种特征统计方式,确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征(即第三统计特征),能够体现处于预设三维点的第二邻域内的三维点的幅值的离散度,离散度表示处于预设三维点的邻域内的三维点的幅值的差异性。例如,三维点云图中背景区域内各三维点的幅值,与三维点云图中包含扫描对象的目标区域内各三维点的幅值的差异较大,因此,第三统计特征能够用于区分背景区域与包含扫描对象的目标区域。Using at least one characteristic statistical method such as variance calculation, gradient calculation, and Laplacian operator, the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image (i.e., the third statistical characteristics ), can reflect the dispersion of the amplitudes of the three-dimensional points in the second neighborhood of the preset three-dimensional point, and the dispersion represents the difference in the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points. For example, the amplitude of each three-dimensional point in the background area in the three-dimensional point cloud image is greatly different from the amplitude of each three-dimensional point in the target area containing the scanned object in the three-dimensional point cloud image. Therefore, the third statistical feature can be used to distinguish the background area and the target area containing the scanned object.
针对安检场景,通过主动式毫米波安检仪获取的三维点云图中电磁散射较强的目标区域在距离向上包含较少的三维点,且包含的三维点的幅值的变化较大。背景区域在距离向包含的三维点的幅值的变化较小。因此,三维点云图中目标区域与背景区域内的三维点的幅值的差异较大。For security inspection scenarios, the target area with strong electromagnetic scattering in the 3D point cloud image obtained by the active millimeter wave security inspection instrument contains fewer 3D points in the distance direction, and the amplitude of the contained 3D points changes greatly. The background region has smaller changes in the magnitude of the three-dimensional points it contains in the distance direction. Therefore, the difference in amplitude of the three-dimensional points in the target area and the background area in the three-dimensional point cloud image is large.
因此,为了使生成的二维投影图能够体现背景区域与包含扫描对象的目标区域,降低背景区域对包含扫描对象的目标区域的影响,可以采用方差计算、梯度计算、或者拉普拉斯算子中的至少一种特征统计方式,计算三维点云图中处于预设三维点的第二邻域内的三维点的幅值对应的第三统计特征,作为预设三维点的第三图像特征。由于第三图像特征用于参与表征图像背景噪声,基于第三图像特征得到的二维投影图,能够降低图像背景噪声,也就能够区分背景区域与包含扫描对象的目标区域,即可以提高生成的二维投影图的质量,方便对二维投影图进行图像识别。Therefore, in order to make the generated two-dimensional projection map reflect the background area and the target area containing the scanning object, and reduce the influence of the background area on the target area containing the scanning object, variance calculation, gradient calculation, or Laplacian operator can be used At least one feature statistics method in the method calculates the third statistical feature corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image as the third image feature of the preset three-dimensional point. Since the third image feature is used to participate in characterizing the image background noise, the two-dimensional projection map obtained based on the third image feature can reduce the image background noise, and can also distinguish the background area from the target area containing the scanned object, that is, the generated image can be improved. The quality of the two-dimensional projection image facilitates image recognition of the two-dimensional projection image.
在确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征时,除了上述列举的特征统计方式外,还可以通过自定义函数实现,以实现保留图像细节、确定图像光滑区域,以及确定图像背景噪声等效果。自定义函数的具体形式可以根据具体处理需求而定。例如,自定义函数为幂次函数,电子设备可以基于幂次函数,计算三维点云图中处于预设三维点的邻域内的三维点的幅值的幂次和,作为预设三维点的图像特征。相应的,得到的图像特征可以表征处于预设三维点的邻域内的三维点的幅值与图像背景噪声的平均幅值的差异性,从而实现在生成二维投影图中保留图像细节。When determining the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, in addition to the characteristic statistical methods listed above, it can also be implemented through custom functions to retain image details and determine Smooth areas of the image, and determine effects such as image background noise. The specific form of the custom function can be determined according to specific processing requirements. For example, if the custom function is a power function, the electronic device can calculate the power sum of the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image based on the power function, as the image feature of the preset three-dimensional point. . Correspondingly, the obtained image features can characterize the difference between the amplitude of the three-dimensional points in the neighborhood of the preset three-dimensional point and the average amplitude of the image background noise, thereby retaining image details in the generated two-dimensional projection map.
在一个实施例中,在图1的基础上,步骤S103可以包括以下操作中的至少之一,其中,图5作为一种示例,将步骤S103包括的操作均进行示出:In one embodiment, based on Figure 1, step S103 may include at least one of the following operations, where Figure 5 is an example, showing all the operations included in step S103:
S1031:将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为预设直线方向上的预设三维点在预设投影平面上的第一投影特征。S1031: Determine the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
S1032:将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为预设直线方向上的预设三维点在预设投影平面上的第二投影特征。S1032: Determine the feature with the largest value among the second image features of the preset three-dimensional point in the preset straight line direction as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
S1033:将处于预设直线方向上的预设三维点的第三图像特征,确定为预设直线方向上的预设三维点在预设投影平面上的第三投影特征。S1033: Determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
上述预设直线方向与预设投影平面垂直,确定第三图像特征过程中考虑的第二邻域所处平面与预设投影平面垂直,因此,当第二邻域为一维邻域,针对同一个预设三维点而言,其第二邻域与其所在的预设直线平行或重合,针对完全重合的情况(第二邻域内的各个预设三维点即预设直线上的各个预设三维点),计算得到的统计特征或第三图像特征即同一值。也就是说,此时各个预设三维点的第二邻域内的三维点的幅值对应的第三统计特征相同,相应的,各个预设三维点的第三图像特征也相同。因此,无需确定第三图像特征中的最大值,可以直接将预设三维点的第三图像特征确定为第三投影特征。The above-mentioned preset straight line direction is perpendicular to the preset projection plane, and the plane of the second neighborhood considered in the process of determining the third image characteristics is perpendicular to the preset projection plane. Therefore, when the second neighborhood is a one-dimensional neighborhood, for the same For a preset three-dimensional point, its second neighborhood is parallel to or coincides with the preset straight line. For a completely coincident situation (each preset three-dimensional point in the second neighborhood is each preset three-dimensional point on the preset straight line). ), the calculated statistical feature or third image feature is the same value. That is to say, at this time, the third statistical features corresponding to the amplitudes of the three-dimensional points in the second neighborhood of each preset three-dimensional point are the same, and correspondingly, the third image features of each preset three-dimensional point are also the same. Therefore, there is no need to determine the maximum value in the third image feature, and the third image feature of the preset three-dimensional point can be directly determined as the third projection feature.
进而,基于得到的预设投影平面上的投影特征,可以确定三维点云图在预设投影平面上的二维投影图。即在一种可选实施例中,根据预设投影平面上的投影特征,确定三维点云图在预设投影平面上的二维投影图,包括:基于预设投影平面上的第一投影特征、第二投影特征、和第三投影特征中的至少之一,确定三维点云图在预设投影平面上的二维投影图。具体地,可以根据对二维投影图的图像特点或图像效果的生成需求,确定参与形成二维投影图的投影特征。Furthermore, based on the obtained projection characteristics on the preset projection plane, the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane can be determined. That is, in an optional embodiment, determining the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the projection characteristics on the preset projection plane includes: based on the first projection characteristics on the preset projection plane, At least one of the second projection feature and the third projection feature determines the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane. Specifically, the projection characteristics involved in forming the two-dimensional projection image can be determined based on the generation requirements for the image characteristics or image effects of the two-dimensional projection image.
在一个实施例中,可以同时基于预设投影平面上的第一投影特征、第二投影特征、和第三投影特征,确定三维点云图在预设投影平面上的二维投影图,即如图5所示,步骤S104可以包括以下步骤:In one embodiment, the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane can be determined based on the first projection feature, the second projection feature, and the third projection feature on the preset projection plane at the same time, as shown in the figure As shown in 5, step S104 may include the following steps:
S1041:基于预设投影平面上的第一投影特征、第二投影特征、第三投影特征,确定三维点云图在预设投影平面上的二维投影图。S1041: Based on the first projection feature, the second projection feature, and the third projection feature on the preset projection plane, determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
示例性的,第一图像特征中值最大的特征可以是幅值离散度最大时对应的特征,离散度体现的是三维点的幅值之间的差异性,而扫描对象的边缘处与其相邻位置的幅值的差异较大,因此,第一图像特征中值最大的特征对应的预设三维点也就是扫描对象的边缘对应的三维点。进而,基于第一图像特征,也就能够有效地确定出三维点云图中扫描对象的边缘,以丰富生成的二维投影图的图像细节。For example, the feature with the largest value among the first image features may be the feature corresponding to the maximum amplitude dispersion. The dispersion reflects the difference between the amplitudes of the three-dimensional points, and the edges of the scanned objects are adjacent to them. The difference in amplitude of the position is large. Therefore, the preset three-dimensional point corresponding to the feature with the largest value among the first image features is also the three-dimensional point corresponding to the edge of the scanned object. Furthermore, based on the first image feature, the edge of the scanned object in the three-dimensional point cloud image can be effectively determined to enrich the image details of the generated two-dimensional projection image.
因此,为了使生成的二维投影图能够体现扫描对象的细节,电子设备可以将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为预设直线方向上的预设三维点在预设投影平面上的第一投影特征。Therefore, in order to make the generated two-dimensional projection map reflect the details of the scanned object, the electronic device can determine the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the feature in the preset straight line direction. The first projection feature of the preset three-dimensional point on the preset projection plane.
示例性的,第二图像特征中值最大的特征可以是幅值有序度最大时对应的特征,有序度能够体现三维点云图对应的扫描对象中不同区域的光滑程度,有序度最大时的图像特征对应的预设三维点所在区域为光滑程度最大的图像区域,为了使生成的二维投影图能体现扫描对象的光滑区域,电子设备可以将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为预设直线方向上的预设三维点在预设投影平面上的第二投影特征。For example, the feature with the largest value among the second image features may be the feature corresponding to the maximum amplitude order degree. The order degree can reflect the smoothness of different areas in the scanned object corresponding to the three-dimensional point cloud image. When the order degree is the maximum The area where the preset three-dimensional point corresponding to the image feature is located is the image area with the greatest smoothness. In order to make the generated two-dimensional projection map reflect the smooth area of the scanned object, the electronic device can place the preset three-dimensional point in the preset straight line direction. The feature with the largest value among the second image features is determined as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
预设三维点的第三图像特征用于参与确定图像背景噪声,为了降低图像背景噪声对包含扫描对象的目标区域的影响,使生成的二维投影图能够区分背景区域与包含扫描对象的目标区域,电子设备可以将处于预设直线方向上的预设三维点的第三图像特征,确定为预设直线方向上的预设三维点在预设投影平面上的第三投影特征。The third image feature of the preset three-dimensional point is used to participate in determining the image background noise. In order to reduce the impact of the image background noise on the target area containing the scanned object, the generated two-dimensional projection map can distinguish the background area from the target area containing the scanned object. , the electronic device can determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane.
进而,电子设备可以采用不同的方式,基于预设投影平面上的第一投影特征、第二投影特征和第三 投影特征,生成三维点云图在预设投影平面上的二维投影图。Furthermore, the electronic device can use different methods to generate a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the first projection feature, the second projection feature, and the third projection feature on the preset projection plane.
另一种实现方式中,基于预设投影平面上的第一投影特征、第二投影特征、和第三投影特征中的至少之一,确定三维点云图在预设投影平面上的二维投影图,包括:In another implementation, the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined based on at least one of the first projection feature, the second projection feature, and the third projection feature on the preset projection plane. ,include:
确定预设投影平面上的第一投影特征对应的预设三维点在三维点云图中的第一像素值,并基于第一像素值生成第一投影子图;Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate the first projection sub-image based on the first pixel value;
确定预设投影平面上的第二投影特征对应的预设三维点在三维点云图中的第二像素值,并基于第二像素值生成第二投影子图;Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value;
将预设投影平面上的第三投影特征作为第三像素值,并基于第三像素值生成第三投影子图;Use the third projection feature on the preset projection plane as the third pixel value, and generate a third projection sub-image based on the third pixel value;
对第一投影子图、第二投影子图和第三投影子图中的至少两种进行融合处理,得到三维点云图对应的二维投影图。具体地,可以根据对二维投影图的图像特点或图像效果的生成需求,确定参与形成二维投影图的投影特征,进而基于确定的投影特征,确定需要生成投影子图,最终通过图像融合处理生成需求的二维投影图。Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image. Specifically, according to the image characteristics or image effect generation requirements of the two-dimensional projection image, the projection characteristics participating in the formation of the two-dimensional projection image can be determined, and then based on the determined projection characteristics, it is determined that the projection sub-image needs to be generated, and finally through image fusion processing Generate a two-dimensional projection of the requirements.
作为本申请实施例的一种示例,参见图6,示出了对第一投影子图、第二投影子图和第三投影子图进行融合处理,得到三维点云图对应的二维投影图,即步骤S1041可以包括以下步骤:As an example of the embodiment of the present application, see Figure 6, which shows the fusion process of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image. That is, step S1041 may include the following steps:
S10411:确定预设投影平面上的第一投影特征对应的预设三维点在三维点云图中的第一像素值,并基于第一像素值生成第一投影子图。S10411: Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate the first projection sub-image based on the first pixel value.
S10412:确定预设投影平面上的第二投影特征对应的预设三维点在三维点云图中的第二像素值,并基于第二像素值生成第二投影子图。S10412: Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value.
S10413:将预设投影平面上的第三投影特征作为第三像素值,并基于第三像素值生成第三投影子图。S10413: Use the third projection feature on the preset projection plane as the third pixel value, and generate the third projection sub-image based on the third pixel value.
S10414:对第一投影子图、第二投影子图和第三投影子图进行融合处理,得到三维点云图对应的二维投影图。S10414: Fusion process the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
一个预设直线对应于预设投影平面中的一个像素点,根据投影特征反向确定该预设直线上的预设三维点,该预设三维点所对应的像素值也就是预设投影平面中该像素点的像素值。相应的,确定各预设直线方向上的预设三维点所对应的像素值,也就是确定预设投影平面中的各像素点的像素值,也就可以得到对应的投影子图。A preset straight line corresponds to a pixel point in the preset projection plane. The preset three-dimensional point on the preset straight line is reversely determined according to the projection characteristics. The pixel value corresponding to the preset three-dimensional point is also the pixel value in the preset projection plane. The pixel value of this pixel. Correspondingly, by determining the pixel values corresponding to the preset three-dimensional points in each preset straight line direction, that is, determining the pixel values of each pixel point in the preset projection plane, the corresponding projection sub-image can be obtained.
一个预设三维点所对应的像素值为该预设三维点在三维点云图中的幅值。The pixel value corresponding to a preset three-dimensional point is the amplitude of the preset three-dimensional point in the three-dimensional point cloud image.
示例性的,针对图2所示的实施例,图2中长方体所示的三维范围表示人物的三维点云图,预设投影平面为XOY平面。预设直线方向为垂直于XOY平面的方向。预设三维点的第一邻域所处的平面平行于XOY平面。Exemplarily, for the embodiment shown in Figure 2, the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character, and the preset projection plane is the XOY plane. The default straight line direction is perpendicular to the XOY plane. It is preset that the plane where the first neighborhood of the three-dimensional point is located is parallel to the XOY plane.
针对预设直线方向上的每一预设三维点,该预设三维点在三维坐标系中的位置可以表示为(x,y,z)。由于各三维点均位于垂直于XOY平面的预设直线方向上,各三维点的x和y相同。For each preset three-dimensional point in the preset straight line direction, the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z). Since each three-dimensional point is located in the preset straight line direction perpendicular to the XOY plane, the x and y of each three-dimensional point are the same.
相应的,第一投影特征对应的预设三维点可以记为(x,y,z
1)。第一投影特征对应的预设三维点在三维点云图中的第一像素值为:该预设三维点在三维点云图中的幅值,则第一投影特征对应的预设三维点对应的第一像素值σ
1可以表示为:
Correspondingly, the preset three-dimensional point corresponding to the first projection feature can be recorded as (x, y, z 1 ). The first pixel value of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image is: the amplitude of the preset three-dimensional point in the three-dimensional point cloud image, then the first pixel value corresponding to the preset three-dimensional point corresponding to the first projection feature A pixel value σ 1 can be expressed as:
σ
1=σ(x,y,z
1) (1)
σ 1 =σ(x,y,z 1 ) (1)
σ(x,y,z
1)表示第一投影特征对应的预设三维点在三维点云图中的幅值。
σ(x,y,z 1 ) represents the amplitude of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image.
参见图7a和图7b,图7a为本申请实施例提供的一种第一投影子图。电子设备可以基于第一投影 特征生成如图7a所示的第一投影子图。Referring to Figures 7a and 7b, Figure 7a is a first projection sub-image provided by an embodiment of the present application. The electronic device may generate a first projection sub-image as shown in Figure 7a based on the first projection feature.
图7b中左侧的图像为图7a所示的第一投影子图中背景区域(除人物所在图像区域以外的区域)的放大图。图7b中间的图像为图7a所示的第一投影子图中人物背部所在区域的放大图。图7b中右侧的图像为图7a所示的第一投影子图中人物腿部所在区域的放大图。The image on the left side in Figure 7b is an enlarged view of the background area (areas other than the image area where the character is located) in the first projection sub-image shown in Figure 7a. The middle image of Figure 7b is an enlarged view of the area where the character's back is located in the first projection sub-image shown in Figure 7a. The image on the right side in Figure 7b is an enlarged view of the area where the character's legs are located in the first projection sub-image shown in Figure 7a.
结合图7a和图7b可以得到,图7a和图7b中均包含较多的图像细节。例如,图7b中间的图像中可以体现人物背部区域的不同位置,图7b中右侧的图像中可以体现人物腿部位置放置了扳手。It can be obtained by combining Figure 7a and Figure 7b that both Figure 7a and Figure 7b contain more image details. For example, the image in the middle of Figure 7b can reflect the different positions of the character's back area, and the image on the right side of Figure 7b can reflect that a wrench is placed on the character's leg.
示例性的,针对图2所示的实施例,图2中长方体所示的三维范围表示人物的三维点云图,预设投影平面为XOY平面。预设直线方向为垂直于XOY平面的方向。预设三维点的第一邻域所处的平面平行于XOY平面。Exemplarily, for the embodiment shown in Figure 2, the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character, and the preset projection plane is the XOY plane. The default straight line direction is perpendicular to the XOY plane. It is preset that the plane where the first neighborhood of the three-dimensional point is located is parallel to the XOY plane.
针对预设直线方向上的每一预设三维点,该预设三维点在三维坐标系中的位置可以表示为(x,y,z)。由于各三维点均位于垂直于XOY平面的预设直线方向上,各三维点的x和y相同。For each preset three-dimensional point in the preset straight line direction, the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z). Since each three-dimensional point is located in the preset straight line direction perpendicular to the XOY plane, the x and y of each three-dimensional point are the same.
相应的,第二投影特征对应的预设三维点可以记为(x,y,z
2)。第二投影特征对应的预设三维点在三维点云图中的第二像素值为:该预设三维点在三维点云图中的幅值,则第二投影特征对应的预设三维点对应的第二像素值σ
2可以表示为:
Correspondingly, the preset three-dimensional point corresponding to the second projection feature can be recorded as (x, y, z 2 ). The second pixel value of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is: the amplitude of the preset three-dimensional point in the three-dimensional point cloud image, then the second pixel value corresponding to the preset three-dimensional point corresponding to the second projection feature The two-pixel value σ 2 can be expressed as:
σ
2=σ(x,y,z
2) (2)
σ 2 =σ(x,y,z 2 ) (2)
σ(x,y,z
2)表示第二投影特征对应的预设三维点在三维点云图中的幅值。
σ(x,y,z 2 ) represents the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image.
参见图8a和图8b,图8a为本申请实施例提供的一种第二投影子图。电子设备可以基于第二投影特征生成图8a所示的第二投影子图。Referring to Figures 8a and 8b, Figure 8a is a second projection sub-image provided by an embodiment of the present application. The electronic device may generate the second projection sub-image shown in FIG. 8a based on the second projection feature.
图8b中左侧的图像为图8a所示的第二投影子图中背景区域的放大图。图8b中间的图像为图8a所示的第二投影子图中人物背部所在区域的放大图。图8b中右侧的图像为图8a所示的第二投影子图中人物腿部所在区域的放大图。The image on the left in Figure 8b is an enlarged view of the background area in the second projected sub-image shown in Figure 8a. The middle image of Figure 8b is an enlarged view of the area where the character's back is located in the second projection sub-image shown in Figure 8a. The image on the right side in Figure 8b is an enlarged view of the area where the character's legs are located in the second projection sub-image shown in Figure 8a.
结合图8a和图8b可以得到,图8a和图8b均可以体现图像光滑区域,即可以体现扫描对象的光滑程度。例如,由于人体表面的光滑程度与其他物品(例如,人物携带的扳手)的光滑程度不同,图8a中包含扫描对象的区域与背景区域的亮度不同。图8b中间的图像中各位置的亮度相差较小。图8b中右侧的图像中人物腿部所在区域与人物携带的扳手所在区域的亮度不同。It can be obtained by combining Figure 8a and Figure 8b that both Figure 8a and Figure 8b can reflect the smooth area of the image, that is, they can reflect the smoothness of the scanned object. For example, since the smoothness of the human body surface is different from that of other items (e.g., a wrench carried by the character), the area containing the scanned object in Figure 8a has a different brightness than the background area. In the middle image of Figure 8b, the brightness difference at each position is small. In the image on the right side of Figure 8b, the brightness of the area where the character's legs are located and the area where the wrench carried by the character is located are different.
示例性的,针对图2所示的实施例,图2中长方体所示的三维范围表示人物的三维点云图,预设投影平面为XOY平面。预设直线方向为垂直于XOY平面的方向。预设三维点的第二邻域所处的平面垂直于XOY平面。Exemplarily, for the embodiment shown in Figure 2, the three-dimensional range shown by the cuboid in Figure 2 represents the three-dimensional point cloud image of the character, and the preset projection plane is the XOY plane. The default straight line direction is perpendicular to the XOY plane. The plane where the second neighborhood of the preset three-dimensional point is located is perpendicular to the XOY plane.
针对预设直线方向上的每一预设三维点,该预设三维点在三维坐标系中的位置可以表示为(x,y,z)。相应的,第三投影特征对应的预设三维点对应的第三像素值σ
3可以表示为:
For each preset three-dimensional point in the preset straight line direction, the position of the preset three-dimensional point in the three-dimensional coordinate system can be expressed as (x, y, z). Correspondingly, the third pixel value σ 3 corresponding to the preset three-dimensional point corresponding to the third projection feature can be expressed as:
σ
3=f(σ(x,y,z)) (3)
σ 3 =f(σ(x,y,z)) (3)
f()表示目标函数,目标函数用于计算预设三维点的第三图像特征。σ(x,y,z)表示三维点云图中处于预设三维点的邻域内的三维点的幅值。f() represents the objective function, which is used to calculate the third image feature of the preset three-dimensional point. σ(x,y,z) represents the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image.
参见图9a和图9b,图9a为本申请实施例提供的一种第三投影子图。电子设备可以基于第三投影特征生成图9a所示的第三投影子图。Referring to Figures 9a and 9b, Figure 9a is a third projection sub-image provided by an embodiment of the present application. The electronic device may generate the third projection sub-image shown in FIG. 9a based on the third projection feature.
图9b中左侧的图像为图9a所示的第三投影子图中背景区域的放大图。图9b中间的图像为图9a所示的第三投影子图中人物背部所在区域的放大图。图9b中右侧的图像为图9a所示的第三投影子图中人物腿部所在区域的放大图。The image on the left in Figure 9b is an enlarged view of the background area in the third projection sub-image shown in Figure 9a. The middle image of Figure 9b is an enlarged view of the area where the character's back is located in the third projection sub-image shown in Figure 9a. The image on the right side in Figure 9b is an enlarged view of the area where the character's legs are located in the third projection sub-image shown in Figure 9a.
图9a和图9b中能够清晰地显示扫描对象所在的目标区域与背景区域的边界,以及显示扫描对象完整的轮廓,且背景区域中的噪声较少。Figures 9a and 9b can clearly show the boundary between the target area where the scanned object is located and the background area, as well as the complete outline of the scanned object, and there is less noise in the background area.
进而,在得到第一投影子图、第二投影子图和第三投影子图之后,电子设备可以对第一投影子图、第二投影子图和第三投影子图中至少两种进行融合处理,得到三维点云图对应的二维投影图。Furthermore, after obtaining the first projection sub-image, the second projection sub-image and the third projection sub-image, the electronic device can fuse at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image. After processing, the two-dimensional projection image corresponding to the three-dimensional point cloud image is obtained.
示例性地,在一种实现方式中,若第一投影子图、第二投影子图和第三投影子图的数据量级相同,例如,电子设备将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为第一投影特征,并将第一投影特征对应的预设三维点在三维点云图中的幅值作为像素值,得到第一投影子图。将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为第二投影特征,并将第二投影特征对应的预设三维点在三维点云图中的幅值作为像素值,得到第二投影子图。将处于预设直线方向上的预设三维点的第三图像特征中值最大的特征,确定为第三投影特征,并将第三投影特征对应的预设三维点在三维点云图中的幅值作为像素值,得到第三投影子图。相应的,第一投影子图、第二投影子图和第三投影子图均是三维点云图中的三维点的幅值所对应的数据量级。For example, in one implementation, if the data magnitudes of the first projection sub-image, the second projection sub-image and the third projection sub-image are the same, for example, the electronic device will be in a preset three-dimensional direction in the preset straight line direction. The feature with the largest value among the first image features of the point is determined as the first projection feature, and the amplitude of the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image is used as the pixel value to obtain the first projection sub-image. The feature with the largest value in the second image feature of the preset three-dimensional point in the preset straight line direction is determined as the second projection feature, and the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is determined As pixel values, the second projected sub-image is obtained. The feature with the largest median value of the third image feature of the preset three-dimensional point in the preset straight line direction is determined as the third projection feature, and the amplitude of the preset three-dimensional point corresponding to the third projection feature in the three-dimensional point cloud image is determined As pixel values, the third projected sub-image is obtained. Correspondingly, the first projection sub-image, the second projection sub-image and the third projection sub-image are all data magnitudes corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image.
电子设备可以基于预设融合算法直接对第一投影子图、第二投影子图和第三投影子图进行融合处理,得到三维点云图对应的二维投影图。The electronic device can directly perform fusion processing on the first projection sub-image, the second projection sub-image and the third projection sub-image based on the preset fusion algorithm to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
示例性的,电子设备可以按照如下公式,对各投影子图进行融合处理。For example, the electronic device can perform fusion processing on each projection sub-image according to the following formula.
H(x,y)=g(h
1(x,y),h
2(x,y),h
3(x,y)) (4)
H(x,y)=g(h 1 (x, y), h 2 (x, y), h 3 (x, y)) (4)
H(x,y)表示二维投影图中坐标为(x,y)的像素点的像素值。g()表示融合函数。h
1(x,y)表示第一投影子图中坐标为(x,y)的像素点的像素值。h
2(x,y)表示第二投影子图中坐标为(x,y)的像素点的像素值。h
3(x,y)表示第三投影子图中坐标为(x,y)的像素点的像素值。
H(x,y) represents the pixel value of the pixel point with coordinates (x,y) in the two-dimensional projection image. g() represents the fusion function. h 1 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the first projection sub-image. h 2 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the second projection sub-image. h 3 (x, y) represents the pixel value of the pixel point with coordinates (x, y) in the third projection sub-image.
在一个实施例中,电子设备可以对第一投影子图、第二投影子图和第三投影子图中数据量级相同的两个投影子图进行融合处理,得到中间图像。另外,电子设备可以对另一个投影子图进行预处理(例如,归一化处理),并对中间图像与预处理的结果进行融合处理,得到对应的二维投影图。In one embodiment, the electronic device can perform fusion processing on two projection sub-images with the same data magnitude among the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain an intermediate image. In addition, the electronic device can pre-process (for example, normalize) another projection sub-image, and fuse the intermediate image with the pre-processing result to obtain a corresponding two-dimensional projection image.
例如,电子设备将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为第一投影特征,并将第一投影特征对应的预设三维点在三维点云图中的幅值作为像素值,得到第一投影子图。将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为第二投影特征,并将第二投影特征对应的预设三维点在三维点云图中的幅值作为像素值,得到第二投影子图。第一投影子图和第二投影子图均是三维点云图中的三维点的幅值所对应的数据量级。电子设备将处于预设直线方向上的预设三维点的第三图像特征确定为第三投影特征,并将第三投影特征作为像素值,得到第三投影子图。第三投影子图是三维点云图中三维点的幅值的统计特征所对应的数据量级。For example, the electronic device determines the feature with the largest value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature, and adds the preset three-dimensional point corresponding to the first projection feature in the three-dimensional point cloud image. The amplitude in is used as the pixel value to obtain the first projected sub-image. The feature with the largest value in the second image feature of the preset three-dimensional point in the preset straight line direction is determined as the second projection feature, and the amplitude of the preset three-dimensional point corresponding to the second projection feature in the three-dimensional point cloud image is determined As pixel values, the second projected sub-image is obtained. The first projection sub-image and the second projection sub-image are both data magnitudes corresponding to the amplitudes of the three-dimensional points in the three-dimensional point cloud image. The electronic device determines the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature, and uses the third projection feature as the pixel value to obtain the third projection sub-image. The third projection sub-image is the data magnitude corresponding to the statistical characteristics of the amplitude of the three-dimensional points in the three-dimensional point cloud image.
由于第一投影子图与第二投影子图的数据量级相同,电子设备可以对第一投影子图和第二投影子图进行融合处理,得到中间图像。另外,电子设备可以对第三投影子图进行预处理,并对中间图像与预处理的结果进行融合处理,得到对应的二维投影图。Since the data magnitudes of the first projection sub-image and the second projection sub-image are the same, the electronic device can perform fusion processing on the first projection sub-image and the second projection sub-image to obtain an intermediate image. In addition, the electronic device can pre-process the third projection sub-image, and fuse the intermediate image with the pre-processing result to obtain a corresponding two-dimensional projection image.
相应的,在图6的基础上,参见图10,步骤S10414可以包括以下步骤:Correspondingly, based on Figure 6, referring to Figure 10, step S10414 may include the following steps:
S104141:利用预设融合算法,对第一投影子图和第二投影子图进行处理,得到第一中间图像。S104141: Use the preset fusion algorithm to process the first projection sub-image and the second projection sub-image to obtain the first intermediate image.
S104142:对第三投影子图进行归一化处理以及灰度变换处理,得到第二中间图像。S104142: Perform normalization processing and grayscale transformation processing on the third projection sub-image to obtain the second intermediate image.
其中,灰度变换处理用于调整第三投影子图上背景区域与目标区域之间像素值的差异。Among them, the grayscale transformation process is used to adjust the difference in pixel values between the background area and the target area on the third projection sub-image.
S104143:确定第一中间图像和第二中间图像上对应相同位置处的像素值的乘积,并基于乘积生成三维点云图对应的二维投影图。S104143: Determine the product of the pixel values corresponding to the same position on the first intermediate image and the second intermediate image, and generate a two-dimensional projection map corresponding to the three-dimensional point cloud image based on the product.
在本申请实施例中,涉及的融合算法可以包括但不限于加权融合算法、金字塔融合算法和最大值映射算法中的任一种。In the embodiment of this application, the fusion algorithm involved may include but is not limited to any one of the weighted fusion algorithm, the pyramid fusion algorithm, and the maximum mapping algorithm.
例如,上述提及的预设融合算法为加权融合算法,第一投影子图和第二投影子图中对应相同位置处的像素值可以对应不同的权重,电子设备可以按照对应的权重,计算第一投影子图和第二投影子图中对应相同位置处的像素值的加权和,并将计算得到的加权和作为像素值,可以得到第一中间图像。或者,预设融合算法为最大值映射算法,电子设备可以从第一投影子图和第二投影子图中对应相同位置处的像素值中,选择最大的像素值,可以得到第一中间图像。For example, the above-mentioned preset fusion algorithm is a weighted fusion algorithm. The pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image can correspond to different weights. The electronic device can calculate the third projection according to the corresponding weights. The first intermediate image can be obtained by taking the weighted sum of the pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image, and using the calculated weighted sum as the pixel value. Alternatively, the preset fusion algorithm is a maximum mapping algorithm, and the electronic device can select the largest pixel value from the pixel values corresponding to the same position in the first projection sub-image and the second projection sub-image to obtain the first intermediate image.
电子设备还可以对第三投影子图进行归一化处理,示例性的,可以将第三投影子图的像素值归一化至[0,1],以调整第三投影子图的数据量级,使得第三投影子图的数据量级与第一投影子图、第二投影子图的数据量级相差较小。然后,对归一化后的第三投影子图进行灰度变换处理,示例性的,电子设备可以对归一化后的第三投影子图进行线性的灰度变换,也可以基于非线性变换函数(例如,指数函数、对数函数、幂次函数和伽马函数等)对归一化后的第三投影子图进行非线性的灰度变换,以调整归一化后的第三投影子图中不同区域的对比度,从而避免后续图像融合过程中损失图像细节信息,或者保证达到更好的降噪效果。The electronic device can also normalize the third projection sub-image. For example, the pixel value of the third projection sub-image can be normalized to [0, 1] to adjust the data amount of the third projection sub-image. level, so that the data magnitude of the third projection sub-image is smaller than the data magnitude of the first projection sub-image and the second projection sub-image. Then, a grayscale transformation is performed on the normalized third projection subimage. For example, the electronic device can perform a linear grayscale transformation on the normalized third projection subimage, or it can also be based on a nonlinear transformation. The function (for example, exponential function, logarithmic function, power function, gamma function, etc.) performs nonlinear grayscale transformation on the normalized third projection sub-image to adjust the normalized third projection sub-image. The contrast of different areas in the image can avoid the loss of image detail information in the subsequent image fusion process, or ensure a better noise reduction effect.
具体地,通过灰度变换处理,可以降低第三投影子图上背景区域与目标区域之间像素值的差异(即降低不同图像区域的对比度),或者提高第三投影子图上背景区域与目标区域之间像素值的差异(即提高不同图像区域的对比度),这与图像处理需求以及灰度变换处理过程中使用的函数形式有关。例如为了图像融合过程中可以达到较好的降噪效果,可以在灰度变换处理过程中采用指数函数,提高投影子图像中不同区域的对比度;或者,例如为了图像融合过程中可以保留较多的图像细节信息,可以采用对数函数,降低投影子图像中不同区域的对比度。Specifically, through grayscale transformation processing, the difference in pixel values between the background area and the target area on the third projection sub-image can be reduced (that is, the contrast of different image areas is reduced), or the difference between the background area and the target area on the third projection sub-image can be improved. The difference in pixel values between areas (i.e., improving the contrast of different image areas) is related to the image processing requirements and the functional form used in the grayscale transformation process. For example, in order to achieve a better noise reduction effect during the image fusion process, an exponential function can be used in the grayscale transformation process to improve the contrast of different areas in the projected sub-image; or, for example, in order to retain more noise during the image fusion process For image detail information, a logarithmic function can be used to reduce the contrast of different areas in the projected sub-image.
以毫米波安检场景为例,人体边缘部位由于散射角度的原因,回波较弱,如果投影子图像对比度过高,则在后续图像融合过程中可能会损失部分人体边缘信息,因此,通过灰度变换降低图像对比度,可以保证在降低背景噪声的同时尽可能多地保留有用的细节信息。如果投影子图像对比度较低,则会导致图像融合过程中降噪效果一般,因此,可以通过灰度变换提高对比度,从而使得图像融合过程中达到更好的降噪效果。Taking the millimeter wave security inspection scene as an example, the echo at the edge of the human body is weak due to the scattering angle. If the contrast of the projected sub-image is too high, part of the human body edge information may be lost in the subsequent image fusion process. Therefore, through grayscale Transformation reduces image contrast, ensuring that as much useful detail information is retained as possible while reducing background noise. If the contrast of the projected sub-image is low, the noise reduction effect during the image fusion process will be average. Therefore, the contrast can be improved through grayscale transformation, thereby achieving better noise reduction effect during the image fusion process.
对归一化后的第三投影子图进行灰度变换处理,可以调整归一化后的第三投影子图上背景区域与包含扫描对象的目标区域之间像素值的差异,进而使得最终得到的二维投影图中能够明显区分背景区域与包含扫描对象的目标区域。Performing grayscale transformation processing on the normalized third projection sub-image can adjust the difference in pixel values between the background area on the normalized third projection sub-image and the target area containing the scanned object, so that the final result is The two-dimensional projection image can clearly distinguish the background area from the target area containing the scanned object.
进而,电子设备可以计算第一中间图像和第二中间图像上对应相同位置处的像素值的乘积,并基于计算得到的乘积生成三维点云图对应的二维投影图。例如,电子设备可以按照预设归一化区间(例如,0至255)对计算得到的乘积进行归一化处理,将归一化结果作为像素值,可以得到三维点云图对应的二维投影图。Furthermore, the electronic device can calculate a product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image, and generate a two-dimensional projection image corresponding to the three-dimensional point cloud image based on the calculated product. For example, the electronic device can normalize the calculated product according to the preset normalization interval (for example, 0 to 255), and use the normalized result as a pixel value to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image. .
基于上述处理,可以通过预设三维点不同的图像特征,保留图像细节,并确定图像光滑区域,以及确定图像背景噪声,实现降低图像背景噪声对包含扫描对象的目标区域的影响,即实现降噪。相应的,得到的二维投影图可以体现扫描对象的细节和光滑区域,以及能够区分背景区域与包含扫描对象的目标区域,并且背景噪声较低,可以提高生成的二维投影图的质量,后续二维投影图应用于不同的场景时,提高图像的应用效果。Based on the above processing, by presetting different image features of three-dimensional points, image details can be retained, image smooth areas can be determined, and image background noise can be determined to reduce the impact of image background noise on the target area containing the scanned object, that is, to achieve noise reduction. . Correspondingly, the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object. The background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
参见图11,图11为本申请实施例提供的一种图像生成方法的原理示意图。图11中较大的立方体 表示3D点云,3D点云为本申请实施例中的三维点云图。不同统计特征表示本申请实施例中预设三维点不同的图像特征。图11中三个较小的立方体分别表示处于预设直线方向上的三个预设三维点各自的邻域。预设投影平面为图11中较大的立方体的正面,该三个预设三维点的连线(即预设直线)垂直于预设投影平面。Refer to Figure 11, which is a schematic diagram of the principle of an image generation method provided by an embodiment of the present application. The larger cube in Figure 11 represents the 3D point cloud, which is the three-dimensional point cloud image in the embodiment of this application. Different statistical features represent different image features of the preset three-dimensional points in the embodiment of the present application. The three smaller cubes in Figure 11 respectively represent the respective neighborhoods of the three preset three-dimensional points in the preset straight line direction. The preset projection plane is the front of the larger cube in Figure 11, and the line connecting the three preset three-dimensional points (ie, the preset straight line) is perpendicular to the preset projection plane.
电子设备可以获取三维点云图,并基于不同的特征统计方式,分别计算处于预设三维点的邻域内的三维点的幅值对应的统计特征,基于计算得到的统计特征,确定预设三维点不同的图像特征,也就可以得到预设三维点的不同维度的图像特征。然后,电子设备可以基于预设三维点的每一种图像特征,确定该图像特征对应的预设三维点在预设投影平面上的投影特征。进而,电子设备可以根据预设三维点在预设投影平面上的每一投影特征,得到该投影特征对应的投影子图。进而,电子设备可以对各个投影子图进行融合处理,得到三维点云图在预设投影平面上的二维投影图。The electronic device can obtain a three-dimensional point cloud image, and based on different feature statistics methods, calculate the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points. Based on the calculated statistical characteristics, it is determined that the preset three-dimensional points are different. The image features of the preset three-dimensional points can be obtained in different dimensions. Then, the electronic device can determine the projection characteristics of the preset three-dimensional point corresponding to the image feature on the preset projection plane based on each image feature of the preset three-dimensional point. Furthermore, the electronic device can obtain the projection sub-image corresponding to the projection feature based on each projection feature of the preset three-dimensional point on the preset projection plane. Furthermore, the electronic device can fuse each projection sub-image to obtain a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
基于上述处理,可以通过预设三维点不同的图像特征,保留图像细节,并确定图像光滑区域,以及确定图像背景噪声,实现降低图像背景噪声对包含扫描对象的目标区域的影响。相应的,得到的二维投影图可以体现扫描对象的细节和光滑区域,以及能够区分背景区域与包含扫描对象的目标区域,并且背景噪声较低,可以提高生成的二维投影图的质量,后续二维投影图应用于不同的场景时,提高图像的应用效果。Based on the above processing, different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object. Correspondingly, the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object. The background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
参见图12,图12为本申请实施例提供的一种图像生成方法的流程图。Referring to Figure 12, Figure 12 is a flow chart of an image generation method provided by an embodiment of the present application.
电子设备可以获取3D点云,3D点云为本申请实施例中的三维点云图,并确定预设投影视角下的平面为预设投影平面。如果三维点云图与预设投影视角相匹配,例如预设投影平面与三维点云图的三维坐标系中的一个坐标平面相平行,电子设备可以直接进行邻域统计特征计算,也就是按照不同的特征统计方式,分别计算预设三维点不同的图像特征。如果三维点云图与预设投影视角不匹配,例如预设投影平面与三维点云图的三维坐标系中的所有坐标平面均不平行,电子设备可以进行三维旋转,也就是电子设备基于预设投影视角对三维点云图进行坐标转换,以使得进行坐标转换之后,三维点云图的三维坐标系中的一个坐标平面与预设投影平面相平行,得到与预设投影视角相匹配的三维点云图。然后电子设备可以进行邻域统计特征计算,也就是按照不同的特征统计方式,分别计算预设三维点不同的图像特征。The electronic device can obtain the 3D point cloud, which is the three-dimensional point cloud image in the embodiment of the present application, and determine the plane under the preset projection perspective as the preset projection plane. If the three-dimensional point cloud image matches the preset projection perspective, for example, the preset projection plane is parallel to a coordinate plane in the three-dimensional coordinate system of the three-dimensional point cloud image, the electronic device can directly calculate the neighborhood statistical features, that is, according to different features Statistical method, calculate different image features of preset three-dimensional points respectively. If the three-dimensional point cloud image does not match the preset projection perspective, for example, the preset projection plane is not parallel to all coordinate planes in the three-dimensional coordinate system of the three-dimensional point cloud image, the electronic device can perform three-dimensional rotation, that is, the electronic device is based on the preset projection perspective Coordinate conversion is performed on the three-dimensional point cloud image so that after the coordinate conversion, a coordinate plane in the three-dimensional coordinate system of the three-dimensional point cloud image is parallel to the preset projection plane, thereby obtaining a three-dimensional point cloud image that matches the preset projection perspective. The electronic device can then perform neighborhood statistical feature calculations, that is, calculate different image features of preset three-dimensional points according to different feature statistical methods.
进而,电子设备可以进行投影,也就是基于预设三维点不同的图像特征,确定预设三维点在预设投影平面上的不同的投影特征,并基于不同的投影特征,生成该投影特征对应的投影子图。进而,电子设备可以对各投影子图进行融合处理,得到三维点云图在预设投影平面上的二维投影图。Furthermore, the electronic device can perform projection, that is, based on the different image characteristics of the preset three-dimensional point, determine the different projection characteristics of the preset three-dimensional point on the preset projection plane, and based on the different projection characteristics, generate the corresponding projection characteristics. Projected subgraph. Furthermore, the electronic device can perform fusion processing on each projection sub-image to obtain a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane.
基于上述处理,可以通过预设三维点不同的图像特征,保留图像细节,并确定图像光滑区域,以及确定图像背景噪声,实现降低图像背景噪声对包含扫描对象的目标区域的影响。相应的,得到的二维投影图可以体现扫描对象的细节和光滑区域,以及能够区分背景区域与包含扫描对象的目标区域,并且背景噪声较低,可以提高生成的二维投影图的质量,后续二维投影图应用于不同的场景时,提高图像的应用效果。Based on the above processing, different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object. Correspondingly, the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object. The background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
参见图13,图13为本申请实施例提供的一种图像生成方法的流程图。Referring to Figure 13, Figure 13 is a flow chart of an image generation method provided by an embodiment of the present application.
电子设备可以获取3D点云,3D点云为本申请实施例中的三维点云图。然后,电子设备可以按照不同的特征统计方式,分别计算三维点云图中预设三维点不同的图像特征,一个预设三维点的图像特征为:按照不同的特征统计方式,计算得到的以该预设三维点为中心点的邻域内的各三维点的幅值的统计特征。该预设三维点的统计特征包括:统计特征1,统计特征2,……,统计特征n,每一统计特征与一种特征统计方式相对应,n表示特征统计方式的数目,n大于等于1。特征统计方式可以包括:众数 计算、最值计算、方差计算、信杂比计算、熵值计算、卷积核算子和自定义函数等。The electronic device can obtain a 3D point cloud, which is a three-dimensional point cloud image in the embodiment of the present application. Then, the electronic device can calculate different image features of the preset three-dimensional points in the three-dimensional point cloud image according to different feature statistics methods. The image characteristics of a preset three-dimensional point are: According to different feature statistics methods, the calculated image characteristics of the preset three-dimensional point are: Assume that the three-dimensional point is the statistical characteristic of the amplitude of each three-dimensional point in the neighborhood of the central point. The statistical characteristics of the preset three-dimensional point include: statistical characteristics 1, statistical characteristics 2,..., statistical characteristics n. Each statistical characteristic corresponds to a characteristic statistical method. n represents the number of characteristic statistical methods, and n is greater than or equal to 1. . Feature statistical methods can include: mode calculation, maximum value calculation, variance calculation, signal to noise ratio calculation, entropy value calculation, convolution kernel operator and custom function, etc.
然后,针对预设三维点的每一图像特征,电子设备基于该图像特征,确定该图像特征对应的预设三维点在预设投影平面上的投影特征,并基于每一种投影特征,生成该投影特征对应的投影子图。生成的投影子图包括:投影图1,投影图2,……,投影图n。电子设备可以对各投影子图(即投影图1,投影图2,……,投影图n)进行融合处理,得到扫描对象的正视图,扫描对象的正视图也就是三维点云图在主视方向的二维投影图。Then, for each image feature of the preset three-dimensional point, the electronic device determines the projection feature of the preset three-dimensional point corresponding to the image feature on the preset projection plane based on the image feature, and generates the projection feature based on each projection feature. The projected subgraph corresponding to the projected feature. The generated projection sub-images include: projection image 1, projection image 2,..., projection image n. The electronic device can fuse each projection sub-image (i.e., projection image 1, projection image 2,..., projection image n) to obtain a front view of the scanned object. The front view of the scanned object is also a three-dimensional point cloud image in the main viewing direction. 2D projection diagram.
基于上述处理,可以通过预设三维点不同的图像特征,保留图像细节,并确定图像光滑区域,以及确定图像背景噪声,实现降低图像背景噪声对包含扫描对象的目标区域的影响。相应的,得到的二维投影图可以体现扫描对象的细节和光滑区域,以及能够区分背景区域与包含扫描对象的目标区域,并且背景噪声较低,可以提高生成的二维投影图的质量,后续二维投影图应用于不同的场景时,提高图像的应用效果。Based on the above processing, different image features of three-dimensional points can be preset to retain image details, determine image smooth areas, and determine image background noise, thereby reducing the impact of image background noise on the target area containing the scanned object. Correspondingly, the obtained two-dimensional projection image can reflect the details and smooth areas of the scanned object, and can distinguish the background area from the target area containing the scanned object. The background noise is low, which can improve the quality of the generated two-dimensional projection image. Subsequent When the two-dimensional projection map is applied to different scenes, the application effect of the image is improved.
为了更清楚的说明本申请实施例提供的图像生成方法的技术效果,通过图14至图22所示的二维投影图的对比图,对本申请实施例提供的图像生成方法与相关技术的图像生成方法进行对比。相关技术中的图像生成方法为最大值投影方法。最大值投影方法通过以下公式,确定预设直线方向上的预设三维点在预设投影平面上投影的投影特征:In order to more clearly illustrate the technical effects of the image generation method provided by the embodiments of the present application, through the comparison of the two-dimensional projection diagrams shown in Figures 14 to 22, the image generation method provided by the embodiments of the present application and the image generation of related technologies are compared. methods for comparison. The image generation method in the related art is the maximum projection method. The maximum projection method uses the following formula to determine the projection characteristics of a preset three-dimensional point in the preset straight line direction projected on the preset projection plane:
σ
4=max(σ(x,y,x)) (5)
σ 4 =max(σ(x,y,x)) (5)
σ
4表示基于最大值投影方法确定的预设直线方向上的预设三维点在预设投影平面上投影的投影特征,σ(x,y,x)表示预设直线方向上的预设三维点在三维点云图中的幅值,max()表示最大值函数。
σ 4 represents the projection characteristics of the preset three-dimensional point in the preset straight line direction determined based on the maximum projection method on the preset projection plane, and σ (x, y, x) represents the preset three-dimensional point in the preset straight line direction. In the three-dimensional point cloud map, max() represents the maximum value function.
图14至图22中左侧的图像为基于相关技术中的图像生成方法得到的二维投影图,右侧的图像为基于本申请实施例提供的图像生成方法得到的二维投影图。图15、图16和图17为多组局部图像区域的放大图,每一组放大图为图14中两个二维投影图中对应相同位置的局部图像区域的放大图。The image on the left in Figures 14 to 22 is a two-dimensional projection image obtained based on the image generation method in the related art, and the image on the right is a two-dimensional projection image obtained based on the image generation method provided by the embodiment of the present application. Figures 15, 16 and 17 are multiple sets of enlarged views of local image areas. Each set of enlarged views is an enlarged view of the local image area corresponding to the same position in the two two-dimensional projections in Figure 14.
图14中右侧的二维投影图相对于左侧的二维投影图不同区域的边缘更加清楚,并且图14中右侧的二维投影图相对于左侧的二维投影图包含较多的图像细节。The two-dimensional projection on the right in Figure 14 has clearer edges in different areas than the two-dimensional projection on the left, and the two-dimensional projection on the right in Figure 14 contains more edges than the two-dimensional projection on the left. Image details.
图15中的图像为二维投影图中背景区域的放大图。可见,图15中右侧的图像中背景区域相对于左侧的图像中背景区域更暗,也就是右侧的图像中背景区域相对于左侧的图像中背景区域的幅值更低。The image in Figure 15 is an enlarged view of the background area in the two-dimensional projection image. It can be seen that the background area in the image on the right in Figure 15 is darker than the background area in the image on the left, that is, the background area in the image on the right has a lower amplitude than the background area in the image on the left.
图16中的图像为二维投影图中人物膝盖所在区域的放大图。可见,图16中右侧的图像相对于左侧的图像护膝的边缘更加清楚。The image in Figure 16 is an enlarged view of the area where the character's knees are located in the two-dimensional projection. It can be seen that the edge of the knee pad in the image on the right side of Figure 16 is clearer than that in the image on the left side.
图17和图18中的扫描对象为扳手。图17和图18中右侧的图像相对于左侧的图像扳手的边缘更加清楚。并且,图17中右侧的图像相对于左侧的图像包含较多的图像细节,例如,图17中右侧的图像中可以清楚的观察到扳手的头部细节、手柄细节等。The scanning object in Figures 17 and 18 is a wrench. The image on the right in Figures 17 and 18 has a clearer edge of the wrench than the image on the left. Moreover, the image on the right side of Figure 17 contains more image details than the image on the left side. For example, in the image on the right side of Figure 17, the details of the head of the wrench, the details of the handle, etc. can be clearly observed.
图19和图20中的扫描对象为手机和耳机盒。图19和图20中右侧的图像相对于左侧的图像手机和耳机盒的边缘更加清楚。并且,图20中右侧的图像相对于左侧的图像包含较多的图像细节,例如,图20中右侧的图像中可以清楚的观察到扫描对象包括手机。The scanning objects in Figures 19 and 20 are mobile phones and headphone boxes. In Figures 19 and 20, the edges of the mobile phone and headphone box are clearer in the images on the right than in the images on the left. Moreover, the image on the right side of Figure 20 contains more image details than the image on the left side. For example, it can be clearly observed in the image on the right side of Figure 20 that the scanned object includes a mobile phone.
图21中的扫描对象为橡皮泥。毫米波信号在橡皮泥处反射的回波信号,相对于毫米波信号在人体处反射的回波信号较弱。图21中右侧的图像相对于左侧的图像扫描对象的轮廓更清晰,且明暗对比更明显。The scanned object in Figure 21 is plasticine. The echo signal of the millimeter wave signal reflected from the plasticine is weaker than the echo signal of the millimeter wave signal reflected from the human body. The image on the right in Figure 21 has a clearer outline of the scanned object than the image on the left, and the contrast between light and dark is more obvious.
图22中的扫描对象为刀具。图22中右侧的图像相对于左侧的图像中的刀具的边缘更加清楚,并且,图22中右侧的图像相对于左侧的图像包含较多的图像细节。The scanning object in Figure 22 is the tool. The edge of the tool in the image on the right side of Figure 22 is clearer than the image on the left side, and the image on the right side of Figure 22 contains more image details than the image on the left side.
可见,相对于基于相关技术的图像生成方法生成的二维投影图,基于本申请实施例提供的图像生成方法生成的二维投影图中不同区域的边缘更加清楚,即扫描对象的轮廓更加清楚,且包含较多的图像细节,并且可以清楚的区分不同区域,并且背景噪声较低,可以提高生成的二维投影图的质量,后续二维投影图应用于不同的场景时,提高图像的应用效果。It can be seen that compared with the two-dimensional projection image generated by the image generation method based on related technologies, the edges of different areas in the two-dimensional projection image generated based on the image generation method provided by the embodiment of the present application are clearer, that is, the outline of the scanned object is clearer. And it contains more image details, and can clearly distinguish different areas, and the background noise is low, which can improve the quality of the generated two-dimensional projection map. When the subsequent two-dimensional projection map is applied to different scenes, the application effect of the image is improved. .
与图1的方法实施例相对应,参见图23,图23为本申请实施例提供的一种图像生成装置的结构图,以下实施例中未详细解释的内容可以参考上述实施例的描述。如图23所示,所述图像生成装置包括:Corresponding to the method embodiment of Figure 1, see Figure 23, which is a structural diagram of an image generation device provided by an embodiment of the present application. Contents not explained in detail in the following embodiments can refer to the description of the above embodiments. As shown in Figure 23, the image generation device includes:
三维点云图获取模块2301,用于获取三维点云图;The three-dimensional point cloud image acquisition module 2301 is used to obtain the three-dimensional point cloud image;
图像特征获取模块2302,用于确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;The image feature acquisition module 2302 is used to determine the statistical features corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image of the preset three-dimensional points based on the statistical features. Features; wherein, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
投影特征获取模块2303,用于根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;The projection feature acquisition module 2303 is used to determine the projection features of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image features of the preset three-dimensional point in the preset straight line direction; wherein, the The preset straight line direction is perpendicular to the preset projection plane;
二维投影图获取模块2304,用于根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。The two-dimensional projection image acquisition module 2304 is used to determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
在一个可选实施例中,所述预设三维点的图像特征包括以下至少之一:用于参与表征图像细节的第一图像特征、用于参与确定图像光滑区域的第二图像特征、以及用于参与表征图像背景噪声的第三图像特征。In an optional embodiment, the image features of the preset three-dimensional point include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining smooth areas of the image, and The third image feature participates in characterizing the image background noise.
在一个可选实施例中,所述图像特征获取模块2302,具体用于以下至少之一:In an optional embodiment, the image feature acquisition module 2302 is specifically used for at least one of the following:
通过方差计算、梯度计算、和拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第一统计特征,作为所述预设三维点的第一图像特征;其中,所述第一邻域所处平面与所述预设投影平面平行;Through at least one characteristic statistical method among variance calculation, gradient calculation, and Laplacian operator, the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image is determined. Statistical features, as the first image features of the preset three-dimensional point; wherein the plane where the first neighborhood is located is parallel to the preset projection plane;
通过熵值计算、积分旁瓣比计算、和峰值旁瓣比计算中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的所述第一邻域内的三维点的幅值对应的第二统计特征,作为所述预设三维点的第二图像特征;Determine the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among entropy value calculation, integrated side-lobe ratio calculation, and peak side-lobe ratio calculation. The second statistical feature corresponding to the value is used as the second image feature of the preset three-dimensional point;
通过方差计算、梯度计算、或者拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第二邻域内的三维点的幅值对应的第三统计特征,作为所述预设三维点的第三图像特征;其中,所述第二邻域所处平面与所述预设投影平面垂直。Determine the third value corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among variance calculation, gradient calculation, or Laplacian operator. Statistical features, as the third image features of the preset three-dimensional point; wherein the plane where the second neighborhood is located is perpendicular to the preset projection plane.
在一个可选实施例中,所述投影特征获取模块2303,具体用于以下至少之一:In an optional embodiment, the projection feature acquisition module 2303 is specifically used for at least one of the following:
将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第一投影特征;将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第二投影特征;Determine the feature with the largest median value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane; The feature with the largest median value of the second image features of the preset three-dimensional point in the preset straight line direction is determined as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;
将处于预设直线方向上的预设三维点的第三图像特征,确定为所述预设直线方向上的预设三维点在所述预设投影平面上的第三投影特征;Determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;
所述二维投影图获取模块2304,具体用于基于所述预设投影平面上的第一投影特征、第二投影特征和第三投影特征中的至少之一,确定所述三维点云图在所述预设投影平面上的二维投影图。The two-dimensional projection image acquisition module 2304 is specifically configured to determine where the three-dimensional point cloud image is based on at least one of the first projection feature, the second projection feature, and the third projection feature on the preset projection plane. The two-dimensional projection image on the preset projection plane.
在一个可选实施例中,所述二维投影图获取模块2304,具体用于:In an optional embodiment, the two-dimensional projection image acquisition module 2304 is specifically used to:
确定所述预设投影平面上的第一投影特征对应的预设三维点在所述三维点云图中的第一像素值,并基于所述第一像素值生成第一投影子图;Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a first projection sub-image based on the first pixel value;
确定所述预设投影平面上的第二投影特征对应的预设三维点在所述三维点云图中的第二像素值,并基于所述第二像素值生成第二投影子图;Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value;
将所述预设投影平面上的第三投影特征作为第三像素值,并基于所述第三像素值生成第三投影子图;Use the third projection feature on the preset projection plane as a third pixel value, and generate a third projection sub-image based on the third pixel value;
对所述第一投影子图、所述第二投影子图和所述第三投影子图中的至少两种进行融合处理,得到所述三维点云图对应的二维投影图。Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
在一个可选实施例中,所述二维投影图获取模块2304,具体用于:In an optional embodiment, the two-dimensional projection image acquisition module 2304 is specifically used to:
利用预设融合算法,对所述第一投影子图和所述第二投影子图进行处理,得到第一中间图像;Using a preset fusion algorithm, process the first projection sub-image and the second projection sub-image to obtain a first intermediate image;
对所述第三投影子图进行归一化处理以及灰度变换处理,得到第二中间图像;其中,所述灰度变换处理用于调整所述第三投影子图上背景区域与目标区域之间像素值的差异;Perform normalization processing and grayscale transformation processing on the third projection subimage to obtain a second intermediate image; wherein the grayscale transformation process is used to adjust the relationship between the background area and the target area on the third projection subimage. The difference in pixel values between
确定所述第一中间图像和所述第二中间图像上对应相同位置处的像素值的乘积,并基于所述乘积生成所述三维点云图对应的二维投影图。A product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image is determined, and a two-dimensional projection image corresponding to the three-dimensional point cloud image is generated based on the product.
在一个可选实施例中,所述三维点云图获取模块2301,具体用于获取扫描对象的三维点云图;In an optional embodiment, the three-dimensional point cloud image acquisition module 2301 is specifically used to acquire a three-dimensional point cloud image of the scanned object;
所述预设投影平面包括:与所述扫描对象的正面和/或背面平行的投影平面。The preset projection plane includes: a projection plane parallel to the front and/or back of the scanned object.
在一个可选实施例中,所述三维点云图获取模块2301,具体用于基于预设投影视角对原始三维点云图进行旋转,得到与所述预设投影视角匹配的三维点云图;In an optional embodiment, the three-dimensional point cloud image acquisition module 2301 is specifically configured to rotate the original three-dimensional point cloud image based on a preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective;
所述预设投影平面为所述预设投影视角下的投影平面。The preset projection plane is the projection plane under the preset projection viewing angle.
基于本申请实施例提供的图像生成装置,可以确定三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据确定出的统计特征,确定预设三维点的图像特征,并根据处于预设直线方向上的预设三维点的图像特征,确定三维点云图在预设投影平面上的二维投影图,即能够生成三维点云图对应的二维图像。并且,不同的统计特征可以体现不同的图像特征,相应的,针对不同的应用场景,可以基于实际需求采用不同的统计特征,则生成的二维图像能够体现对应的图像特征,以适应于当前的应用场景,即本申请实施例可以提高投影得到的二维图像的质量,进而可以提高图像的应用效果。Based on the image generation device provided by the embodiment of the present application, the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image can be determined, and based on the determined statistical characteristics, the preset three-dimensional points can be determined. Image characteristics, and determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane based on the image characteristics of the preset three-dimensional point in the preset straight line direction, that is, the two-dimensional image corresponding to the three-dimensional point cloud image can be generated. Moreover, different statistical features can reflect different image features. Correspondingly, for different application scenarios, different statistical features can be used based on actual needs, and the generated two-dimensional image can reflect the corresponding image features to adapt to the current situation. Application scenarios, that is, embodiments of the present application can improve the quality of the projected two-dimensional image, thereby improving the application effect of the image.
本申请实施例还提供了一种电子设备,如图24所示,包括处理器2401、通信接口2402、存储器2403和通信总线2404,其中,处理器2401,通信接口2402,存储器2403通过通信总线2404完成相互间的通信,The embodiment of the present application also provides an electronic device, as shown in Figure 24, including a processor 2401, a communication interface 2402, a memory 2403, and a communication bus 2404. The processor 2401, the communication interface 2402, and the memory 2403 communicate through the communication bus 2404. complete mutual communication,
存储器2403,用于存放计算机程序; Memory 2403, used to store computer programs;
处理器2401,用于执行存储器2403上所存放的程序时,实现如下步骤:The processor 2401 is used to implement the following steps when executing the program stored in the memory 2403:
获取三维点云图;Obtain a three-dimensional point cloud image;
确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;Determine the statistical characteristics corresponding to the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional point based on the statistical characteristics; wherein, the three-dimensional point The amplitude of is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;
根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;According to the image characteristics of the preset three-dimensional point in the preset straight line direction, the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane are determined; wherein the preset straight line direction is the same as the preset straight line direction. The default projection plane is vertical;
根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。According to the projection characteristics on the preset projection plane, a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect,PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,EISA)总线等。该通信总线可以分 为地址总线、数据总线、控制总线等。为便于表示,图中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。The communication bus mentioned in the above-mentioned electronic equipment can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
通信接口用于上述电子设备与其他设备之间的通信。The communication interface is used for communication between the above-mentioned electronic devices and other devices.
存储器可以包括随机存取存储器(Random Access Memory,RAM),也可以包括非易失性存储器(Non-Volatile Memory,NVM),例如至少一个磁盘存储器。在一个可选实施例中,存储器还可以是至少一个位于远离前述处理器的存储装置。The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory. In an optional embodiment, the memory may also be at least one storage device located remotely from the aforementioned processor.
上述的处理器可以是通用处理器,包括中央处理器(Central Processing Unit,CPU)、网络处理器(Network Processor,NP)等;还可以是数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processor, DSP), special integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
在本申请提供的又一实施例中,还提供了一种非临时性计算机可读存储介质,该非临时性计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现上述任一图像生成方法的步骤。In yet another embodiment provided by this application, a non-transitory computer-readable storage medium is also provided. The non-transitory computer-readable storage medium stores a computer program, and the computer program is implemented when executed by a processor. The steps for any of the above image generation methods.
在本申请提供的又一实施例中,还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述实施例中任一图像生成方法。In yet another embodiment provided by this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image generation methods in the above embodiments.
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在非临时性计算机可读存储介质中,或者从一个非临时性计算机可读存储介质向另一个非临时性计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述非临时性计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a non-transitory computer-readable storage medium or transmitted from one non-transitory computer-readable storage medium to another. For example, the computer instructions may be transferred from a non-transitory computer-readable storage medium to another. A website, computer, server or data center transmits data to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) Make the transfer. The non-transitory computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), etc.
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.
本说明书中的各个实施例均采用相关的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于装置、电子设备、非临时性计算机可读存储介质以及计算机程序产品实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a related manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus, electronic equipment, non-transitory computer-readable storage medium and computer program product embodiments are described simply because they are basically similar to the method embodiments. For relevant details, please refer to the partial description of the method embodiments. That’s it.
以上所述仅为本申请的较佳实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所做的任何修改、等同替换、改进等,均应包含在本申请保护的范围之内。The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.
Claims (19)
- 一种图像生成方法,所述方法包括:An image generation method, the method includes:获取三维点云图;Obtain a three-dimensional point cloud image;确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;Determine the statistical characteristics corresponding to the amplitude of the three-dimensional point in the neighborhood of the preset three-dimensional point in the three-dimensional point cloud image, and determine the image characteristics of the preset three-dimensional point based on the statistical characteristics; wherein, the three-dimensional point The amplitude of is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;According to the image characteristics of the preset three-dimensional point in the preset straight line direction, the projection characteristics of the preset three-dimensional point in the preset straight line direction on the preset projection plane are determined; wherein the preset straight line direction is the same as the preset straight line direction. The default projection plane is vertical;根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。According to the projection characteristics on the preset projection plane, a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
- 根据权利要求1所述的方法,其中,所述预设三维点的图像特征包括以下至少之一:用于参与表征图像细节的第一图像特征、用于参与确定图像光滑区域的第二图像特征、以及用于参与表征图像背景噪声的第三图像特征。The method according to claim 1, wherein the image features of the preset three-dimensional point include at least one of the following: a first image feature used to participate in characterizing image details, a second image feature used to participate in determining a smooth area of the image , and the third image feature used to participate in characterizing the image background noise.
- 根据权利要求2所述的方法,其中,所述确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征,包括以下至少之一:The method according to claim 2, wherein the statistical characteristics corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image are determined, and the predetermined three-dimensional points are determined based on the statistical characteristics. Assume that the image characteristics of a three-dimensional point include at least one of the following:通过方差计算、梯度计算、和拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第一统计特征,作为所述预设三维点的第一图像特征;其中,所述第一邻域所处平面与所述预设投影平面平行;Through at least one characteristic statistical method among variance calculation, gradient calculation, and Laplacian operator, the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image is determined. Statistical features, as the first image features of the preset three-dimensional point; wherein the plane where the first neighborhood is located is parallel to the preset projection plane;通过熵值计算、积分旁瓣比计算、和峰值旁瓣比计算中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的所述第一邻域内的三维点的幅值对应的第二统计特征,作为所述预设三维点的第二图像特征;Determine the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among entropy value calculation, integrated side-lobe ratio calculation, and peak side-lobe ratio calculation. The second statistical feature corresponding to the value is used as the second image feature of the preset three-dimensional point;通过方差计算、梯度计算、或者拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第二邻域内的三维点的幅值对应的第三统计特征,作为所述预设三维点的第三图像特征;其中,所述第二邻域所处平面与所述预设投影平面垂直。Determine the third value corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among variance calculation, gradient calculation, or Laplacian operator. Statistical features, as the third image features of the preset three-dimensional point; wherein the plane where the second neighborhood is located is perpendicular to the preset projection plane.
- 根据权利要求2所述的方法,其中,所述根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征,包括以下至少之一:The method according to claim 2, wherein the projection of the preset three-dimensional point in the preset straight line direction on the preset projection plane is determined based on the image characteristics of the preset three-dimensional point in the preset straight line direction. Characteristics, including at least one of the following:将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第一投影特征;Determine the feature with the largest median value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第二投影特征;Determine the feature with the largest median value among the second image features of the preset three-dimensional point in the preset straight line direction as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;将处于预设直线方向上的预设三维点的第三图像特征,确定为所述预设直线方向上的预设三维点在所述预设投影平面上的第三投影特征;Determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;所述根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图,包括:Determining the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane includes:基于所述预设投影平面上的第一投影特征、第二投影特征和第三投影特征中的至少之一,确定所述三维点云图在所述预设投影平面上的二维投影图。Based on at least one of the first projection feature, the second projection feature and the third projection feature on the preset projection plane, a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
- 根据权利要求4所述的方法,其中,所述基于所述预设投影平面上的第一投影特征、第二投影特征和第三投影特征中的至少之一,确定所述三维点云图在所述预设投影平面上的二维投影图,包括:The method according to claim 4, wherein the location of the three-dimensional point cloud image is determined based on at least one of the first projection feature, the second projection feature and the third projection feature on the preset projection plane. The two-dimensional projection image on the above-mentioned preset projection plane includes:确定所述预设投影平面上的第一投影特征对应的预设三维点在所述三维点云图中的第一像素值,并基于所述第一像素值生成第一投影子图;Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a first projection sub-image based on the first pixel value;确定所述预设投影平面上的第二投影特征对应的预设三维点在所述三维点云图中的第二像素值,并基于所述第二像素值生成第二投影子图;Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value;将所述预设投影平面上的第三投影特征作为第三像素值,并基于所述第三像素值生成第三投影子图;Use the third projection feature on the preset projection plane as a third pixel value, and generate a third projection sub-image based on the third pixel value;对所述第一投影子图、所述第二投影子图和所述第三投影子图中的至少两种进行融合处理,得到所述三维点云图对应的二维投影图。Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- 根据权利要求5所述的方法,其中,对所述第一投影子图、所述第二投影子图和所述第三投影子图中的至少两种进行融合处理,得到所述三维点云图对应的二维投影图,包括:The method according to claim 5, wherein at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image are fused to obtain the three-dimensional point cloud image. The corresponding two-dimensional projection images include:利用预设融合算法,对所述第一投影子图和所述第二投影子图进行处理,得到第一中间图像;Using a preset fusion algorithm, process the first projection sub-image and the second projection sub-image to obtain a first intermediate image;对所述第三投影子图进行归一化处理以及灰度变换处理,得到第二中间图像;其中,所述灰度变换处理用于调整所述第三投影子图上背景区域与目标区域之间像素值的差异;Perform normalization processing and grayscale transformation processing on the third projection subimage to obtain a second intermediate image; wherein the grayscale transformation process is used to adjust the relationship between the background area and the target area on the third projection subimage. The difference in pixel values between确定所述第一中间图像和所述第二中间图像上对应相同位置处的像素值的乘积,并基于所述乘积生成所述三维点云图对应的二维投影图。A product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image is determined, and a two-dimensional projection image corresponding to the three-dimensional point cloud image is generated based on the product.
- 根据权利要求1所述的方法,其中,所述获取三维点云图,包括:The method according to claim 1, wherein said obtaining a three-dimensional point cloud image includes:获取扫描对象的三维点云图;Obtain a three-dimensional point cloud image of the scanned object;所述预设投影平面包括:与所述扫描对象的正面和/或背面平行的投影平面。The preset projection plane includes: a projection plane parallel to the front and/or back of the scanned object.
- 根据权利要求1所述的方法,其中,所述获取三维点云图,包括:The method according to claim 1, wherein said obtaining a three-dimensional point cloud image includes:基于预设投影视角对原始三维点云图进行旋转,得到与所述预设投影视角匹配的三维点云图;Rotate the original three-dimensional point cloud image based on the preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective;所述预设投影平面为所述预设投影视角下的投影平面。The preset projection plane is the projection plane under the preset projection viewing angle.
- 一种图像生成装置,所述装置包括:An image generating device, the device includes:三维点云图获取模块,用于获取三维点云图;The three-dimensional point cloud image acquisition module is used to obtain the three-dimensional point cloud image;图像特征获取模块,用于确定所述三维点云图中处于预设三维点的邻域内的三维点的幅值对应的统计特征,并依据所述统计特征,确定所述预设三维点的图像特征;其中,所述三维点的幅值用于表征所述三维点对应位置处的电磁散射特性,所述预设三维点的邻域指包括所述预设三维点的预设点云区域;The image feature acquisition module is used to determine the statistical features corresponding to the amplitudes of the three-dimensional points in the neighborhood of the preset three-dimensional points in the three-dimensional point cloud image, and determine the image features of the preset three-dimensional points based on the statistical features. ; Wherein, the amplitude of the three-dimensional point is used to characterize the electromagnetic scattering characteristics at the corresponding position of the three-dimensional point, and the neighborhood of the preset three-dimensional point refers to the preset point cloud area including the preset three-dimensional point;投影特征获取模块,用于根据处于预设直线方向上的预设三维点的图像特征,确定所述预设直线方向上的预设三维点在预设投影平面上的投影特征;其中,所述预设直线方向与所述预设投影平面垂直;The projection feature acquisition module is used to determine the projection features of the preset three-dimensional point in the preset straight line direction on the preset projection plane according to the image features of the preset three-dimensional point in the preset straight line direction; wherein, the The preset straight line direction is perpendicular to the preset projection plane;二维投影图获取模块,用于根据所述预设投影平面上的投影特征,确定所述三维点云图在所述预设投影平面上的二维投影图。A two-dimensional projection image acquisition module is used to determine the two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane according to the projection characteristics on the preset projection plane.
- 根据权利要求9所述的装置,其中,所述预设三维点的图像特征包括以下至少之一:The device according to claim 9, wherein the image characteristics of the preset three-dimensional point include at least one of the following:用于参与表征图像细节的第一图像特征、用于参与确定图像光滑区域的第二图像特征、以及用于参与表征图像背景噪声的第三图像特征。A first image feature is used to participate in characterizing image details, a second image feature is used to participate in determining smooth areas of the image, and a third image feature is used to participate in characterizing image background noise.
- 根据权利要求10所述的装置,其中,所述图像特征获取模块,具体用于以下至少之一:The device according to claim 10, wherein the image feature acquisition module is specifically used for at least one of the following:通过方差计算、梯度计算、和拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第一邻域内的三维点的幅值对应的第一统计特征,作为所述预设三维点的第一图像特征;其中,所述第一邻域所处平面与所述预设投影平面平行;Through at least one characteristic statistical method among variance calculation, gradient calculation, and Laplacian operator, the first value corresponding to the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image is determined. Statistical features, as the first image features of the preset three-dimensional point; wherein the plane where the first neighborhood is located is parallel to the preset projection plane;通过熵值计算、积分旁瓣比计算、和峰值旁瓣比计算中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的所述第一邻域内的三维点的幅值对应的第二统计特征,作为所述预设三维点的 第二图像特征;Determine the amplitude of the three-dimensional point in the first neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among entropy value calculation, integrated side-lobe ratio calculation, and peak side-lobe ratio calculation. The second statistical feature corresponding to the value is used as the second image feature of the preset three-dimensional point;通过方差计算、梯度计算、或者拉普拉斯算子中的至少一种特征统计方式,确定所述三维点云图中处于预设三维点的第二邻域内的三维点的幅值对应的第三统计特征,作为所述预设三维点的第三图像特征;其中,所述第二邻域所处平面与所述预设投影平面垂直。Determine the third value corresponding to the amplitude of the three-dimensional point in the second neighborhood of the preset three-dimensional point in the three-dimensional point cloud image through at least one characteristic statistical method among variance calculation, gradient calculation, or Laplacian operator. Statistical features, as the third image features of the preset three-dimensional point; wherein the plane where the second neighborhood is located is perpendicular to the preset projection plane.
- 根据权利要求10所述的装置,其中,所述投影特征获取模块,具体用于以下至少之一:The device according to claim 10, wherein the projection feature acquisition module is specifically used for at least one of the following:将处于预设直线方向上的预设三维点的第一图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第一投影特征;Determine the feature with the largest median value among the first image features of the preset three-dimensional point in the preset straight line direction as the first projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;将处于预设直线方向上的预设三维点的第二图像特征中值最大的特征,确定为所述预设直线方向上的预设三维点在预设投影平面上的第二投影特征;Determine the feature with the largest median value among the second image features of the preset three-dimensional point in the preset straight line direction as the second projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;将处于预设直线方向上的预设三维点的第三图像特征,确定为所述预设直线方向上的预设三维点在所述预设投影平面上的第三投影特征;Determine the third image feature of the preset three-dimensional point in the preset straight line direction as the third projection feature of the preset three-dimensional point in the preset straight line direction on the preset projection plane;所述二维投影图获取模块,具体用于:The two-dimensional projection map acquisition module is specifically used for:基于所述预设投影平面上的第一投影特征、第二投影特征和第三投影特征中的至少之一,确定所述三维点云图在所述预设投影平面上的二维投影图。Based on at least one of the first projection feature, the second projection feature and the third projection feature on the preset projection plane, a two-dimensional projection image of the three-dimensional point cloud image on the preset projection plane is determined.
- 根据权利要求12所述的装置,其中,所述二维投影图获取模块,具体用于:The device according to claim 12, wherein the two-dimensional projection image acquisition module is specifically used for:确定所述预设投影平面上的第一投影特征对应的预设三维点在所述三维点云图中的第一像素值,并基于所述第一像素值生成第一投影子图;Determine the first pixel value of the preset three-dimensional point corresponding to the first projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a first projection sub-image based on the first pixel value;确定所述预设投影平面上的第二投影特征对应的预设三维点在所述三维点云图中的第二像素值,并基于所述第二像素值生成第二投影子图;Determine the second pixel value of the preset three-dimensional point corresponding to the second projection feature on the preset projection plane in the three-dimensional point cloud image, and generate a second projection sub-image based on the second pixel value;将所述预设投影平面上的第三投影特征作为第三像素值,并基于所述第三像素值生成第三投影子图;Use the third projection feature on the preset projection plane as a third pixel value, and generate a third projection sub-image based on the third pixel value;对所述第一投影子图、所述第二投影子图和所述第三投影子图中的至少两种进行融合处理,得到所述三维点云图对应的二维投影图。Fusion processing is performed on at least two of the first projection sub-image, the second projection sub-image and the third projection sub-image to obtain a two-dimensional projection image corresponding to the three-dimensional point cloud image.
- 根据权利要求13所述的装置,其中,所述二维投影图获取模块,具体用于:利用预设融合算法,对所述第一投影子图和所述第二投影子图进行处理,得到第一中间图像;The device according to claim 13, wherein the two-dimensional projection image acquisition module is specifically configured to use a preset fusion algorithm to process the first projection sub-image and the second projection sub-image to obtain first intermediate image;对所述第三投影子图进行归一化处理以及灰度变换处理,得到第二中间图像;其中,所述灰度变换处理用于调整所述第三投影子图上背景区域与目标区域之间像素值的差异;Perform normalization processing and grayscale transformation processing on the third projection subimage to obtain a second intermediate image; wherein the grayscale transformation process is used to adjust the relationship between the background area and the target area on the third projection subimage. The difference in pixel values between确定所述第一中间图像和所述第二中间图像上对应相同位置处的像素值的乘积,并基于所述乘积生成所述三维点云图对应的二维投影图。A product of pixel values corresponding to the same position on the first intermediate image and the second intermediate image is determined, and a two-dimensional projection image corresponding to the three-dimensional point cloud image is generated based on the product.
- 根据权利要求9所述的装置,其中,所述三维点云图获取模块,具体用于获取扫描对象的三维点云图;所述预设投影平面包括:与所述扫描对象的正面和/或背面平行的投影平面。The device according to claim 9, wherein the three-dimensional point cloud image acquisition module is specifically used to acquire a three-dimensional point cloud image of the scanned object; the preset projection plane includes: parallel to the front and/or back of the scanned object. the projection plane.
- 根据权利要求9所述的装置,其中,所述三维点云图获取模块,具体用于基于预设投影视角对原始三维点云图进行旋转,得到与所述预设投影视角匹配的三维点云图;所述预设投影平面为所述预设投影视角下的投影平面。The device according to claim 9, wherein the three-dimensional point cloud image acquisition module is specifically configured to rotate the original three-dimensional point cloud image based on a preset projection perspective to obtain a three-dimensional point cloud image that matches the preset projection perspective; The preset projection plane is the projection plane under the preset projection angle.
- 一种电子设备,包括处理器、通信接口、存储器和通信总线,其中,处理器,通信接口,存储器通过通信总线完成相互间的通信;存储器,用于存放计算机程序;处理器,用于执行存储器上所存放的程序时,实现权利要求1-8任一所述的方法步骤。An electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, communication interface and memory complete communication with each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the memory When the stored program is loaded on the computer, the method steps described in any one of claims 1-8 are implemented.
- 一种非临时性计算机可读存储介质,所述非临时性计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现权利要求1-8任一所述的方法步骤。A non-transitory computer-readable storage medium. A computer program is stored in the non-transitory computer-readable storage medium. When the computer program is executed by a processor, the method steps described in any one of claims 1-8 are implemented.
- 一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述权利要求1-8任一所述的方法的步骤。A computer program product containing instructions that, when run on a computer, causes the computer to perform the steps of the method described in any one of claims 1-8.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210305072.1A CN114677454B (en) | 2022-03-25 | 2022-03-25 | Image generation method and device |
CN202210305072.1 | 2022-03-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023179011A1 true WO2023179011A1 (en) | 2023-09-28 |
Family
ID=82076155
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/127355 WO2023179011A1 (en) | 2022-03-25 | 2022-10-25 | Image generation method and device |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114677454B (en) |
WO (1) | WO2023179011A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114677454B (en) * | 2022-03-25 | 2022-10-04 | 杭州睿影科技有限公司 | Image generation method and device |
CN115660789B (en) * | 2022-11-23 | 2023-08-04 | 广州锐竞信息科技有限责任公司 | Product image management system based on intelligent electronic commerce platform |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108573231A (en) * | 2018-04-17 | 2018-09-25 | 中国民航大学 | Human bodys' response method based on the Depth Motion figure that motion history point cloud generates |
US20190108654A1 (en) * | 2017-10-06 | 2019-04-11 | Interdigital Vc Holdings, Inc. | Method and apparatus for encoding a point cloud representing three-dimensional objects |
CN111602171A (en) * | 2019-07-26 | 2020-08-28 | 深圳市大疆创新科技有限公司 | Point cloud feature point extraction method, point cloud sensing system and movable platform |
CN114219894A (en) * | 2021-12-15 | 2022-03-22 | 中国科学院空天信息创新研究院 | Three-dimensional modeling method, device, equipment and medium based on chromatography SAR point cloud |
CN114677454A (en) * | 2022-03-25 | 2022-06-28 | 杭州睿影科技有限公司 | Image generation method and device |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107316325B (en) * | 2017-06-07 | 2020-09-22 | 华南理工大学 | Airborne laser point cloud and image registration fusion method based on image registration |
CN110517193B (en) * | 2019-06-28 | 2022-04-12 | 西安理工大学 | Seabed sonar point cloud data processing method |
CN112154446B (en) * | 2019-09-19 | 2024-03-19 | 深圳市大疆创新科技有限公司 | Stereo lane line determining method and device and electronic equipment |
-
2022
- 2022-03-25 CN CN202210305072.1A patent/CN114677454B/en active Active
- 2022-10-25 WO PCT/CN2022/127355 patent/WO2023179011A1/en unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190108654A1 (en) * | 2017-10-06 | 2019-04-11 | Interdigital Vc Holdings, Inc. | Method and apparatus for encoding a point cloud representing three-dimensional objects |
CN108573231A (en) * | 2018-04-17 | 2018-09-25 | 中国民航大学 | Human bodys' response method based on the Depth Motion figure that motion history point cloud generates |
CN111602171A (en) * | 2019-07-26 | 2020-08-28 | 深圳市大疆创新科技有限公司 | Point cloud feature point extraction method, point cloud sensing system and movable platform |
CN114219894A (en) * | 2021-12-15 | 2022-03-22 | 中国科学院空天信息创新研究院 | Three-dimensional modeling method, device, equipment and medium based on chromatography SAR point cloud |
CN114677454A (en) * | 2022-03-25 | 2022-06-28 | 杭州睿影科技有限公司 | Image generation method and device |
Also Published As
Publication number | Publication date |
---|---|
CN114677454A (en) | 2022-06-28 |
CN114677454B (en) | 2022-10-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023179011A1 (en) | Image generation method and device | |
US9521391B2 (en) | Settings of a digital camera for depth map refinement | |
US20160245641A1 (en) | Projection transformations for depth estimation | |
JP2018503066A (en) | Accuracy measurement of image-based depth detection system | |
CN107534731B (en) | Image processing apparatus, image processing method, and program | |
WO2022116104A1 (en) | Image processing method and apparatus, and device and storage medium | |
JP2020067748A (en) | Image processing apparatus, image processing method, and program | |
CN113160328A (en) | External reference calibration method, system, robot and storage medium | |
JP5361758B2 (en) | Image generation method, image generation apparatus, and program | |
CN115147333A (en) | Target detection method and device | |
US11816857B2 (en) | Methods and apparatus for generating point cloud histograms | |
WO2016175043A1 (en) | Image processing device and image processing method | |
US11748908B1 (en) | Systems and methods for generating point-accurate three-dimensional models with point-accurate color information from a non-cosited capture | |
US11830220B1 (en) | Systems and methods for the accurate mapping of in-focus image data from two-dimensional images of a scene to a three-dimensional model of the scene | |
US10306146B2 (en) | Image processing apparatus and image processing method | |
CN113628286B (en) | Video color gamut detection method, device, computing equipment and computer storage medium | |
US11195290B2 (en) | Apparatus and method for encoding in structured depth camera system | |
CN112652056A (en) | 3D information display method and device | |
JPWO2016175046A1 (en) | Image processing apparatus and image processing method | |
JP6741154B2 (en) | Information processing apparatus, information processing method, and program | |
CN112634439A (en) | 3D information display method and device | |
CN111383262A (en) | Occlusion detection method, system, electronic terminal and storage medium | |
US11830127B1 (en) | Systems and methods for generating consistently sharp, detailed, and in-focus three-dimensional models from pixels of two-dimensional images | |
CN116386016B (en) | Foreign matter treatment method and device, electronic equipment and storage medium | |
GB2533450B (en) | Settings of a digital camera for depth map refinement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22933062 Country of ref document: EP Kind code of ref document: A1 |