WO2024001959A1

WO2024001959A1 - Scanning processing method and apparatus, and electronic device and storage medium

Info

Publication number: WO2024001959A1
Application number: PCT/CN2023/102125
Authority: WO
Inventors: 赵斌涛; 江腾飞; 张健; 林忠威
Original assignee: 先临三维科技股份有限公司
Priority date: 2022-06-28
Filing date: 2023-06-25
Publication date: 2024-01-04
Also published as: CN115115593A

Abstract

The embodiments of the present disclosure relate to a scanning processing method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring the current frame image, and performing rigid-body splicing on the current frame image and historical image data, so as to obtain a first splicing result; when an error of the first splicing result is less than a preset first threshold value, performing non-rigid-body verification on the current frame image and the historical image data, so as to obtain a verification result; and when the verification result indicates that the verification is successful, marking the current frame image as an effective scanning image. By using the technical solution, during a non-rigid-body real-time scanning process, non-rigid-body verification can be performed on the current frame image, and when the verification is successful, the current frame image is marked as an effective scanning image, such that non-rigid-body scanning is performed at a relatively high frame rate and efficiency; and subsequent three-dimensional model reconstruction is performed on the basis of the effective scanning image, thereby further improving the precision and reliability of three-dimensional model reconstruction.

Description

Scanning processing method, device, electronic equipment and storage medium

This disclosure requests the priority of the Chinese patent application submitted to the China Patent Office on June 28, 2022, with the application number 202210753949.3 and the invention title "A scanning processing method, device, electronic equipment and storage medium", and its entire content is approved by This reference is incorporated into this disclosure.

Technical field

The present disclosure relates to the field of scanning technology, and in particular, to a scanning processing method, device, electronic equipment and storage medium.

Background technique

At present, scanning technology is developing rapidly. For example, 3D scanners can realize 3D scanning of objects and are widely used in fields such as machinery and medical plastic surgery.

In related technologies, when scanning, it is assumed that the scanning object is a rigid body and non-rigid body factors are not considered, resulting in inaccurate real-time scanning results and thus affecting the subsequent three-dimensional model reconstruction effect.

Contents of the invention

(1) Technical problems to be solved

The technical problem to be solved by this disclosure is to solve the existing problem of inaccurate real-time scanning results due to non-rigid factors of the scanning object, thereby affecting the three-dimensional model reconstruction effect.

(2) Technical solutions

In order to solve the above technical problems, embodiments of the present disclosure provide a scanning processing method, device, electronic device, and storage medium.

In a first aspect, an embodiment of the present disclosure provides a scanning processing method, which method includes:

Obtain the current frame image, perform rigid body splicing on the current frame image and historical image data, and obtain the first splicing result;

When the error of the first splicing result is less than the preset first threshold, perform non-rigid body verification on the current frame image and the historical image data to obtain a verification result;

When the verification result is that the verification is successful, the current frame image is marked as a valid scanned image.

In a second aspect, embodiments of the present disclosure also provide a scanning processing device, which includes:

The first acquisition module is used to acquire the current frame image;

A splicing module, used to rigidly splice the current frame image and historical image data to obtain a first splicing result;

A first verification module, configured to perform non-rigid body verification on the current frame image and the historical image data to obtain a verification result when the error of the first splicing result is less than a preset first threshold;

A marking module, configured to mark the current frame image as a valid scanned image when the verification result is successful.

In a third aspect, embodiments of the present disclosure further provide an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; and the processor, configured to retrieve instructions from the memory. The executable instructions are read and executed to implement the scan processing method provided by the embodiment of the first aspect of the present disclosure.

In a fourth aspect, an embodiment of the present disclosure also provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the scanning processing method provided by the embodiment of the first aspect of the present disclosure.

(3) Beneficial effects

Compared with the existing technology, the above technical solutions provided by the embodiments of the present disclosure have the following advantages:

The scanning processing solution provided by the embodiment of the present disclosure acquires the current frame image, performs rigid body splicing on the current frame image and historical image data, and obtains the first splicing result. When the error of the first splicing result is less than the preset first threshold, Perform non-rigid body verification on the current frame image and historical image data to obtain the verification result. When the verification result is that the verification is successful, the current frame image is marked as a valid scanned image. Using the above technical solution, during the process of non-rigid real-time scanning, the current frame image can be verified as non-rigid body. When the verification is successful, the current frame image will be The frame image is marked as a valid scan image, enabling non-rigid body scanning at a faster frame rate and efficiency, and subsequent 3D model reconstruction based on the valid scan image, further improving the accuracy and reliability of 3D model reconstruction.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those of ordinary skill in the art, It is said that other drawings can be obtained based on these drawings without exerting creative labor.

Figure 1 is a schematic flowchart of a scanning processing method provided by an embodiment of the present disclosure;

Figure 2 is a schematic flowchart of another scanning processing method provided by an embodiment of the present disclosure;

Figure 3 is a schematic structural diagram of a scanning processing device provided by an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than All examples. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without any creative efforts fall within the scope of protection of this disclosure.

FIG. 1 is a schematic flowchart of a scanning processing method provided by an embodiment of the present disclosure. The method can be executed by a scanning processing device, where the device can be implemented using software and/or hardware, and can generally be integrated in an electronic device. As shown in Figure 1, the method includes:

Step 101: Obtain the current frame image, perform rigid body splicing of the current frame image and historical image data, and obtain the first splicing result.

Among them, the current frame image refers to the Nth frame image obtained by continuously moving the camera during the real-time scanning process. N is a positive integer greater than 1, that is, the current frame image can be the 2nd frame image. frame image, 3rd frame image, etc. Historical image data refers to a frame of image that has been effectively scanned before acquiring the current frame image or an image that has been effectively scanned and spliced into multiple frames. For example, the current 3rd frame image means that the scanner has captured the 3rd frame image and determined Two frames of images have been effectively scanned, and the spliced image of the previous two frames can be selected as historical image data.

The first splicing result refers to the result of rigid body splicing of the current frame image and the historical image data, which is the spliced image frame, including the overlapping area between the current frame image and the historical image data.

In the embodiments of the present disclosure, the current frame image and historical image data are rigidly spliced, and there are many ways to obtain the first splicing result. In some embodiments, the same features between the current frame image and historical image data are obtained, The overlapping area is determined based on the same characteristics, and the current frame image and the historical image data are rigidly spliced based on the overlapping area to obtain the first splicing result.

In other embodiments, a matching feature point pair between the current frame image and historical image data is obtained, a transformation matrix is determined based on the matching feature point pair, and the current frame image is transformed based on the transformation matrix and spliced with the historical image data to obtain the first A splicing result.

The above two methods are only examples of rigidly splicing the current frame image and historical image data to obtain the first splicing result. This disclosure does not elaborate on the method of rigidly splicing the current frame image and historical image data to obtain the first splicing result. limit.

Step 102: When the error of the first splicing result is less than the preset first threshold, perform non-rigid body verification on the current frame image and historical image data to obtain the verification result.

Among them, the error of the first splicing result refers to the error between the observation position of the same point in the image frame after splicing and the observation position of the image frame before splicing, that is, the error between the observation positions of the same point in different image frames. . Among them, the observation position is affected by the position and deformation field of the image frame.

In the embodiment of the present disclosure, after obtaining the first splicing result, the error of the first splicing result is calculated, and the error of the first splicing result is compared with the first threshold. When the error of the first splicing result is less than the preset th When a threshold is reached, non-rigid body verification needs to be performed on the current frame image and historical image data. Among them, the first threshold can be selectively set according to the needs of the application scenario, such as adjusting the first threshold based on the accuracy of the scanning device, etc.

It can be understood that rigid splicing means splicing using overlapping areas of images, such as scanning objects. When a person breathes and the scanned chest moves, the overlap area between the current frame image captured and the historical image data will become smaller, so non-rigid verification is required. Therefore, the error in the first stitching result is less than When the preset first threshold is set, non-rigid body verification needs to be performed on the current frame image and historical image data.

In the embodiments of the present disclosure, non-rigid body verification is performed on the current frame image and historical image data, and there are many ways to obtain the verification results. In some embodiments, the frame positions and corresponding frame positions of the current frame image and historical image data are respectively compared. / Or the deformation field is adjusted multiple times. After each adjustment, non-rigid body splicing is performed based on the adjusted current frame image and historical image data. If the adjusted splicing result error is less than a certain threshold, it means that the verification is passed. Otherwise, the verification is deemed to have failed.

In other embodiments, the image frame position between the current frame image and historical image data is adjusted, and rigid body splicing is performed based on the adjusted current frame image and historical image data. If the adjusted splicing result error is less than a certain threshold, , it means the verification passed, otherwise it is considered that the verification failed.

The above two methods are only examples of performing non-rigid body verification on the current frame image and historical image data to obtain the verification results. The embodiments of the present disclosure do not perform non-rigid body verification on the current frame image and historical image data to obtain the verification results. specific restrictions.

Step 103: When the verification result is successful, mark the current frame image as a valid scanned image.

Among them, when the verification result is successful, it means that the current frame image acquired is valid, and the current frame image is further marked as a valid scan image, so that subsequent three-dimensional model reconstruction can be performed based on the valid scan image, further improving the accuracy and reliability of three-dimensional model reconstruction. Improved the scanning effect and efficiency of non-rigid body scanning scenes.

It should be noted that when the error of the first splicing result is greater than or equal to the first threshold or the verification result is verification failure, it is necessary to further perform matching and other processing based on the characteristics of the current frame image and the global characteristics, so as to further determine whether the current frame image retains Or discard it to further improve scanning accuracy.

The scanning processing solution provided by the embodiment of the present disclosure acquires the current frame image, performs rigid body splicing on the current frame image and historical image data, and obtains the first splicing result. When the error of the first splicing result is less than the preset first threshold, the The current frame image and historical image data are Perform non-rigid body verification and obtain the verification result. When the verification result is that the verification is successful, the current frame image is marked as a valid scanned image. Using the above technical solution, during the process of non-rigid real-time scanning, the current frame image can be verified as non-rigid body. When the verification is successful, the current frame image will be marked as a valid scanning image, achieving faster frame rate and efficiency. Non-rigid body scanning and subsequent 3D model reconstruction based on valid scanned images further improve the accuracy and reliability of 3D model reconstruction.

Based on the description of the foregoing embodiments, when the error of the first splicing result is greater than or equal to the first threshold or the verification result is verification failure, it is necessary to further perform matching processing based on the characteristics of the current frame image and the global characteristics, and how to calculate the global characteristics. And the global features are optimized to further ensure scanning efficiency and effect, which will be described in detail below in conjunction with Figure 2.

FIG. 2 is a schematic flowchart of another scanning processing method provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes the above scanning processing method. As shown in Figure 2, the method includes:

Step 201: Obtain the current frame image, perform rigid body splicing of the current frame image and historical image data, and obtain the first splicing result.

It should be noted that step 201 is the same as step 101. For details, please refer to the description of step 101, which will not be described in detail here.

Step 202: When the error of the first splicing result is less than the preset first threshold, the frame position and deformation field corresponding to the current frame image and the historical image data are respectively adjusted according to the preset number of adjustments. After each adjustment, Rigid body splicing is performed based on the adjusted current frame image and historical image data to obtain a second splicing result.

In the embodiment of the present disclosure, the deformation field consists of a small number of control points and is used to control the point cloud to deform. The frame image refers to the depth point cloud and landmark points seen by the camera at a certain position. In the camera coordinate system Texture map, each frame contains an independent deformation field. Frame position refers to the six dimensions corresponding to non-rigid body changes (can rotate and translate).

In the embodiment of the present disclosure, by adjusting the frame position and deformation field, the relationship between image frames can be adjusted, so that the second splicing result is different. The preset number of adjustments can be selected and set according to the needs of the application scenario.

Specifically, the frame position and deformation field of the current frame image and historical image data are cyclically optimized multiple times (because after the position and deformation field change, the correspondence between frames will also change; times is a parameter, for example, taken 5 times), if the error of the second splicing result can be reduced to less than the given second threshold, then the non-rigid body verification is successful, otherwise the non-rigid body verification fails. The second threshold can be set as needed, for example, adjusted according to the accuracy of the scanning device.

In the embodiment of the present disclosure, the adjusted current frame image and historical image data are rigidly spliced to obtain the second splicing result. The current frame image and historical image data are rigidly spliced to obtain the first splicing result in the same way. Specifically, Refer to the detailed description of performing rigid body splicing of the current frame image and historical image data to obtain the first splicing result, which will not be described in detail here.

Step 203: When the error of the second splicing result is less than the preset second threshold, it is determined that the verification is successful, and the current frame image is marked as a valid scanned image.

Step 204: When the error of the second splicing result is greater than or equal to the second threshold, it is determined that the verification fails, and the current frame image is deleted.

In the embodiment of the present disclosure, when the error of the second splicing result is less than the preset second threshold, it means that the non-rigid body verification is successful, that is, the scanning object is a non-rigid body such as a person. When the person breathes, the scanned chest moves. It will affect the obtained stitching result of the current frame image. Therefore, if the non-rigid body verification is successful, the current frame image will be marked as a valid scanned image.

In the embodiment of the present disclosure, when the error of the second splicing result is greater than or equal to the second threshold, it means that the error of the second splicing result after adjusting the relationship between image frames is still relatively large, so it is determined that the verification failed, and the The current frame image is deleted.

Step 205: When the error of the first splicing result is greater than or equal to the first threshold or the verification result is verification failure, obtain the current features of the current frame image, match the current features with the preset stored global features, and obtain the matching result.

Step 206: When the matching result is successful, obtain multiple frame images that successfully match the current frame image, perform non-rigid body verification on the current frame image and the multi-frame image, and when the verification is successful, mark the current frame image as Scan images efficiently.

Step 207: When the matching result is a matching failure, delete the current frame image.

In the embodiment of the present disclosure, the current features may be features calculated from the depth point cloud, landmark points, and texture maps of the current frame image, such as three-dimensional coordinates, color information or reflection intensity information in the point cloud features, characteristics of landmark points, and The characteristics of the texture map, etc., are selected according to the needs of the application scenario.

In the embodiment of the present disclosure, global features refer to a feature set of all frame images that have been marked as scanned images, and redundant features are removed.

In the embodiment of the present disclosure, the current features of the current frame image are obtained, and the current features are matched with the preset stored global features. When the matching result is a successful match, multiple frame images that successfully match the current frame image are obtained, and the current Non-rigid body verification is performed on frame images and multi-frame images, and when the verification is successful, the current frame image is marked as a valid scanned image. Among them, the non-rigid body verification method is the same as the aforementioned non-rigid body verification method for the current frame image and historical image data. Please refer to the specific description of non-rigid body verification for the current frame image and historical image data, which will not be repeated here. Elaborate.

Further, when the verification is successful, the current frame image is marked as a valid scanned image, and when the matching result is a matching failure, the current frame image is deleted.

Specifically, during the real-time scanning process, the current frame image is continuously processed with the same scheme. Taking advantage of the continuous movement of the camera, the current frame image and historical image data are rigidly spliced. If the error of the splicing result is less than the first threshold, then At the position after rigid body splicing, the current frame image and historical image data undergo non-rigid body verification. If the non-rigid body verification passes, the current frame image is tracked successfully and the tracking process ends.

If the error is greater than or equal to the first threshold or the non-rigid body verification fails, the features of the current frame image are calculated and matched with the global features. If the matching fails, the tracking of the current frame image fails and is discarded; if the match is successful, the image with the current frame is selected. All frame images with successful image matching undergo non-rigid body verification. If the non-rigid body verification passes, the current frame is tracked successfully. If the non-rigid body verification fails, the current frame image fails to track and is discarded.

After step 203, step 206 or step 207, step 208 or step 209 or step 210 may be performed.

Step 208: Update the global features based on the current features to obtain the updated global features and store them.

In the embodiment of the present disclosure, after the current frame image is marked as a valid scan image, it means that the valid scan image has been increased and the global features need to be updated. Therefore, the global features are updated based on the current features to obtain the updated global features and store them. .

Specifically, the current features are merged into the global features to obtain updated global features. In order to further improve the accuracy of subsequent processing, the updated global features can also be de-redundant. Store after processing.

Step 209: Acquire all valid scanned images, perform feature extraction on all valid scanned images, obtain initial global features, perform redundant processing on the initial global features, obtain the target global features and store them.

In the embodiments of the present disclosure, all valid scanned images refer to scanned images marked as valid after processing in all the foregoing embodiments. Feature extraction is performed on all valid scanned images to obtain initial global features, and redundancy processing is performed on the initial global features. The global characteristics of the target are obtained and stored, and can be directly called during subsequent matching to further improve the efficiency of scanning processing.

Step 210, obtain all valid scanned images, and adjust the frame positions and deformation fields corresponding to all valid scanned images, so that the error between the observation positions of the same scanned position point in different frame images is less than the preset third threshold; where , the third threshold is smaller than the first threshold.

In the embodiment of the present disclosure, the third threshold can be selectively set according to the needs of the application scenario.

Specifically, by adjusting the frame position and deformation field of the currently marked effective scanned images of all frames, the error between image frames is as small as possible, solving the problem of inability to loop or loopback errors caused by deformation and accumulated errors, and optimizing global features. The function is to change the position and deformation field of the effective scanned image in all frames, which is equivalent to acting as a global deformation field.

The scanning processing solution provided by the embodiment of the present disclosure acquires the current frame image, rigidly splices the current frame image and historical image data to obtain the first splicing result. When the error of the first splicing result is less than the preset first threshold, respectively The frame position and deformation field corresponding to the current frame image and the historical image data are adjusted according to a preset number of adjustments. After each adjustment, rigid body splicing is performed based on the adjusted current frame image and historical image data to obtain For the second splicing result, when the error of the second splicing result is less than the preset second threshold, it is determined that the verification is successful, and the current frame image is marked as a valid scanned image. When the error of the second splicing result is greater than or equal to the second threshold, When it is determined that the verification fails, when the error of the first splicing result is greater than or equal to the first threshold or the verification result is verification failure, the current features of the current frame image are obtained, and the current features are matched with the preset stored global features to obtain a match. As a result, when the matching result is successful, multiple frame images that successfully match the current frame image are obtained, non-rigid body verification is performed on the current frame image and the multi-frame image, and when the verification is successful, the current frame image is marked as valid Scan the image, and when the matching result is a matching failure or when the non-rigid body verification of the current frame image and multi-frame images fails, the current frame image is deleted. Obtain all valid scanned images, perform feature extraction on all valid scanned images, obtain initial global features, perform deduplication processing on the initial global features, obtain the target global features and store them, obtain all valid scanned images, and extract the frames corresponding to all valid scanned images. The position and deformation field are adjusted so that the error between the observation positions of the same scanning position point in different frame images is less than a preset third threshold; wherein the third threshold is less than the first threshold. As a result, non-rigid body scanning can be performed in real time, and global features can be obtained asynchronously to improve real-time scanning efficiency. Optimizing global features can solve the problem of inability to loop or loop errors caused by deformation and accumulated errors, further improving scanning processing efficiency and effect to meet user needs.

FIG. 3 is a schematic structural diagram of a scanning processing device provided by an embodiment of the present disclosure. The device can be implemented by software and/or hardware, and can generally be integrated in electronic equipment. As shown in Figure 3, the device includes:

The first acquisition module 301 is used to acquire the current frame image;

The splicing module 302 is used to rigidly splice the current frame image and historical image data to obtain the first splicing result;

The first verification module 303 is configured to perform non-rigid body verification on the current frame image and the historical image data to obtain a verification result when the error of the first splicing result is less than the preset first threshold;

The marking module 304 is configured to mark the current frame image as a valid scanned image when the verification result is successful.

Optionally, the first verification module 303 is specifically used for:

Adjust the frame position and deformation field corresponding to the current frame image and the historical image data respectively according to a preset number of adjustments;

After each adjustment, perform non-rigid body splicing based on the adjusted current frame image and the historical image data to obtain a second splicing result;

When the error of the second splicing result is less than the preset second threshold, it is determined that the verification is successful;

When the error of the second splicing result is greater than or equal to the second threshold, it is determined that the verification fails.

Optionally, the device also includes:

A second acquisition module, configured to acquire the current characteristics of the current frame image when the error of the first splicing result is greater than or equal to the first threshold or the verification result is a verification failure;

A matching module, used to match the current features with preset stored global features to obtain a matching result;

A third acquisition module, configured to acquire multiple frames of images that successfully match the current frame image when the matching result is a successful match;

The second verification module is configured to perform non-rigid body verification on the current frame image and the multi-frame images, and mark the current frame image as a valid scanned image when the verification is successful.

Optionally, the device also includes:

A deletion module, configured to delete the current frame image when the matching result is a matching failure or when the non-rigid body verification of the current frame image and the multi-frame image fails.

Optionally, the device also includes:

An update module, configured to update the global features based on the current features, obtain and store the updated global features.

Optionally, the device also includes:

The third acquisition module is used to acquire all valid scanned images;

An extraction module, used to extract features from all valid scanned images to obtain initial global features;

The deduplication module is used to deduplicate the initial global features to obtain and store the target global features.

Optionally, the device also includes:

The fourth acquisition module is used to acquire all valid scanned images;

The adjustment module is used to adjust the frame positions and deformation fields corresponding to all valid scanned images, so that the error between the observation positions of the same scan position point in different frame images is less than a preset third threshold; wherein, The third threshold is smaller than the first threshold.

The scan processing device provided by the embodiment of the present disclosure can execute the scan processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

Embodiments of the present disclosure also provide a computer program product, including a computer program/instruction Let the computer program/instruction implement the scanning processing method provided by any embodiment of the present disclosure when executed by the processor.

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring specifically to FIG. 4 below, a schematic structural diagram of an electronic device 400 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 400 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 4 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 4, the electronic device 400 may include a processing device (eg, central processing unit, graphics processor, etc.) 401, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 402 or from a storage device 408. The program in the memory (RAM) 403 executes various appropriate actions and processes. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, ROM 402 and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

Generally, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 407 such as a computer; a storage device 408 including a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 4 illustrates electronic device 400 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network via communication device 409, Either it is installed from the storage device 408 or it is installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the scan processing method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

In some implementations, the client and server can communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can communicate with digital data in any form or medium. Data communications (e.g., communications network) interconnections. Examples of communications networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: acquires the current frame image, performs rigid body splicing of the current frame image and historical image data, and obtains For the first splicing result, when the error of the first splicing result is less than the preset first threshold, non-rigid body verification is performed on the current frame image and the historical image data to obtain the verification result. When the verification result is that the verification is successful, Mark the current frame image as a valid scanned image.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

processor;

memory for storing instructions executable by the processor;

The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the scanning processing methods provided by this disclosure.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, the computer program is used to perform any one of the scanning provided by the present disclosure. Approach.

It should be noted that in this article, relational terms such as “first” and “second” are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these There is no such actual relationship or sequence between entities or operations. Furthermore, the terms "comprises,""comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or also include Elements inherent to such a process, method, article, or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The above descriptions are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the disclosure. Therefore, the present disclosure is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Industrial applicability

The scanning processing method provided by the present disclosure can perform non-rigid body verification on the current frame image during the non-rigid body real-time scanning process. When the verification is successful, the current frame image will be marked as a valid scanning image, achieving a faster frame rate. , efficient non-rigid body scanning, and subsequent 3D model reconstruction based on effective scanned images, further improving the accuracy and reliability of 3D model reconstruction, and has strong industrial practicability.

Claims

A scanning processing method, characterized by including:

Obtain the current frame image, perform rigid body splicing on the current frame image and historical image data, and obtain the first splicing result;

When the error of the first splicing result is less than the preset first threshold, perform non-rigid body verification on the current frame image and the historical image data to obtain a verification result;

When the verification result is that the verification is successful, the current frame image is marked as a valid scanned image.
The scanning processing method according to claim 1, characterized in that, performing non-rigid body verification on the current frame image and the historical image data to obtain the verification result includes:

Adjust the frame position and deformation field corresponding to the current frame image and the historical image data respectively according to a preset number of adjustments;

After each adjustment, perform non-rigid body splicing based on the adjusted current frame image and the historical image data to obtain a second splicing result;

When the error of the second splicing result is less than the preset second threshold, it is determined that the verification is successful;

When the error of the second splicing result is greater than or equal to the second threshold, it is determined that the verification fails.
The scanning processing method according to claim 1 or 2, further comprising:

When the error of the first splicing result is greater than or equal to the first threshold or the verification result is verification failure, obtain the current characteristics of the current frame image;

Match the current features with the preset stored global features to obtain a matching result;

When the matching result is a successful match, obtain multiple frame images that successfully match the current frame image;

Non-rigid body verification is performed on the current frame image and the multi-frame images, and when the verification is successful, the current frame image is marked as a valid scanned image.
The scanning processing method according to claim 3, further comprising:

When the matching result is a matching failure or when the non-rigid body verification of the current frame image and the multi-frame image fails, the current frame image is deleted.
The scan processing method according to claim 3, characterized in that, after marking the current frame image as a valid scan image, it further includes:

The global features are updated based on the current features to obtain updated global features and store them.
The scanning processing method according to any one of claims 1 to 5, further comprising:

Get all valid scanned images;

Perform feature extraction on all valid scanned images to obtain initial global features;

Perform redundancy processing on the initial global features to obtain the target global features and store them.
The scanning processing method according to any one of claims 1 to 5, further comprising:

Get all valid scanned images;

The frame positions and deformation fields corresponding to all valid scanned images are adjusted so that the error between the observation positions of the same scanned position point in different frame images is less than a preset third threshold; wherein the third threshold is less than the first threshold.
A scanning processing device, characterized in that it includes:

The first acquisition module is used to acquire the current frame image;

A splicing module, used to rigidly splice the current frame image and historical image data to obtain a first splicing result;

A first verification module, configured to perform non-rigid body verification on the current frame image and the historical image data to obtain a verification result when the error of the first splicing result is less than a preset first threshold;

A marking module, configured to mark the current frame image as a valid scanned image when the verification result is successful.
An electronic device, characterized in that the electronic device includes:

processor;

memory for storing instructions executable by the processor;

The processor is configured to read the executable instructions from the memory and execute The instructions are used to implement the scanning processing method described in any one of the above claims 1-7.
A computer-readable storage medium, characterized in that the storage medium stores a computer program, and the computer program is used to execute the scanning processing method described in any one of claims 1-7.