KR20170023425A - 3d image coordinate collection system using 2d flat distance sensor - Google Patents

Abstract

The present invention relates to a three-dimensional image coordinate collection system using a two-dimensional plane distance sensor, which is capable of sensing motion and depth. A development device of a core algorithm according to the present invention completes development of an exclusive device by merging development of a driver, reverse engineering of a protocol, decompiling of a descriptor, designing of Windows registry keys depending on the design of a driver road, and designing of a driver service. As a detail item for enhancing performance of a driver, provided is a method of reinforcing accuracy and tracking, wherein a step of reinforcing accuracy includes reverse engineering of raw data, designing of data alignment, development of interpolation/extrapolation, development of a complex application, and development of data error checking, and a method of reinforcing tracking includes analysis of pulse rate data field, analysis of H/W position data, a task for an I/O rate controller, a task for a point merging module, and development of a multiface shot system.

Description

TECHNICAL FIELD [0001] The present invention relates to a three-dimensional image coordinate acquisition system using a two-dimensional plane distance sensor,

The present invention relates to a three-dimensional image coordinate acquisition system using a two-dimensional plane distance sensor. A two-dimensional plane distance sensor capable of motion and depth sensing is fabricated by using a near-infrared laser (IEC 60825-1 standard class 1) and a general CMOS image sensor. Is a device that allows the user to view a three-dimensional image. A near-infrared laser is used to receive reflected and returned information by using a general CMOS image sensor, to decode received light encoding, and to perform parallel computation of sophisticated algorithms to generate a depth image of the shot scene.

The present invention is a core algorithm for collecting and merging 3D data obtained from a mobile 3D scanner device which allows a person or an object to be photographed by hand. Even in an environment where the object to be shot and the 3D scanner move simultaneously, a series of processes that continuously track and then merge 3D data without error requires a considerably high level of software-like algorithms. The core algorithm is the core technology of a business that manufactures mobile 3D scanners, and the number of companies with such technology is very small in the world. Therefore, portable 3D scanners that maintain high precision on the market have a relatively high price (20 million ~ 100 million), making it difficult for users who want to use the scanner to easily purchase and use the scanner. In the present invention, by developing a more sophisticated algorithm for the signals received from the two-dimensional plane distance sensor, a movable 3D scanner device capable of being used at a reasonable price in various industries can be provided by allowing the product to be realized with a measurement accuracy of less than ± 0.2 mm do.

The present invention provides a portable 3D scanner device having a precision of ± 0.2 mm or less on a 50 cm basis using a two-dimensional plane distance sensor. In order to achieve the above object, the development of a self-generating driver that increases the accuracy of a two-dimensional plane distance sensor is a key challenge.

A CMOS color image sensor attached to various devices such as a PC, a notebook, and a tablet PC to be linked with a portable 3D scanner, and a process of calibrating information received from the Depth Image CMOS by emitting an IR light, It is necessary to decode inherent distortions of image sensors and to merge the data, and to perform an operation process that enhances the merge processing performance. In order to solve the problem, the research and development of the signal check and tracking (tracking module) for the scan object have been carried out in order to strengthen the tracking after constructing the device driver which receives the laser signal outputted from the object and the signal returning from the object as point data. And recognizing the object to be scanned and recognizing the tracked object when the scan point is shifted. The key point of the invention is to prevent the target from being missed even in a portable state.

By developing a data analysis module that can minimize tracking loss by graphing 3D data as one of the means for recognizing an object, a pattern for tracking loss is found and strengthened. Object recognition is an artifact that is recognized when x and y are each sinusoidal wave, which is an easy tool for automatic tracking.

By developing a module to convert the signal data into 3D in the form of portable data and to make the final mesh, we recognize that the research module that can be installed in the software is needed for the gyro sensor concept, A module that can be completed with a final mesh has been developed for the present invention. After stabilizing the tracking function, we can apply the distance and focal values required for the spatial coordinate transformation proposed by the sensor adopter to the result obtained through Point Render as a precision enhancement task, . After forming the n-plane for successive incoming points and acquiring the direction of 1-plane and n-plane, we made a new coordinate and made a correction with the stack according to the distance. Stack is for post-correction processing to enhance tracking, and Correction is for next n-plane. The acquired data is triangulated with neighboring points of the final data, and the neighbors of each polygon are searched, aligned, and formed into a final mesh. In the process, the first raw data point is estimated from the next face that is changed to the starting point, and after obtaining the direction (NV), the previous face And a value for judging the noise between points adjacent to each other in the space of the next face is corrected and corrected based on the obtained value. This is called volume operation. As you proceed with the volume operation, you can get a much smoother surface compared to the previous volume process addition.

Then, color texturing is applied to 3D data. After acquiring bi-directional signal data, calculate color value according to real-time Tracking Build and replace it with real-time Mesh Build. Point cloud and color values are matched to each other in different directions because of the difference of cam position and scanner position. This problem is solved by developing calibrating function to adjust proper distance and direction of 0 point through various shooting experiments can do.

According to the present invention, the probability of maintaining tracking is higher than that of the conventional technique, and the time required for auto-merging can be shortened. Also, shooting accuracy is higher, so users in a variety of industries get more accurate results quickly and easily.

1 is a block diagram showing a core algorithm development step according to the present invention;
2 is a block diagram showing an internal processing processor of a two-dimensional plane distance sensor.
3 shows a real-time signal check, a target object bend check, and a 3D point collection module actual screen.

Driver development item is H / W Interrupt interface development. After analyzing the interrupts (service-to-category) provided on the H / W in details, the driver is designed to call from the driver to the category. Driver development item is H / W Function Service Call interface development. In detail, functions for interrupts are provided for the re-design of the interrupts on the H / W basis, which must be handled by the Driver Interface. Driver development includes Signal backtrace terminal service development. Separate additional signals and signals for cycle and reverse terminal design (service identification). There is an emulation program development as driver development item. It is necessary to develop a program for a simple driver test with details. The following are the technical requirements for driver development. Protocol Search Reversing Protocol Reverse analysis of incoming signal on port (signal gate). WDM Target OS is Window series, so WDM (Window Driver Model) technology based on System Rule for Window OS. I / O Input Output Rule design technology for the corresponding H / W. 32Bit & 64Bit O.S. Assembling technology by number of bits. Descriptor Reverse Descriptor Rule for the corresponding H / W. Handshake Packet Rule design technique for data signal cycle. Big Size Data Processing It is real time large data processing spare design technology.

 There is development of data alignment as improvement item of precision improvement. Sequential data alignment and additional information configuration are required. There is development of Raw Sensor Streams as development item to improve precision. Data Rule analysis is needed to reinterpret unprocessed data with details. Development of interpolation and extrapolation as a development item to improve precision. In detail, data interpreting centered on the human body requires error interpolation and extrapolation between discontinuous v1 ... vn. There is development of data error as improvement item of precision improvement. Details need to be corrected for errors with existing process system data. The following technologies are required for development of precision improvement. Normal Vector Install A technique for automatically generating a Normal Vector for collected data. Alignment Unspecified collation generation technique for data comparison. Interpolation Development of interpolation method for human body, not unspecified objects. Extrapolation Development of an extrapolation method for the human body, not unspecified objects. In / Ex Complex data interpolation method and extrapolation method. Absolute & Relative Error There is a determination technique based on absolute / relative error according to data interpretation.

 Development of tracking function is developed as pulse rate checker. In detail, it is necessary to develop a data pulse checker according to the distance, direction and moving speed coming from the sensor H / W. Tracking function development is Auto Wake Up development. Tracking in Sensor H / W After loss of focus, re-tracking If the focus is re-entered again, it automatically initializes H / W and develops a tracking module. Development of Tracking function There is development of Fix Face shot. Developed the ability to shoot multiple times at fixed positions in detail. The following technologies are needed to develop the tracking function. Pulse Rate Due to the large amount of incoming signal data from Parallel H / W, Parallel Stack technology due to the limitation of serial method. Counter Balance New scan data collection and existing collection data correction technology when tracking data after loss of track while collecting existing scan data. Tracking Loss Checking Technique for tracking loss during scanning data collection for re - tracking. Synchronized Tracking Signal Checking Synchronized Tracking Signal Rule setting technology for Sensor H / W. Auto Point Merge for Mesh Correction error correction technique for incoming data after re-tracking after loss of focus. Tracking Real-time Normal Vector Identity Calculate A technique for calculating a simple vector in real time during tracking.

100: Driver Development Items
110: Reverse protocol design process
120: Desceiptor inverse analysis course
130: Driver load module
140: Windows Registry Key Module
150: Driver service module
200: Precision Enhancement Items
210: Raw Data Extraction Module
220: Data Alignment module
230: Interpolation / Extrapolation module
240: Complex Apply module
250: Data Error Checking Module
300: Tracking enhancement items
310: Pulse Rate Data Field Analysis Module
320: H / W Position Data Analysis Module
330: I / O Rate Controller Module
340: Point Merging module
350: Multiface Shot System Module

Claims (19)

Using the signal data obtained through the driver interface, develop software for the service purpose. There are driver development (100) items for 32bit & 64bit OS driver development. Assembly (Mnemonic Symbol Language) is different for each operating system and the size of registers (temporal variable storage for arithmetic processing) is 32Bit for 4Byte and 64Bit for 8Byte.
The method according to claim 1,
The protocol reverse design 110 includes a Token (header information to be transmitted, in / out / start of frame), Data (pure data), Handshake (ID for a packet, ack / nak / not yet) / busy), and reverse the signal data.
The method according to claim 1,
The descriptor reverse analysis 120 for the enumeration operation is performed for the sensor configuration, the vendor information, the number of END-POINTs, the power consumption (mA), the number of interfaces, Interval Analyze the maximum packet size (transmission data size) and the like.
The method according to claim 1,
The driver load design 130 includes a host controller / controller driver, a bus driver, and a client device driver for controlling devices connected to the bus. And the serial interface engine (SIE) and END-POINTs that carry out the communication protocol, and the processor-related firmware to control them. A logical connection channel called a pipe is created between the host and the device, which is a non-existent connection, and the connection between the Host and the END-POINTs present in the device can be logically expressed . When a host and a device are connected by a physical connector or cable defined by the sensor (physical connection) and the bus is released, the data is transferred to a specific end-point of the specific device by using the packets determined through the bus In this way, the data path (channel) to which the END-POINT of the host and the specific device are connected is called a pipe. Depending on the purpose of END-POINT, control (status check), Isochronous (periodic data) Polling for conversation) and Bulk (non-periodic data) transmission. The driver load / unload design is centered on this basis.
The method according to claim 1,
The Windows Registry Key Design (140) is divided into two categories of Windows OS drivers. First, the VxD (virtual driver) is required before OS OS boot. When plug-in code is inserted, It is not necessary for the current development target, and records the service registration necessary for quasi-boot as a DLL type in the Window system.
The method according to claim 1,
The Driver Service Design (150) further develops the functions required by the software to be developed after the basic functions communicating with the Sensor H / W are designed. In other words, the role of Driver is not only H / W, but also interface development with S / W is essential.
7. The method of claim 6, wherein the emulation design develops an emulation for the developed driver operation test.
Scanners are largely laser and optical. The optical method is to re-analyze the reflectance and the bend into geometry information point clouds (point data) by shooting a pattern photon in a desired direction in a similar manner to a conventional camera. The laser method emits a laser beam to a measurement object, (Point data) by calculating the round trip time of Point Clouds. Because the intended product development characteristics require Portable format along with the enhancement of precision (200), H / W size and continuous scanning are required. In the case of the optical type, since the space charge ratio is large and the photographing angle and the number of times are large in order to collect the entire data, it is not suitable for the portable type scanning. In case of the laser type, all the data is collected only once, Because it is 50cm or so, use a portable laser.
9. The method of claim 8,
The Raw Data Inverse Design (210) method constructs a data field by inverse analysis of the raw data format.
9. The method of claim 8,
The data alignment design 220 generates a normal vector in the order of Left->Top->Right-> Bottom Rule, where the generated data is referenced and then reapplied to the rule. The reapplied data is necessary to generate the node again and process the neighboring data quickly.
9. The method of claim 8,
Interpolation / Extrapolation development (230) has been developed only for the graph statistics and photo image correction. However, since the main object of the intended development product is the human body or real objects, It is a task that needs to be newly developed.
9. The method of claim 8,
The complex application development 240 can perform numerical correction even if only one of the interpolation and the extrapolation is applied when the data sample is two-dimensional in the forward direction. However, since the neighboring dispersion values are different in the three-dimensional direction, , And interpolation / extrapolation must be determined in part according to this analytical work.
9. The method of claim 8,
The Data Error Checking development unit 250 re-analyzes the re-processed data after obtaining the error rate with respect to the curvature distribution for the existing S / W data, and reduces the distribution error of the re- Research work.
In the portable laser method, continuous data occurs periodically through tracking, and the scanning range is subject to tracking with a limit of 1m as the object center. For example, if there is a sudden change in distance due to the tracking data extraction for an object coming within 50 cm, it is necessary to re-record due to a tracking error. Therefore, There is a need to develop a tracking enhancement 300 that can be picked up and collected.
15. The method of claim 14,
The Pulse Rate Data Field analysis 310 is an analysis of the incoming data format and cycle after the hardware scanning operation.
15. The method of claim 14,
The H / W Position Data Analysis (320) sets the distance from 30 cm to 100 cm at regular intervals of 10 cm, analyzes the incoming signal data, and analyzes the tracking change rate and loss width while dynamically changing each set position. This data analysis will be used as an important manual for the method of shooting at the point of sales, and it is an inference about the probability of tracking loss.
15. The method of claim 14,
The I / O rate controller operation 330 sequentially receives the incoming signal in the cycle 2, and if the tracking loss is judged, the hardware is periodically waited / initiated / executed / checked in order until the condition of the given rule is satisfied It is a task to take a sleep (wait) form.
15. The method of claim 14,
The Point Merging module operation 340 starts the Key of the data photographed by the first tracking and then calculates the NV (Normal Vector) of the incoming data at the time of the new tracking, and generates a new key, It is a reference work.
15. The method of claim 14,
The development of Multiface Shot System function (350) refers to the development of a service provided in S / W so that it can be shot n times when there is a limit of shooting once due to environmental factors. So that they can be completely integrated into one.

Images