TWI458339B - 3d image sensor alignment detection method - Google Patents

Description

3D image sensor calibration method

The invention relates to a method for correcting a 3D image sensor, in particular to accurately finding the horizontal and vertical displacement between two images in two images captured by a 3D image sensor, thereby further sensing the 3D image. A method by which the detector is calibrated.

The content of stereoscopic 3D images has developed rapidly in recent years, but the quality of 3D images also determines the comfort of users. In general, 3D images usually capture external images by a 3D camera with left and right sets of lens and image sensing modules to simulate the 3D visual effects of the left and right eyes of the human eye. If the 3D camera has an error in the position or angle of the lens and the image sensing module during the manufacturing process, the quality of the captured 3D image is poor and cannot be viewed comfortably by the user. Therefore, how to ensure that the position or angle of the lens and image sensing module can be quickly and accurately positioned during the manufacturing process of the 3D camera is a subject that the 3D camera manufacturer is committed to.

The main purpose of the present invention is to provide a 3D image sensor calibration method, which can detect the position difference values of two images captured by two sets of lenses and image sensing modules of a 3D camera module and adjust accordingly. Correction to ensure that the 3D camera module can capture good 3D image quality.

In order to achieve the above purpose, the present invention discloses a 3D image sensor calibration method, which includes the following steps: Step (A): Positioning a 3D camera module, the 3D camera module includes at least two groups of lenses and Image sensing module; step (B): using at least two sets of lens and image sensing module of the 3D camera module to capture at least two images from the outside; step (C): calculating the device by a control device Taking the position difference value of the at least two images; step (D): determining whether the position difference value is within a preset range; if it is within the preset range, performing step (F), if not at the preset Step (E) is performed in the range; step (E): adjusting the position of at least one of the lens and the image sensing module of the at least two groups of the lens and the image sensing module according to the position difference value, and then Return to step (B) to execute; and step (F): stop.

In a preferred embodiment, the at least two image systems include a first image and a second image, and the first and second images are both P*Q images having a P pixel and a Q pixel. And the method for calculating the position difference value in the step (C) includes the following steps: Step (C1): determining a search block in the second image, the search block is located in the second The image data range of the P*Q of the image; step (C2): establishing a search condition from the first image, the search condition is based on one of the condition areas of the P*Q image data of the first image The data of the block is established, wherein the width and height pixel values of the condition block are respectively smaller than the width and height of the search block; and step (C3): searching from the search block of the second image Corresponding to the location of the corresponding block of the search condition; and step (C4): calculating the difference according to the condition block and the position of the corresponding block relative to the position of the P*Q image data in the P*Q image data range The position difference value between the conditional block and the corresponding block.

In a preferred embodiment, the width and height of the conditional block are W pixels, and the width and height of the search block are W+2t pixels, where (W+2t)<P and ( W+2t)<Q.

In a preferred embodiment, t is an integer multiple of W, and the method for finding a corresponding block corresponding to the search condition in the step (C3) is performed by using the data of the condition block first. At least one mathematical function calculates at least one condition value, and divides a plurality of secondary search blocks having the same width and height as the W pixel in the search block, and the edge of each adjacent search block Then, the values of each search block are respectively calculated and compared with the at least one condition value by the same at least one mathematical function, and if they are the same, it means that the search block meets the search condition. The corresponding block.

In a preferred embodiment, the method for finding a corresponding block corresponding to one of the search conditions described in the step (C3) is performed in a multi-stage search manner.

In a preferred embodiment, the multi-stage search method includes: first, using a relatively large value of W1 as the width and height of the first conditional block for performing a first stage search, and according to Finding a first corresponding block that meets the search condition and whose width and height are the same as W1 in a first search block whose width and height are W1+2t pixels; when the width and height are the same as W1, After the first corresponding block, the first corresponding block is used as one of the second search blocks for performing a second stage search, and a relatively small value of W2 is used as one of the second stage search. The width and height of the two conditional blocks are found in the second search block whose width and height are W1, and the second corresponding block that meets the search condition and whose width and height are the same as W2.

In a preferred embodiment, in the step (A), the 3D camera module is positioned by a positioning device, and the positioning device includes an adjustable lens and image sense of the at least two groups. A mechanism for measuring the position of at least one of the lenses and the image sensing module in the module.

In order to more clearly describe the 3D image sensor calibration method proposed by the present invention, the following will be described in detail in conjunction with the drawings.

The 3D image sensor calibration method of the present invention is mainly for improving the shooting effect of the 3D camera. Therefore, the alignment correction detection system of the 3D image sensor is developed, and the precision hardware tool is matched with the software assisted design, which can effectively Position correction of the horizontal and vertical aspects of the lens and image sensing module enables the 3D camera to capture the best quality 3D images. In the present invention, the correlation between the two images is calculated by using various alignment methods in the two adjacent images captured by the 3D camera to find the corresponding relationship between the two images. The invention mainly uses the motion estimation method to calculate the horizontal and vertical movement amounts of the two images. First, in order to get an accurate amount of movement, a specific constant block is selected as the detection sample. Second, the motion estimation consumes most of the processing time, and uses a fast and efficient search method to find the best matching position. To obtain the displacement of the overall image, and effectively use the alignment correction detection system of the 3D image sensor.

1 is a schematic diagram of an embodiment of a registration correction detection system for a 3D image sensor of the present invention, which includes: a 3D camera module 10, a positioning device 20, a control device 22, and a test. Figure 23.

The 3D camera module 10 includes at least two sets of lens and image sensing modules 11 and 12. In this embodiment, the first lens and the image sensing module 11 on the left side and the image sensing module 11 are located. The second lens and the image sensing module 12 on the right side respectively capture the left and right images of the left and right eyes to simulate the images seen by the left and right eyes when viewing the same scene. Each lens and image sensing module 11 and 12 respectively includes a lens set (Lens Set) and an image sensor (Image Sensor), and the light image from the external scene is projected on the sensor by the lens group. And converted into an electrical signal corresponding to the image. The lens group can be a fixed focus or a zoom lens group, and the sensor can be a CCD or a CMOS sensor. In this embodiment, the 3D camera module 10 can include only one circuit board, the two sets of lens and image sensing modules 11 and 12 on the circuit board, and the positioning of the lens and image sense. One of the test modules positions the module formed by the frame, and other components including a 3D camera such as an LCD display screen or an operation button are preferred; however, the 3D camera module 10 may also have a 3D All or part of the components of the camera.

The positioning device 20 is configured to position the 3D camera module 10 so that all the lens and image sensing modules 11 and 12 on the 3D camera module 10 can capture a specific area or range of the test pattern 23. Pattern 24. The positioning device 20 is configured to include at least a width and a height of at least one of the lens and image sensing modules 11 and 12 of the lens and image sensing modules 11 and 12 that can accurately adjust the at least two groups. The position of the direction and the adjustment mechanism of the tilt angle (not shown).

The control device 22 is connected to the circuit board of the 3D camera module 10 via a transmission line 21, and can receive and detect and analyze the electrical signal from the 3D camera module 10. In this embodiment, the control device 22 may be a computer device including a display, an operating keyboard, and the like, which implements a specific analysis software for analyzing and processing the aforementioned electrical signals and implementing the 3D image sensing of the present invention. Various detection, calculation and analysis in the calibration method.

2 is a flowchart of an embodiment of a 3D image sensor calibration method according to the present invention, which includes the following steps: Step 31: Position a 3D camera module 10 on the positioning device 20, in the 3D The camera module includes at least two sets of lens and image sensing modules 11, 12 (such as but not limited to the first and second lens and image sensing modules 11, 12).

Step 32: The at least two sets of lens and image sensing modules 11 and 12 of the 3D camera module 10 are used to capture the test pattern 23 from the outside to capture at least the specific area pattern 24 of the test pattern 23. Two images. As shown in FIG. 4 , the at least two images include the first image 41 (left image) captured by the first lens and the image sensing module, and the second lens and image sensing module. The second image 42 (right image) is captured; wherein the first and second images 41 and 42 are both P*Q image data having a P pixel and a Q pixel. In the present embodiment, the test pattern 23 may include a specific pattern 24 for image test analysis. However, in another embodiment, the test pattern 23 may also be a general scene pattern.

Step 33: The electrical signals of the two images 41 and 42 are received by the control device 22, and the position difference values of the captured at least two images 41 and 42 are calculated accordingly.

Step 34: It is determined by the control device 22 whether the position difference value is within a predetermined range. If it is located within the preset range, it indicates that the first and second lens and image sensing modules 11 and 12 are in the correct position and angle within the allowable range, and are captured by the 3D camera module 10. The 3D image quality is qualified, and then step 35 is executed to stop the 3D image sensor correction method of the present invention. On the contrary, if the result of the judgment indicates that the position difference value is not within the preset range, it indicates that the 3D image captured by the 3D camera module 10 is of poor quality and cannot be viewed comfortably by the user, so that it is required Then go to step 36.

Step 36: The positioning device 20 is operated according to the position difference value to adjust the position of at least one of the two lens and image sensing modules 11 and 12 and the image sensing module, and then return to the step. 32 re-execution.

Referring to FIG. 3 and FIG. 4 , FIG. 3 is a flowchart of a method for calculating the position difference value in step 33 shown in FIG. 2 , and FIG. 4 is a schematic diagram illustrating the first and second images 41 , Schematic diagram of the block range on 42. In the present invention, the calculation method of the step 33 further includes the following steps:

Step 331: Determine a search block 422 in the second image captured by the second lens and the image sensing module 12, and the search block 422 is located in the P*Q image data of the second image 42. Within the scope. In this embodiment, the search block 422 is a search range when searching later, and the width and height are W+2t pixels; wherein the value of t is due to the positional deviation between the lens and the image sensing module. The expected maximum deviation amount that may cause image shift, and the value of W will be described later, (W+2t)<P and (W+2t)<Q.

Step 332: Establish a search condition in the first image 41 captured by the first lens and the image sensing module 11, and the search condition is based on the P*Q image data range of the first image 41. The data of one of the condition blocks 411 is established; wherein the width and height of the condition block 411 are W pixels, in other words, the width and height of the condition block 411 are smaller than the search block respectively. Width and height of 422. In this embodiment, the method for analyzing and calculating the method is based on a block, so t is an integer multiple of W. However, in another embodiment, t may not be an integer multiple of W.

Step 333: From the search block 422 of the second image 42, find a location corresponding to the block 421 corresponding to the search condition.

Step 334: Calculate the difference between the condition block 411 and the corresponding block 421 according to the difference between the condition block 411 and the position of the corresponding block 421 in the P*Q image data range. The position difference value, that is, the position difference value between the first and second images 41, 42.

Please refer to FIG. 5 , which is a schematic diagram of searching for a corresponding block in accordance with the search condition in the 3D image sensor calibration method of the present invention. In the present invention, the method for searching for a corresponding block corresponding to the search condition in the step 333 is to first calculate at least one of the data of the conditional block located on the first image by using at least one mathematical function. a condition value, and a plurality of sub-search blocks having the same width and height as the W pixel are segmented in the search block located on the second image, and the edges of the adjacent sub-search blocks are connected . As shown in FIG. 5, in the search block on the second image, a total of 225 search blocks are divided into a width and a height of 15 equal parts. Then, according to the block unit, the values of each search block are respectively calculated and compared with the at least one condition value by the same at least one mathematical function, and if they are the same, the search block is The corresponding block that meets the search condition, if the value is not exactly the same, the next search block having the closest value is used as the corresponding block that meets the search condition. As shown in FIG. 5, it is assumed that the position of the conditional block is located at the center of the first image, that is, the coordinate is (0, 0), and the search finds the same image as the conditional block (that is, it conforms to the When the corresponding block of the search condition is located at the coordinates (-1, -5) of the X image on the second image, it can be known that the position difference between the first and second images is "width direction". Offset -1W pixel, height direction offset -5W pixel".

In a preferred embodiment of the present invention, the method for finding a corresponding block corresponding to one of the search conditions described in step 333 can be performed in a multi-stage search manner. Taking a three-stage search as an example, W1 of a relatively large value may be used as a first conditional search for the width and height of the first conditional block, and according to the width and height, W1+2t pixels are used. A first search block of the first search block is found to meet the search condition and the width and height are the first corresponding block of W1. After the first corresponding block whose width and height are the same as W1 is found, the first corresponding block is used as one of the second search blocks when performing a second stage search, and the value is a relatively small value. W2 is used as the width and height of the second conditional block of the second stage search, and the second search block whose width and height are the same as W1 is found to meet the search condition and the width and height are the same as W2. The second corresponding block. Then, the second corresponding block is used as one of the third search blocks when performing a third stage search, and the W3 of a relative minimum is used as the width of the third condition block of the third stage search. And the high, in the third search block whose width and height are W2, find the third corresponding block that meets the search condition and the width and height are the same as W3, so that the efficiency of the search can be accelerated. As shown in Figure 6, in the first stage of the large block search process, the width and height of the first conditional block are both W1, and W1 is equal to 7 times of W3, that is, at least width and height are required. Find the first corresponding block a1 of width and height of 7 times W3 pixels for the first search block of 15 or more of numbers -7 to 7 (each of which is W3 pixels). The width number is -7 to -1, and the height number is from 1 to 7.) After the first corresponding block a1 is found, the middle block search process of the second stage is performed; at this time, the width and the height of the second condition block are both W2, and W2 is equal to 3 times of W3, that is, it needs to be wide Find a second corresponding block a2 (width number -3 to -1) with a width and height of 3 times W3 pixels in the range of the second search block numbered -7 to -1 and high number 1 to 7. The height is numbered from 5 to 7). After the second corresponding block a2 is found, the cell block search process of the third stage is performed, and the width and the height of the third condition block are both W3, that is, the width number is -3 to -1. A third corresponding block a3 of width and height of each W3 pixel is found in the range of the third search block numbered 5 to 7. Once the third corresponding block a3 located at the wide number -1 and the high number 5 is found, the position difference between the first and second images is obtained as "width shift -1 W3 pixel, height" Direction shift -5W3 pixels".

The above-mentioned embodiments are not intended to limit the scope of application of the present invention, and the scope of the present invention should be based on the technical spirit defined by the content of the patent application scope of the present invention and the scope thereof. It is to be understood that the scope of the present invention is not limited by the spirit and scope of the present invention, and should be considered as a further embodiment of the present invention.

10. . . 3D camera module

11,12. . . Lens and image sensing module

20. . . Positioning means

twenty one. . . Transmission line

twenty two. . . Control device

twenty three. . . Test pattern

24‧‧‧ pattern

31~35, 331~334‧‧‧ steps

FIG. 1 is a schematic diagram of an embodiment of a registration correction detection system for a 3D image sensor of the present invention.

FIG. 2 is a flow chart of an embodiment of a 3D image sensor calibration method according to the present invention.

FIG. 3 is a flow chart of a method for calculating the position difference value described in step 33 shown in FIG.

FIG. 4 is a schematic diagram illustrating the range of blocks on the first and second images 41, 42.

FIG. 5 is a schematic diagram of an embodiment of performing a 3D image sensor calibration method according to the present invention for finding a corresponding block that meets a search condition.

FIG. 6 is a schematic diagram of another embodiment of performing a search for a corresponding block that meets a search condition in the 3D image sensor calibration method of the present invention.

31~35. . . step

Claims (4)

A 3D image sensor calibration method includes the following steps: Step (A): positioning a 3D camera module, the 3D camera module includes at least two sets of lens and image sensing modules; and step (B) And using at least two sets of the lens and the image sensing module of the 3D camera module to capture at least two images from the outside; and (C): calculating, by a control device, a position difference value of the captured at least two images Step (D): determining whether the position difference value is within a predetermined range; if it is within the preset range, performing step (F), if not within the preset range, performing step (E); (E): adjusting the position of at least one of the lens and the image sensing module of the at least two groups of lenses and the image sensing module according to the position difference value, and then returning to step (B) to execute; and the step (F): stopping; wherein the at least two images comprise a first image and a second image, the first and second images being the same P*Q image having a P pixel and a Q pixel. And the method for calculating the position difference value described in step (C) includes There is the following steps: Step (C1): determining a search block in the second image, the search block is located in the image data range of the P*Q of the second image; Step (C2): from the first A search condition is established in the image, and the search condition is established according to data of one of the conditional blocks in the P*Q image data range of the first image, wherein the condition block has a width and a high pixel value. Is less than the width and height of the search block respectively; Step (C3): searching for a location corresponding to the block corresponding to the search condition from the search block of the second image; and step (C4): according to the condition block and the corresponding block relative to the two The difference between the position of the P*Q image data and the position difference between the condition block and the corresponding block is calculated; wherein the width and height of the condition block are W pixels. And the width and height of the search block are W+2t pixels, where (W+2t)<P and (W+2t)<Q; wherein t is an integer multiple of W, and the step (C3) The method for finding a block corresponding to one of the search conditions is to first calculate at least one condition value by using at least one mathematical function of the data of the condition block, and segmenting the search block in the search block a plurality of sub-search blocks having the same width and height as the W pixel, and the edges of the adjacent sub-search blocks are connected; then, the search regions are respectively calculated in sequence by the same at least one mathematical function The value of the block is compared with the at least one condition value, and if it is the same, it means that the search block is in accordance with the search The condition of the corresponding block. The method for correcting a 3D image sensor according to claim 1, wherein the method for finding a corresponding block corresponding to the search condition in the step (C3) is performed in a multi-stage search manner. The method for correcting a 3D image sensor according to claim 2, wherein the multi-stage search method comprises: first performing a first stage search with a relatively large value of W1. The width and height of a conditional block, and according to the first search block whose width and height are W1+2t pixels are found to meet the search condition and the width and height are the same One of the first corresponding blocks of W1; after finding the first corresponding block whose width and height are the same as W1, the first corresponding block is used as one of the second search blocks when performing a second stage search And using W2 of a relatively small value as the width and height of the second condition block of the second stage search, and searching for the search condition in the second search block whose width and height are the same as W1 And the width and height are the second corresponding block of W2. The method for correcting a 3D image sensor according to claim 1, wherein in the step (A), the 3D camera module is positioned by a positioning device, and the positioning device is included in the positioning device. There is a mechanism for adjusting the position of at least one of the lens and the image sensing module of the at least two groups of the lens and the image sensing module.