TWI790854B - Multi-modal image alignment method and system - Google Patents
Multi-modal image alignment method and system Download PDFInfo
- Publication number
- TWI790854B TWI790854B TW110146318A TW110146318A TWI790854B TW I790854 B TWI790854 B TW I790854B TW 110146318 A TW110146318 A TW 110146318A TW 110146318 A TW110146318 A TW 110146318A TW I790854 B TWI790854 B TW I790854B
- Authority
- TW
- Taiwan
- Prior art keywords
- dimensional
- transformation matrix
- images
- points
- point
- Prior art date
Links
Images
Landscapes
- Apparatus For Radiation Diagnosis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Radiation-Therapy Devices (AREA)
Abstract
Description
本發明係關於一種多模影像對位方法,特別係關於一種三維影像與二維影像的對位方法。The invention relates to a method for alignment of multi-mode images, in particular to a method for alignment of three-dimensional images and two-dimensional images.
在後疫情時代,科技、經濟、文化、公衛等層面在全球都進入一個新常態,各國從抗疫走向與疫共存,民眾的生活習慣亦隨之改變。除了出門戴口罩之外,民眾進入各種室內場所皆須量測體溫。目前市面上常見的體溫偵測系統,多數僅針對溫度量測,而在體溫量測已為生活日常的情況下,應考量系統功能的擴充,例如生理訊號量測、行動偵測等。In the post-epidemic era, technology, economy, culture, public health and other aspects have entered a new normal around the world. Countries have shifted from fighting the epidemic to coexisting with the epidemic, and people's living habits have also changed accordingly. In addition to wearing masks when going out, people must measure their body temperature when entering various indoor places. Most of the body temperature detection systems currently on the market are only for temperature measurement, and when body temperature measurement is already a part of daily life, the expansion of system functions should be considered, such as physiological signal measurement, motion detection, etc.
上述一機多功能之訴求的基礎量測訊號多為體溫、心搏、呼吸及行為等,所需的非接觸式感測來源多為可見光影像、熱影像及三維影像(點雲)。以成本為考量,目前主流方式係以多種平價品牌的影像擷取裝置混搭以嘗試滿足一機多功能的訴求。然而,各影像擷取裝置的座標系統彼此理應不相同,因此無法同時取得某標的物的多模。The basic measurement signals for the above-mentioned one-machine multi-function demands are mostly body temperature, heartbeat, respiration, and behavior, etc., and the required non-contact sensing sources are mostly visible light images, thermal images, and 3D images (point clouds). Taking cost into consideration, the current mainstream method is to mix and match image capture devices of various affordable brands in an attempt to meet the demands of one machine with multiple functions. However, the coordinate systems of the image capture devices are supposed to be different from each other, so it is impossible to obtain multiple modes of a certain object at the same time.
鑒於上述,本發明提供一種多模影像對位方法及系統。In view of the above, the present invention provides a multi-mode image alignment method and system.
依據本發明一實施例的多模影像對位方法,包含以處理裝置執行:取得關聯於校正體的多張二維影像及多張三維影像,校正體具有中央頂點及多個側頂點,二維影像關聯於第一三維座標系統,且三維影像關聯於第二三維座標系統;從所述多張二維影像取得對應於中央頂點的多個第一點及對應於所述多個側頂點的多個第二點群;從所述多張三維影像取得對應於中央頂點的多個第三點;利用待解第一轉換矩陣、所述多個第一點及所述多個第三點,基於第一三維座標系統執行第一最佳化運算,以取得最佳化第一轉換矩陣;利用最佳化第一轉換矩陣處理所述多張三維影像,以分別產生多張一次轉換影像;利用所述多張一次轉換影像、所述多個第一點、所述多個第二點群以及校正體的預設規格參數組,基於第一三維座標系統執行第二最佳化運算,以取得最佳化第二轉換矩陣;以及利用最佳化第一轉換矩陣及最佳化第二轉換矩陣執行轉換運算,以將待處理影像轉換至第二三維座標系統或第一三維座標系統。The multi-mode image alignment method according to an embodiment of the present invention includes executing by a processing device: obtaining multiple 2D images and multiple 3D images associated with a calibration body, the calibration volume has a central vertex and multiple side vertices, and the 2D images are associated with In the first three-dimensional coordinate system, and the three-dimensional image is associated with the second three-dimensional coordinate system; obtaining a plurality of first points corresponding to the central vertex and a plurality of second points corresponding to the plurality of side vertices from the plurality of two-dimensional images group; obtain a plurality of third points corresponding to the central vertex from the plurality of three-dimensional images; use the first transformation matrix to be solved, the plurality of first points and the plurality of third points, based on the first three-dimensional coordinates The system executes a first optimization operation to obtain an optimized first transformation matrix; process the plurality of 3D images by using the optimized first transformation matrix to generate a plurality of primary transformation images respectively; converting the image, the plurality of first points, the plurality of second point groups, and the preset specification parameter set of the calibration body, and performing a second optimization operation based on the first three-dimensional coordinate system to obtain an optimized second transformation matrix; and performing a transformation operation by using the optimized first transformation matrix and the optimized second transformation matrix, so as to transform the image to be processed into the second 3D coordinate system or the first 3D coordinate system.
依據本發明一實施例的多模影像對位系統,包含校正體、二維影像擷取裝置、三維影像擷取裝置及處理裝置,其中處理裝置連接於該二維影像擷取裝置及該三維影像擷取裝置。校正體包含立體本體及多個指示元件,其中立體本體具有一中央頂點及多個側頂點,且指示元件分別設置於該中央頂點及該些側頂點。二維影像擷取裝置具有第一三維座標系統,且用於產生關聯於校正體的多張二維影像。三維影像擷取裝置具有第二三維座標系統,且用於產生關聯於校正體的多張三維影像。處理裝置用於依據所述多張二維影像及所述多張三維影像,取得座標轉換矩陣,且利用座標轉換矩陣將待處理影像轉換至第二三維座標系統或第一三維座標系統。A multi-mode image alignment system according to an embodiment of the present invention includes a calibration body, a 2D image capture device, a 3D image capture device, and a processing device, wherein the processing device is connected to the 2D image capture device and the 3D image Capture device. The correction body includes a three-dimensional body and a plurality of indicating elements, wherein the three-dimensional body has a central vertex and a plurality of side vertices, and the indicating elements are respectively arranged on the central apex and the side vertices. The 2D image capture device has a first 3D coordinate system and is used to generate multiple 2D images associated with the calibration body. The 3D image capture device has a second 3D coordinate system and is used to generate multiple 3D images associated with the calibration body. The processing device is used to obtain a coordinate transformation matrix according to the plurality of 2D images and the plurality of 3D images, and use the coordinate transformation matrix to transform the image to be processed into the second 3D coordinate system or the first 3D coordinate system.
藉由上述架構,本案所揭示的多模影像對位方法可以透過兩次最佳化運算取得不同座標系統之間的轉換矩陣,不須使用複雜的機器學習訓練即可達成高精準度的對位效果,且透過以三維角點特徵為求取轉換矩陣之根據,相較於傳統以平面式棋盤校正板為求取轉換矩陣之根據,所需之取樣資料數量甚少,即所需取樣時間較少。本案所揭示的多模影像對位系統同樣具有所需之取樣資料數量及取樣時間少的效果,且藉由設置有指示元件之特殊立體校正體設計,系統可以實現二維/三維影像中之特徵點的自動擷取。With the above structure, the multi-mode image alignment method disclosed in this case can obtain the transformation matrix between different coordinate systems through two optimization operations, and can achieve high-precision alignment without using complicated machine learning training effect, and by using the three-dimensional corner features as the basis for obtaining the transformation matrix, compared with the traditional planar checkerboard correction board as the basis for obtaining the transformation matrix, the amount of sampling data required is very small, that is, the required sampling time is shorter few. The multi-mode image alignment system disclosed in this case also has the effect of less sampling data and less sampling time, and the system can realize the characteristics of 2D/3D images through the design of a special stereo calibration body equipped with indicating elements Automatic extraction of points.
以上之關於本揭露內容之說明及以下之實施方式之說明係用以示範與解釋本發明之精神與原理,並且提供本發明之專利申請範圍更進一步之解釋。The above description of the disclosure and the following description of the implementation are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the patent application scope of the present invention.
以下在實施方式中詳細敘述本發明之詳細特徵以及優點,其內容足以使任何熟習相關技藝者了解本發明之技術內容並據以實施,且根據本說明書所揭露之內容、申請專利範圍及圖式,任何熟習相關技藝者可輕易地理解本發明相關之目的及優點。以下之實施例係進一步詳細說明本發明之觀點,但非以任何觀點限制本發明之範疇。The detailed features and advantages of the present invention are described in detail below in the implementation mode, and its content is enough to make any person familiar with the related art understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings , anyone skilled in the art can easily understand the purpose and advantages of the present invention. The following examples are to further describe the concept of the present invention in detail, but not to limit the scope of the present invention in any way.
請參考圖1,圖1係依據本發明一實施例所繪示的多模影像對位系統1的功能方塊圖。如圖1所示,多模影像對位系統1包含處理裝置11、二維影像擷取裝置12、三維影像擷取裝置13及校正體14,其中處理裝置14可以有線或無線的方式連接於二維影像擷取裝置12及三維影像擷取裝置13。Please refer to FIG. 1 , which is a functional block diagram of a multi-mode
處理裝置11可以包含但不限於單一處理器以及多個微處理器之集成,例如中央處理器(CPU)、繪圖處理器(GPU)等。處理裝置11用於依據二維影像擷取裝置12及三維影像擷取裝置13對校正體14進行拍攝而產生的影像,取得兩影像擷取裝置之座標系統的轉換矩陣,詳細的執行內容將於後描述。處理裝置11可以利用所述轉換矩陣,將三維影像擷取裝置13所產生的影像映射至二維影像擷取裝置12所產生的影像,以產生疊合影像。The
二維影像擷取裝置12例如為可見光攝影機、近紅外光攝影機、熱像儀等。二維影像擷取裝置12用於進行拍攝以產生二維影像,且具有一相機座標系統及一影像平面座標系統。三維影像擷取裝置13例如為三維點雲感測器、深度攝影機等。三維影像擷取裝置13用於進行拍攝以產生三維影像,且具有一相機座標系統。二維影像擷取裝置12及三維影像擷取裝置13可以設置於同一機殼內,或可以分別設置於不同機殼。The two-dimensional
校正體14包含本體140及多個指示元件141a~141d。立體本體140可以具有四個以上的頂點。圖1示例性地繪示立體本體140為六面體,然不限於此。指示元件141a~141d可以分別設置於立體本體140的頂點,其數量至少為四個。指示元件141a~141d可以皆為發光元件、皆為發熱元件,或皆為兼具發光及發熱功能的元件。指示元件141a~141d可以全部具有相同顏色或/及溫度,或可以分別具有不同顏色或/及溫度。The
進一步來說,指示元件141a~141d的種類可以取決於二維影像擷取裝置12的種類。於可見光攝影機作為二維影像擷取裝置的實施態樣中,指示元件141a~141d以發光元件實現。於近紅外光攝影機或熱像儀作為二維影像擷取裝置12的實施態樣中,指示元件141a~141d以發熱元件實現。於另一實施例中,多模影像對位系統1可以更包含另一二維影像擷取裝置。若兩個二維影像擷取裝置分別為可見光相機及熱像儀,則指示元件141a~141d以兼具發光及發熱功能的元件實現。Further, the types of the indicating
多模影像對位系統1的校正體14、二維影像擷取裝置12及三維影像擷取裝置13可以協同運作以產生供處理裝置11進行多模影像校正的資料。校正體14可以輪流擺放於多個校正位置。二維影像擷取裝置12可以受控以拍攝輪流擺放於不同校正位置的校正體14而產生多張二維影像。三維影像擷取裝置13可以受控以拍攝輪流擺放於不同校正位置的校正體14而產生多張三維影像。處理裝置11可以從二維影像擷取裝置12取得所述多張二維影像且從三維影像擷取裝置13取得所述多張三維影像,並據以進行多模影像對位校正,以取得二維影像擷取裝置12的相機座標系統與三維影像擷取裝置13的相機座標系統之間的轉換矩陣。The
另外,處理裝置11更可以依據此轉換矩陣及二維影像擷取裝置12的焦距及投影中心,取得二維影像擷取裝置12的影像平面座標系統與三維影像擷取裝置13的相機座標系統之間的轉換矩陣,且可以利用此轉換矩陣疊合兩裝置所產生的影像。處理裝置11更可以從疊合影像取得對應於特定標的物的多模訊號(例如包含溫度資訊及空間資訊,或包含顏色資訊及空間資訊)。所述特定標的物可由處理裝置11依特定演算法(例如人臉辨識演算法)來選取或由操作人員選取,本發明不予限制。或者,處理裝置11可以將疊合影像透過顯示器呈現給操作人員,或輸出至其他多模訊號量測應用裝置。In addition, the
請一併參考圖1及圖2,其中圖2係依據本發明一實施例所繪示的校正位置示意圖。如圖2所示,校正體14的校正位置P1~P6位於二維影像擷取裝置12的視角v1及三維影像擷取裝置13的視角v2之重疊範圍內,其數量至少為六。進一步來說,在每個校正位置P1~P6上,校正體14的指示元件141a~141d對於二維影像擷取裝置12及三維影像接取裝置13而言皆為可視。三維影像的數量可以相同於校正位置P1~P6的數量,二維影像的數量則取決於校正體14的指示元件141a~141d的控制方法。Please refer to FIG. 1 and FIG. 2 together, wherein FIG. 2 is a schematic diagram of a calibration position according to an embodiment of the present invention. As shown in FIG. 2 , the calibration positions P1 - P6 of the
於一實施態樣中,校正體14的指示元件141a~141d分別具有不同顏色或不同溫度,且在拍攝過程中被一同致能。二維影像擷取裝置12受控以拍攝輪流擺放於不同校正位置P1~P6的校正體14以產生同於校正位置P1~P6之數量的二維影像。每張二維影像包含對應於不同顏色或/及不同溫度的指示元件141a~141d的影像區塊。In an embodiment, the indicating
於另一實施態樣中,校正體14的指示元件141a~141d在一次拍攝程序中被依特定次序致能而發光或發熱。二維影像擷取裝置12受控以對輪流擺放於多個校正位置P1~P6的校正體14分別進行多次拍攝程序以產生多張二維影像。舉例來說,校正體14被擺放於校正位置P1~P6中的任一個時,指示元件141a被致能且二維影像擷取裝置12受控以產生包含對應於指示元件141a之影像區塊的二維影像,接著指示元件141b被致能且二維影像擷取裝置12受控以產生包含對應於指示元件141b之影像區塊的二維影像,指示元件141c及141d所對應之二維影像的產生為相同道理,便不予贅述。於此實施態樣中,二維影像的數量為校正位置P1~P6數量的N倍,其中N為校正體14之指示元件141a~141d的數量。In another embodiment, the indicating
上述校正體14的指示元件141a~141b的控制、二維影像擷取裝置12的拍攝及三維影像擷取裝置13的拍攝可以由操作人員所控制,或可以由儲存有相應控制指令的控制器藉由有線或無線的方式來控制,本發明不予限制。The control of the indicating
請一併參考圖1及圖3,其中圖3係依據本發明一實施例所繪示的多模影像對位方法的流程圖。如圖3所示,多模影像對位方法包含步驟S101:取得關聯於校正體的多張二維影像及多張三維影像,校正體具有中央頂點及多個側頂點,二維影像對應於第一三維座標系統 ,且三維影像對應於第二三維座標系統;步驟S102:從所述多張二維影像取得對應於中央頂點的多個第一點及對應於所述多個側頂點的多個第二點群;步驟S103:從所述多張三維影像取得對應於中央頂點的多個第三點;步驟S104:利用待解第一轉換矩陣、所述多個第一點及所述多個第三點,基於第一三維座標系統執行第一最佳化運算,以取得最佳化第一轉換矩陣;步驟S105:利用最佳化第一轉換矩陣處理所述多張三維影像,以分別產生多張一次轉換影像;步驟S106:利用所述多張一次轉換影像、所述多個第一點、所述多個第二點群及校正體的預設規格參數組,基於第一三維座標系統執行第二最佳化運算,以取得最佳化第二轉換矩陣;以及步驟S107:利用最佳化第一轉換矩陣及最佳化第二轉換矩陣執行轉換運算,以將待處理影像轉換至第二三維座標系統或第一三維座標系統。於此要特別說明的是,本發明不限制步驟S102及步驟S103的執行順序。Please refer to FIG. 1 and FIG. 3 together, wherein FIG. 3 is a flowchart of a multi-mode image alignment method according to an embodiment of the present invention. As shown in FIG. 3 , the multi-mode image alignment method includes step S101: obtaining multiple 2D images and multiple 3D images associated with the calibration body, the calibration volume has a central vertex and multiple side vertices, and the 2D images correspond to the first 3D A coordinate system, and the 3D image corresponds to a second 3D coordinate system; Step S102: Obtain a plurality of first points corresponding to the central vertex and a plurality of second point groups corresponding to the plurality of side vertices from the plurality of 2D images ; Step S103: Obtain a plurality of third points corresponding to the central vertex from the plurality of 3D images; Step S104: Using the first transformation matrix to be solved, the plurality of first points and the plurality of third points, Perform a first optimization operation based on the first three-dimensional coordinate system to obtain an optimized first transformation matrix; step S105: process the plurality of three-dimensional images by using the optimized first transformation matrix to generate multiple primary transformations respectively Image; Step S106: Using the plurality of once-transformed images, the plurality of first points, the plurality of second point groups, and the preset specification parameter set of the correction body, perform the second most accurate operation based on the first three-dimensional coordinate system an optimization operation to obtain an optimized second transformation matrix; and step S107: performing a transformation operation using the optimized first transformation matrix and the optimized second transformation matrix to transform the image to be processed into a second three-dimensional coordinate system or the first 3D coordinate system. It should be noted here that, the present invention does not limit the execution order of step S102 and step S103 .
圖3所示的多模影像對位方法可以適用於圖1所示的多模影像對位系統1,特別係由處理裝置11來執行。以下示例性地以多模影像對位系統1的運作來進一步說明步驟S101~S107。The multi-mode image alignment method shown in FIG. 3 can be applied to the multi-mode
於步驟S101中,處理裝置11取得校正體14的多張二維影像及多張三維影像。所述多張二維影像及所述多張三維影像的取得方法如前所述,於此不予贅述。步驟S101中的第一三維座標系統可以為二維影像擷取裝置12的相機座標系統,第二三維座標系統可以為三維影像擷取裝置13的相機座標系統,中央頂點可以為校正體14設置有指示元件141a的頂點,所述多個側頂點可以為分別設置有指示元件141b~141d的頂點。In step S101 , the
於步驟S102中,處理裝置11從所述多張二維影像取得對應於中央頂點的多個第一點及對應於所述多個側頂點的多個第二點群。於每張二維影像包含對應於不同顏色或/及不同溫度的指示元件141a~141d的影像區塊的實施態樣中,處理裝置11可以預先儲存查找表,查找表記錄中央頂點及側頂點各自所對應的顏色或/及溫度。處理裝置11可以影像處理演算法找到具有不同顏色或/及溫度的影像區塊,其中影像處理演算法例如包含但不限於二值化及圓偵測(circle detection)。處理裝置11可以依據所述查找表,判斷這些影像區塊分別對應於設置有指示元件141a的中央頂點及設置有指示元件141b~141d的側頂點。對應於中央頂點之影像區塊的中心點可以作為第一點,對應於側頂點之影像區塊的中心點可以組成第二點群。進一步來說,第一點及第二點群中的點各具有一二維座標表示其在二維影像擷取裝置12的影像平面座標系統中的位置。In step S102 , the
於指示元件141a~141d在每次拍攝程序中被依特定次序致能的實施態樣中,處理裝置11可以預先儲存所述特定次序。處理裝置11可以影像處理演算法找到具有顏色或/及溫度的影像區塊,其中影像處理演算法例如包含但不限於二值化及圓偵測(circle detection)。處理裝置11可以依據二維影像的產生時間及預存特定次序來判斷二維影像中的這些影像區塊係對應於中央頂點及側頂點中的何者。對應於中央頂點之影像區塊的中心點可以作為第一點,對應於側頂點之影像區塊的中心點可以組成第二點群。In an embodiment where the indicating
於步驟S103中,處理裝置11從所述多張三維影像取得對應於中央頂點的多個第三點。進一步來說,所述多個第三點對與所述多張三維影像具有一對一的關係,且各自具有一三維座標表示其在第二三維座標系統中的位置。處理裝置11可以將每張三維影像作為目標影像,執行:從目標影像取得三個平面,所述三個平面彼此相鄰且具有彼此垂直的法向量;以及取得所述三個平面的交點,作為所述多個第三點中的對應者。更進一步來說,處理裝置11可以從目標影像中找出所有平面,找出所有由彼此相鄰的三個平面組成的組合,計算各組合中的平面的法向量,過濾出組合中平面的法向量彼此垂直的組合,並計算該組合中的平面相交之點,作為所述多個第三點中的對應者。In step S103 , the
於步驟S104中,處理裝置11利用待解第一轉換矩陣、所述多個第一點及所述多個第三點,基於第一三維座標系統執行第一最佳化運算,以取得最佳化第一轉換矩陣。進一步來說,所述多個第一點及所述多個第三點分別對應於前述多個校正位置。處理裝置11可以對每一校正位置的對應第一點及對應第三點執行距離計算作業,以取得多個計算結果,所述多個計算結果亦分別對應於所述多個校正位置。In step S104, the
請一併參考圖1及圖4以進一步說明距離計算作業,其中圖4係依據本發明一實施例所繪示的多模影像對位方法中的距離計算作業的執行示意圖。如圖4所示,第一點D1為二維影像SD中對應於校正體14的中央頂點的點,二維影像SD對應於第一三維座標系統CY1(二維影像擷取裝置12的相機座標系統),第三點D2為三維影像TD中對應於校正體14的中央頂點的點,三維影像TD對應於第二三維座標系統CY2(三維影像擷取裝置13的相機座標系統)。Please refer to FIG. 1 and FIG. 4 together to further explain the distance calculation operation, wherein FIG. 4 is a schematic diagram illustrating the execution of the distance calculation operation in the multi-mode image alignment method according to an embodiment of the present invention. As shown in FIG. 4 , the first point D1 is a point corresponding to the central vertex of the
在距離計算作業的執行過程中,處理裝置11可以取得連接於第一三維座標系統CY1的原點及第一點D1的射線L1,利用待解第一轉換矩陣轉換第三點D2,並計算經轉換的第三點D2與射線L1之間的距離d,以作為距離計算結果。處理裝置11可以透過一收斂函式(cost function)迭代調整該待解第一轉換矩陣,且將經迭代調整後的該待解第一轉換矩陣作為最佳化第一轉換矩陣,其中所述收斂函式指示使對應於所有校正位置的距離計算結果之和為最小值。所述收斂函式可以表示如式(1):
(1)
其中
M
1 指示待解第一轉換矩陣,
Pt3D指示第三點D2,
Line指示射線L1。
During the execution of the distance calculation operation, the
特別來說,待解第一轉換矩陣可以包含旋轉矩陣及位移矩陣,其中旋轉矩陣關聯於三個軸向的角度參數,而位移矩陣關聯於三個軸向的位移參數。為了取得上述六個參數的解以取得最佳化第一轉換矩陣,校正位置的數量至少為六。旋轉矩陣及位移矩陣的詳細參數組成係本發明所屬領域中具有通常知識者能夠基於上述六個參數並依所需而設計者,本發明不予限制。Specifically, the first conversion matrix to be solved may include a rotation matrix and a displacement matrix, wherein the rotation matrix is associated with angle parameters of three axes, and the displacement matrix is associated with displacement parameters of three axes. In order to obtain the solutions of the above six parameters to obtain the optimized first transformation matrix, the number of calibration positions is at least six. The detailed parameter composition of the rotation matrix and the displacement matrix can be designed by those with ordinary knowledge in the field of the present invention based on the above six parameters and according to requirements, and the present invention is not limited thereto.
於圖3的步驟S105中,處理裝置11利用最佳化第一轉換矩陣轉換所述多張三維影像,且於步驟S106中,處理裝置11利用經最佳化第一轉換矩陣轉換的多張三維影像(一次轉換影像)、所述多個第一點、所述多個第二點群及校正體的預設規格參數組,基於第一三維座標系統執行第二最佳化運算,以取得最佳化第二轉換矩陣。理想上,步驟S104所得的最佳化第一轉換矩陣應使對應於所有校正位置的距離計算結果之和趨近於0,然實際上最佳化第一轉換矩陣為近似解。因此,經最佳化第一轉換矩陣轉換的三維影像所屬的座標系統與第一三維座標系統尚有所差異。藉由步驟S106所得之最佳化第二轉換矩陣再轉換經最佳化第一轉換矩陣轉換的三維影像,可以使得三維影像所對應的座標系統更接近第一三維座標系統。In step S105 of FIG. 3 , the
請參考圖1及圖5,其中圖5係依據本發明一實施例所繪示的多模影像對位方法中的第二最佳化運算的流程圖。第二最佳化運算可以包含步驟S601:利用待解第二轉換矩陣處理所述多張一次轉換影像,以分別產生多張二次轉換影像;步驟S602:依據所述多個第一點及所述多個第二點群,分別從所述多個二次轉換影像取得多個第四點群;步驟S603:依據所述多個第四點群,分別取得所述多個估算規格參數組;以及步驟S604:透過收斂函式迭代調整待解第二轉換矩陣,且將經迭代調整的待解第二轉換矩陣作為最佳化第二轉換矩陣,其中收斂函式指示使所述多個估算規格參數組與校正體的預設規格參數組間之差異為最小值。Please refer to FIG. 1 and FIG. 5 , wherein FIG. 5 is a flow chart of the second optimization operation in the multi-mode image alignment method according to an embodiment of the present invention. The second optimization operation may include step S601: using the second transformation matrix to be solved to process the multiple primary transformation images to generate multiple secondary transformation images respectively; step S602: according to the multiple first points and the According to the plurality of second point groups, respectively obtain a plurality of fourth point groups from the plurality of secondary transformed images; Step S603: Obtain the plurality of estimation specification parameter groups respectively according to the plurality of fourth point groups; And step S604: Iteratively adjust the second transformation matrix to be solved through the convergence function, and use the iteratively adjusted second transformation matrix to be solved as the optimized second transformation matrix, wherein the convergence function indicates that the plurality of estimation specifications The difference between the parameter set and the preset specification parameter set of the calibration body is minimal.
進一步來說,所述多個二次轉換影像、所述多個第一點、所述多個第二點群、所述多個第四點群及所述多個估算規格參數組可以分別對應於前述多個校正位置。於步驟S602及S603中,處理裝置11可以對每一校正位置的對應第一點、對應第二點群及對應二次轉換影像進行投影作業以取得對應第四點群,再依據對應第四點群進行規格參數估算作業取得對應估算規格參數組。Further, the plurality of reconverted images, the plurality of first points, the plurality of second point groups, the plurality of fourth point groups and the plurality of estimated specification parameter groups may respectively correspond to In the aforementioned multiple calibration positions. In steps S602 and S603, the
請一併參考圖1及圖6以進一步說明投影作業,其中圖6係依據本發明一實施例所繪示的多模影像對位方法中的投影作業的執行示意圖。如圖6所示,第一點D1為二維影像SD中對應於校正體14的中央頂點的點,第二點D31~D33為二維影像SD中對應於校正體14的側頂點的點且可以組成第二點群,二維影像SD對應於第一三維座標系統CY1,經最佳化第一轉換矩陣及待解第二轉換矩陣轉換的三維影像(二次轉換影像TD’)對應於座標系統CY2’。Please refer to FIG. 1 and FIG. 6 together to further explain the projection operation, wherein FIG. 6 is a schematic diagram of execution of the projection operation in the multi-mode image alignment method according to an embodiment of the present invention. As shown in FIG. 6, the first point D1 is a point corresponding to the central vertex of the
在投影作業的執行過程中,處理裝置11可以將第一點D1投影至二次轉換影像TD’,以取得對應於第一點D1的第五點D1’,且將第二點D1~D33投影至二次轉換影像TD’,以取得分別對應於第二點D31~D33的多個第六點D31’~D33’。以另個角度來說,處理裝置11可以將二次轉換影像TD’中具有第一點D1的x座標及y座標的點作為第五點D1’,且可以將二次轉換影像TD’中具有第二點D31~D33的x座標及y座標的點作為第六點D31’~D33’。 其中,第五點D1’及第六點D31’~D33’可以組成第四點群。對應於其他校正位置的第四點群的取得方式皆同理於上述,便不予贅述。During the execution of the projection operation, the
請一併參考圖1、圖5及圖7以進一步說明規格參數估算作業,圖7係依據本發明一實施例所繪示的多模影像對位方法中的規格參數估算作業的執行示意圖。在規格參數估算作業的執行過程中,處理裝置11取得第五點D1’與第六點D31’~D33’的多個連線,且計算所述多個連線的估算長度及連線之間的估算夾角,其中估算長度及估算夾角組成估算規格參數組。圖7示例性地標示第五點D1’與第六點D31’之連線的估算長度E及第五點D1’與第六點D31’之連線與第五點D1’與第六點D33’之連線的夾角A。Please refer to FIG. 1 , FIG. 5 , and FIG. 7 to further illustrate the standard parameter estimation operation. FIG. 7 is a schematic diagram of the standard parameter estimation operation in the multi-mode image alignment method according to an embodiment of the present invention. During the execution of the specification parameter estimating operation, the
以下進一步說明步驟S604。處理裝置11可以透過一收斂函式迭代調整待解第二轉換矩陣,且將經迭代調整的待解第二轉換矩陣作為最佳化第二轉換矩陣,其中收斂函式指示使所述多個估算規格參數組與校正體14的預設規格參數組間之差異為最小值。第二次最佳化運算所用之收斂函式異於第一次最佳化運算所用之收斂函式。校正體14的預設規格參數組可以包含校正體14的多個預設邊長及多個預設夾角。特別來說,圖7示例性地呈現第五點D1’及第六點D31’~D33’的理想位置I1~I4,所述多個預設邊長可以為理想位置I1~I4之間的多個連線的邊長S,所述多個預設夾角可以為理想位置I1~I4之間的連線的夾角R。Step S604 is further described below. The
步驟S604所述的估算規格參數組與預設規格參數組之間的差異可以指示第一數值與第二數值的加權和,其中第一數值指示所述多個估算長度分別與所述多個預設邊長的多個差值之和,且第二數值指示所述多個估算夾角分別與所述多個預設夾角的多個差值之和。步驟S604之收斂函式可以表示如式(2): (2) 其中 S 1 ~ S 3 指示預設邊長, R 1 ~ R 3 指示預設夾角, E 1 ~ E 3 指示估算長度, A 1 ~ A 3 指示估算夾角, α及 β指示可依所需調整之權重。 The difference between the estimated specification parameter set and the preset specification parameter set in step S604 may indicate a weighted sum of a first value and a second value, wherein the first value indicates that the multiple estimated lengths are respectively related to the multiple preset lengths. A sum of a plurality of difference values of side lengths is set, and a second value indicates a sum of a plurality of difference values of the plurality of estimated included angles and the plurality of preset included angles respectively. The convergence function of step S604 can be expressed as formula (2): (2) Among them, S 1 ~ S 3 indicate the preset side length, R 1 ~ R 3 indicate the preset included angle, E 1 ~ E 3 indicate the estimated length, A 1 ~ A 3 indicate the estimated included angle, α and β indicate the optional The weight to be adjusted.
特別來說,用於迭代產生最佳化第二轉換矩陣的待解第二轉換矩陣可以包含旋轉矩陣及位移矩陣,或可以僅包含旋轉矩陣。旋轉矩陣關聯於三個軸向的角度參數。位移矩陣關聯於三個軸向的位移參數。旋轉矩陣及位移矩陣的詳細參數組成係本發明所屬領域中具有通常知識者能夠基於上述六個參數並依所需而設計者,本發明不予限制。In particular, the second transformation matrix to be solved for iteratively generating the optimized second transformation matrix may include a rotation matrix and a displacement matrix, or may only include a rotation matrix. The rotation matrix is associated with the angle parameters of the three axes. The displacement matrix is associated with the displacement parameters of the three axes. The detailed parameter composition of the rotation matrix and the displacement matrix can be designed by those with ordinary knowledge in the field of the present invention based on the above six parameters and according to requirements, and the present invention is not limited thereto.
請再次參考圖1及圖3。經上述步驟取得最佳化第一轉換矩陣及最佳化第二轉換矩陣之後,於步驟S107中,處理裝置11可以利用最佳化第一轉換矩陣及最佳化第二轉換矩陣進行轉換,將待處理影像從第二三維座標系統轉換至第一三維座標系統,或從第一三維座標系統轉換至第二三維座標系統。如前所述,第一三維座標系統為二維影像擷取裝置12的相機座標系統。Please refer to Figure 1 and Figure 3 again. After obtaining the optimized first transformation matrix and the optimized second transformation matrix through the above steps, in step S107, the
進一步地,在將代處理影像轉換至第一三維座標系統之後,處理裝置11更可以利用二維影像擷取裝置12的參數矩陣,使待處理影像從第一三維座標系統轉換至二維影像擷取裝置12的影像平面座標系統。參數矩陣包含二維影像擷取裝置12的焦距參數及投影中心參數。處理裝置11可以最佳化第一轉換矩陣、最佳化第二轉換矩陣及參數矩陣組成用於執行二維影像擷取裝置12的影像平面座標系統與三維影像擷取裝置13的相機座標系統(第二三維座標系統)間之轉換的座標轉換矩陣,並將其儲存於內部記憶體。處理裝置11可以利用此座標轉換矩陣,將三維影像擷取裝置13所產生的影像映射至二維影像擷取裝置12所產生的二維影像,以產生疊合影像。Further, after converting the processing image to the first three-dimensional coordinate system, the
舉例來說,上述二維影像擷取裝置12的影像平面座標系統與三維影像擷取裝置13的相機座標系統間之轉換可依式(3)及(4)來執行:
(3)
(4)
其中
X、
Y及
Z表示三維影像上的座標,M
1表示最佳化第一轉換矩陣,M
2表示最佳化第二轉換矩陣,
f
x 及
f
y 表示二維影像擷取裝置12的焦距參數,
c
x 及
c
y 表示二維影像擷取裝置12的投影中心參數,且
x
t 及
y
t 表示二維影像上的座標。
For example, the conversion between the image plane coordinate system of the above-mentioned 2D
上列多個實施例所述之多模影像對位系統及方法可應用於設置有多個影像擷取裝置的使用環境,請一併參考圖1及圖8,圖8係依據本發明一實施例所繪示的多模影像對位方法的使用環境示意圖。圖8中的影像擷取裝置A1~A6的組成可以為二維影像擷取裝置及三維影像擷取裝置間隔設置。影像擷取裝置A1~A6中的任二相鄰者可以作為前述多模影像對位系統1中的二維影像擷取裝置12及三維影像擷取裝置13。於此要特別說明的是,圖8僅示例性地繪示校正體14的擺放位置,其擺放規則如前列實施例所述,於此不予贅述。The multi-mode image alignment system and method described in the above-mentioned embodiments can be applied to the use environment provided with multiple image capture devices. Please refer to FIG. 1 and FIG. 8 together. FIG. 8 is an implementation according to the present invention A schematic diagram of the usage environment of the multi-mode image alignment method shown in the example. The image capture devices A1 - A6 in FIG. 8 can be composed of two-dimensional image capture devices and three-dimensional image capture devices arranged at intervals. Any two adjacent ones of the image capture devices A1 - A6 can serve as the 2D
藉由前列多個實施例所述之多模影像對位方法,處理裝置11可以取得影像擷取裝置A1~A6中的任二相鄰者的座標系統之間的轉換矩陣,再透過串疊轉換架構使影像擷取裝置A1~A6中的任一者轉換至與之相隔一個以上的影像擷取裝置的座標系統。By using the multi-mode image alignment method described in the previous embodiments, the
舉例來說,假設處理裝置11藉由多模影像對位方法取得影像擷取裝置A1與影像擷取裝置A2的座標系統之間的轉換矩陣M
12,且取得影像擷取裝置A2與影像擷取裝置A3的座標系統之間的轉換矩陣M
23,處理裝置11可以藉由式(5)來將影像擷取裝置A1所產生之影像座標轉換至影像擷取裝置A3的座標系統:
P
3=M
23*M
12*P
1(5)
其中P1表示影像擷取裝置A1所產生之影像座標,P3表示所述影像座標轉換至影像擷取裝置A3的座標系統的座標。藉由上述串疊轉換架構,處理裝置11可以使所有影像擷取裝置A1~A6轉換至影像擷取裝置A1~A6中的特定者的座標系統。
For example, assume that the
於另一實施例中,多模影像對位系統可以包含三個影像擷取裝置且可以取得三個影像擷取裝置之座標系統間的座標轉換矩陣。請參考圖9,圖9示例性地繪示包含三個影像擷取裝置的多模影像對位系統1’的功能方塊圖。如圖9所示,多模影像對位系統1’包含處理裝置11、兩個二維影像擷取裝置12及15、三維影像擷取裝置13及校正體14。In another embodiment, the multi-mode image alignment system may include three image capture devices and may obtain coordinate transformation matrices between the coordinate systems of the three image capture devices. Please refer to FIG. 9 . FIG. 9 exemplarily shows a functional block diagram of a multi-mode image alignment system 1' including three image capture devices. As shown in FIG. 9 , the multi-mode image alignment system 1' includes a
相較於圖1所示的多模影像對位系統1,多模影像對位系統1’更包含二維影像擷取裝置15。二維影像擷取裝置15例如為可見光攝影機、近紅外光攝影機、熱像儀等,可以連接於處理裝置11,具有一相機座標系統及一影像平面座標系統,且可以受控以拍攝輪流擺放於不同校正位置的校正體14以產生多張二維影像。特別來說,二維影像擷取裝置15的相機座標系統及影像平面座標系統異於二維影像擷取裝置12的相機座標系統及影像平面座標系統。Compared with the multi-mode
多模影像對位系統1’的處理裝置11除了前列多個實施例所述的運作,更可以依據二維影像擷取裝置15所產生之關聯於校正體14的多張二維影像及前述三維影像擷取裝置13所產生之關聯於校正體14的多張三維影像,取得二維影像擷取裝置15的相機座標系統與三維影像擷取裝置13的相機座標系統之間的座標轉換矩陣。利用所述座標轉換矩陣及前列實施例所述之最佳化第一及第二轉換矩陣,處理裝置11可以進行二維影像擷取裝置12的相機座標系統與二維影像擷取裝置15的相機座標系統之間的轉換。利用所述座標轉換矩陣、前列實施例所述之最佳化第一及第二轉換矩陣、二維影像擷取裝置12的參數矩陣及二維影像擷取裝置15的參數矩陣,處理裝置11可以進行二維影像擷取裝置12的影像平面座標系統與二維影像擷取裝置15的影像平面座標系統之間的轉換。In addition to the operations described in the preceding embodiments, the
以下進一步說明適用於多模影像對位系統1’的多模影像對位方法。請一併參考圖3、圖9及圖10,其中圖10係依據本發明另一實施例所繪示的多模影像對位方法的流程圖。適用於多模影像對位系統1’的多模影像對位方法可以包含圖3所示的步驟S101~S107及圖9所示的步驟S108:取得關聯於校正體的多張第二二維影像,第二二維影像對應於第三三維座標系統;步驟S109:從所述多張第二二維影像取得對應於中央頂點的多個第七點及對應於所述多個側頂點的多個第八點群;步驟S110:利用待解第三轉換矩陣、所述多個第七點及所述多個第三點,基於第三三維座標系統執行第一最佳化運算,以取得最佳化第三轉換矩陣;步驟S111:利用最佳化第三轉換矩陣處理所述多張三維影像;步驟S112:利用經處理的三維影像、所述多個第七點、所述多個第八點群及校正體的預設規格參數組,基於第三三維座標系統執行第二最佳化運算,以取得最佳化第四轉換矩陣;以及步驟S113:利用最佳化第一轉換矩陣、最佳化第二轉換矩陣、最佳化第三轉換矩陣及最佳化第四轉換矩陣,執行第一三維座標系統與第三三維座標系統之間的轉換。於此要特別說明的是,本發明不限制步驟S101與S108的執行順序,不限制步驟S102、步驟S103與步驟S109的執行順序,亦不限制步驟S104~S106之組合與步驟S110~S112之組合的執行順序。The multi-mode image alignment method applicable to the multi-mode image alignment system 1' is further described below. Please refer to FIG. 3 , FIG. 9 and FIG. 10 together, wherein FIG. 10 is a flowchart of a multi-mode image alignment method according to another embodiment of the present invention. The multi-mode image alignment method applicable to the multi-mode image alignment system 1' may include steps S101-S107 shown in FIG. 3 and step S108 shown in FIG. 9: obtaining multiple second two-dimensional images associated with the calibration body , the second two-dimensional image corresponds to the third three-dimensional coordinate system; step S109: Obtain a plurality of seventh points corresponding to the central vertex and a plurality of seventh points corresponding to the plurality of side vertices from the plurality of second two-dimensional images The eighth point group; step S110: using the third transformation matrix to be solved, the plurality of seventh points and the plurality of third points, to perform a first optimization operation based on the third three-dimensional coordinate system to obtain an optimal Optimizing the third transformation matrix; step S111: using the optimized third transformation matrix to process the plurality of three-dimensional images; step S112: using the processed three-dimensional images, the plurality of seventh points, and the plurality of eighth points performing a second optimization operation based on the third three-dimensional coordinate system to obtain an optimized fourth transformation matrix; and step S113: using the optimized first transformation matrix, the optimal The second conversion matrix is optimized, the third conversion matrix is optimized, and the fourth conversion matrix is optimized to perform conversion between the first three-dimensional coordinate system and the third three-dimensional coordinate system. It should be noted here that the present invention does not limit the execution order of steps S101 and S108, does not limit the execution order of steps S102, S103 and step S109, and does not limit the combination of steps S104-S106 and the combination of steps S110-S112 order of execution.
步驟S101~步驟S113可以由多模影像對位系統1’的處理裝置11執行。步驟S108~S112的進一步實施方式分別同理於步驟S101、S102、S104~S106的進一步實施方式,換言之,S101、S102、S104~S106中是取得二維影像擷取裝置12之第一三維座標系統與三維影像擷取裝置13之第二三維座標系統間的轉換關係,而S108~S112則是基於相同的原理來取得二維影像擷取裝置15之第三三維座標系統與第二三維座標系統間的轉換關係,於此便不再贅述。於步驟S113中,處理裝置11可以利用最佳化第一轉換矩陣、最佳化第二轉換矩陣、最佳化第三轉換矩陣及最佳化第四轉換矩陣,進行第一三維座標系統與第三三維座標系統之間的轉換。進一步來說,最佳化第一轉換矩陣及最佳化第二轉換矩陣可以組成用於將待處理影像從第二三維座標系統轉換至第一三維座標系統的座標轉換矩陣M
a,最佳化第三轉換矩陣及最佳化第四轉換矩陣可以組成用於將將待處理影像從第二三維座標系統轉換至第三三維座標系統的座標轉換矩陣M
b。處理裝置11可以透過以座標轉換矩陣M
a及M
b進行代數運算而取得用於將待處理影像從第一三維座標系統轉換至第三三維座標系統的座標轉換矩陣M
c,例如表示為式(6):
M
c=M
b*M
a -1(6)
Step S101 to step S113 can be executed by the
更進一步來說,座標轉換矩陣M
a可以更包含二維影像擷取裝置12的參數矩陣,座標轉換矩陣M
b可以更包含二維影像擷取裝置15的參數矩陣。座標轉換矩陣M
a所包含的最佳化第一轉換矩陣、最佳化第二轉換矩陣及參數矩陣之間的關係如前列式(3)及(4)所示,座標轉換矩陣M
b所包含的最佳化第三轉換矩陣、最佳化第四轉換矩陣及參數矩陣之間的關係亦同,於此便不再贅述。於此實施態樣中,經上述代數運算所得之座標轉換矩陣M
c可用於進行二維影像擷取裝置12的影像平面座標系統與二維影像擷取裝置15的影像平面座標系統之間的轉換。
Furthermore, the coordinate transformation matrix M a may further include a parameter matrix of the 2D
利用座標轉換矩陣M
a、M
b及M
c,處理裝置11可以將二維影像擷取裝置12及15及三維影像擷取裝置13中任二者所產生的影像映射至剩餘者所產生的影像。舉例來說,利用座標轉換矩陣M
b及座標轉換矩陣M
c,處理裝置11可以將三維影像擷取裝置13所產生的影像及二維影像擷取裝置12所產生的影像映射至二維影像擷取裝置15所產生的影像,以產生疊合影像。如此一來,多模影像對位系統1’可以從疊合影像一次取得對應於特定標的物的三種資訊。舉例來說,若多模影像對位系統1’的三個影像擷取裝置分別為熱像儀、可見光相機及三維點雲感測器,則多模影像對位系統1’的處理裝置11可以從疊合影像上一次取得特定標的物的溫度、色彩及空間位置資訊。
Using the coordinate transformation matrices M a , M b and M c , the
藉由上述架構,本案所揭示的多模影像對位方法可以透過兩次最佳化運算取得不同座標系統之間的轉換矩陣,不須使用複雜的機器學習訓練即可達成高精準度的對位效果,且透過以三維角點特徵為求取轉換矩陣之根據,相較於傳統以平面式棋盤校正板為求取轉換矩陣之根據,所需之取樣資料數量甚少,即所需取樣時間較少。本案所揭示的多模影像對位系統同樣具有所需之取樣資料數量及取樣時間少的效果,且藉由設置有指示元件之特殊立體校正體設計,系統可以實現二維/三維影像中之特徵點的自動擷取。With the above structure, the multi-mode image alignment method disclosed in this case can obtain the transformation matrix between different coordinate systems through two optimization operations, and can achieve high-precision alignment without using complicated machine learning training effect, and by using the three-dimensional corner features as the basis for obtaining the transformation matrix, compared with the traditional planar checkerboard correction board as the basis for obtaining the transformation matrix, the amount of sampling data required is very small, that is, the required sampling time is shorter few. The multi-mode image alignment system disclosed in this case also has the effect of less sampling data and less sampling time, and the system can realize the characteristics of 2D/3D images through the design of a special stereo calibration body equipped with indicating elements Automatic extraction of points.
雖然本發明以前述之實施例揭露如上,然其並非用以限定本發明。在不脫離本發明之精神和範圍內,所為之更動與潤飾,均屬本發明之專利保護範圍。關於本發明所界定之保護範圍請參考所附之申請專利範圍。Although the present invention is disclosed by the aforementioned embodiments, they are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, all changes and modifications are within the scope of patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the appended scope of patent application.
1,1’:多模影像對位系統
11:處理裝置
12,15:二維影像擷取裝置
13:三維影像擷取裝置
14:校正體
140:立體本體
141a~141:指示元件
P1~P6:校正位置
v1,v2:視角
CY1:第一三維座標系統
CY2:第二三維座標系統
CY2’:座標系統
SD:二維影像
TD:三維影像
TD’:二次轉換影像
D1:第一點
D1’:第五點
D2:第三點
L1:射線
d:距離
D31~D33:第二點
D31’~D33’:第六點
E:估算長度
A:估算夾角
I1~I4:理想位置
S:邊長
R:夾角
A1~A6:影像擷取裝置
S101~S107:步驟
S601~S604:步驟
S108~S113:步驟
1,1': Multi-mode image alignment system
11: Processing
圖1係依據本發明一實施例所繪示的多模影像對位系統的功能方塊圖。 圖2係依據本發明一實施例所繪示的校正位置示意圖。 圖3係依據本發明一實施例所繪示的多模影像對位方法的流程圖。 圖4係依據本發明一實施例所繪示的多模影像對位方法中的距離計算作業的執行示意圖。 圖5係依據本發明一實施例所繪示的多模影像對位方法中的第二最佳化運算的流程圖。 圖6係依據本發明一實施例所繪示的多模影像對位方法中的投影作業的執行示意圖。 圖7係依據本發明一實施例所繪示的多模影像對位方法中的規格參數估算作業的執行示意圖。 圖8係依據本發明一實施例所繪示的多模影像對位方法的使用環境示意圖。 圖9係依據本發明另一實施例所繪示的多模影像對位系統的功能方塊圖。 圖10係依據本發明另一實施例所繪示的多模影像對位方法的流程圖。 FIG. 1 is a functional block diagram of a multi-mode image alignment system according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a calibration position according to an embodiment of the present invention. FIG. 3 is a flow chart of a multi-mode image alignment method according to an embodiment of the present invention. FIG. 4 is a schematic diagram illustrating the execution of a distance calculation operation in a multi-mode image alignment method according to an embodiment of the present invention. FIG. 5 is a flow chart of the second optimization operation in the multi-mode image alignment method according to an embodiment of the present invention. FIG. 6 is a schematic diagram illustrating execution of a projection operation in a multi-mode image alignment method according to an embodiment of the present invention. FIG. 7 is a schematic diagram of execution of specification parameter estimation in the multi-mode image alignment method according to an embodiment of the present invention. FIG. 8 is a schematic diagram of the usage environment of the multi-mode image alignment method according to an embodiment of the present invention. FIG. 9 is a functional block diagram of a multi-mode image alignment system according to another embodiment of the present invention. FIG. 10 is a flowchart of a multi-mode image alignment method according to another embodiment of the present invention.
S101~S107:步驟 S101~S107: steps
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210835561.8A CN115908152A (en) | 2021-08-19 | 2022-07-15 | Multimode image alignment method and system |
US17/866,354 US20230055649A1 (en) | 2021-08-19 | 2022-07-15 | Multi-modal image alignment method and system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202163234924P | 2021-08-19 | 2021-08-19 | |
US63/234,924 | 2021-08-19 |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI790854B true TWI790854B (en) | 2023-01-21 |
TW202309833A TW202309833A (en) | 2023-03-01 |
Family
ID=86670250
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW110146318A TWI790854B (en) | 2021-08-19 | 2021-12-10 | Multi-modal image alignment method and system |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI790854B (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106846461A (en) * | 2016-12-30 | 2017-06-13 | 西安交通大学 | A kind of human body three-dimensional scan method |
US20190279399A1 (en) * | 2018-03-08 | 2019-09-12 | Toshiba Tec Kabushiki Kaisha | Coordinate calibration between two dimensional coordinate system and three dimensional coordinate system |
CN110648367A (en) * | 2019-08-15 | 2020-01-03 | 大连理工江苏研究院有限公司 | Geometric object positioning method based on multilayer depth and color visual information |
-
2021
- 2021-12-10 TW TW110146318A patent/TWI790854B/en active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106846461A (en) * | 2016-12-30 | 2017-06-13 | 西安交通大学 | A kind of human body three-dimensional scan method |
US20190279399A1 (en) * | 2018-03-08 | 2019-09-12 | Toshiba Tec Kabushiki Kaisha | Coordinate calibration between two dimensional coordinate system and three dimensional coordinate system |
CN110648367A (en) * | 2019-08-15 | 2020-01-03 | 大连理工江苏研究院有限公司 | Geometric object positioning method based on multilayer depth and color visual information |
Also Published As
Publication number | Publication date |
---|---|
TW202309833A (en) | 2023-03-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5999615B2 (en) | Camera calibration information generating apparatus, camera calibration information generating method, and camera calibration information generating program | |
US20170127045A1 (en) | Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof | |
US11323674B2 (en) | Projection device, projection system and image correction method | |
JP6359259B2 (en) | Depth image correction apparatus and method based on relationship between depth sensor and photographing camera | |
JP6573419B1 (en) | Positioning method, robot and computer storage medium | |
US20170032565A1 (en) | Three-dimensional facial reconstruction method and system | |
JP2017112602A (en) | Image calibrating, stitching and depth rebuilding method of panoramic fish-eye camera and system thereof | |
TW201812700A (en) | Measurement systems and methods for measuring multi-dimensions | |
JP2009139246A (en) | Device, method and program for processing image, position detecting device and moving body equipped with this | |
WO2018032841A1 (en) | Method, device and system for drawing three-dimensional image | |
WO2023201578A1 (en) | Extrinsic parameter calibration method and device for monocular laser speckle projection system | |
KR101639065B1 (en) | System for registration of virtual space and real space, method for registering display apparatus and image sensor, and electronic device registered using the method | |
WO2022218161A1 (en) | Method and apparatus for target matching, device, and storage medium | |
JP2010276433A (en) | Imaging device, image processor, and distance measuring device | |
WO2021238070A1 (en) | Three-dimensional image generation method and apparatus, and computer device | |
CN107864372A (en) | Solid picture-taking method, apparatus and terminal | |
US20230055649A1 (en) | Multi-modal image alignment method and system | |
TWI790854B (en) | Multi-modal image alignment method and system | |
TWI731430B (en) | Information display method and information display system | |
TWM594322U (en) | Camera configuration system with omnidirectional stereo vision | |
TWI662694B (en) | 3d image capture method and system | |
CN114004880B (en) | Point cloud and strong reflection target real-time positioning method of binocular camera | |
US20160295187A1 (en) | Information processing device and projection apparatus | |
JP2010216969A (en) | System, program and mark set for position measurement, and object for position measurement | |
TWI811162B (en) | Video data conversion device |