TWI516120B - Method for generating panoramic image and image capturing device thereof - Google Patents

Method for generating panoramic image and image capturing device thereof Download PDF

Info

Publication number
TWI516120B
TWI516120B TW101111116A TW101111116A TWI516120B TW I516120 B TWI516120 B TW I516120B TW 101111116 A TW101111116 A TW 101111116A TW 101111116 A TW101111116 A TW 101111116A TW I516120 B TWI516120 B TW I516120B
Authority
TW
Taiwan
Prior art keywords
image
pedestrian
overlapping area
original
humanoid
Prior art date
Application number
TW101111116A
Other languages
Chinese (zh)
Other versions
TW201340706A (en
Inventor
周宏隆
楊岱璋
張文彥
Original Assignee
華晶科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 華晶科技股份有限公司 filed Critical 華晶科技股份有限公司
Priority to TW101111116A priority Critical patent/TWI516120B/en
Publication of TW201340706A publication Critical patent/TW201340706A/en
Application granted granted Critical
Publication of TWI516120B publication Critical patent/TWI516120B/en

Links

Description

產生全景影像的方法及其影像擷取裝置Method for generating panoramic image and image capturing device thereof

本發明是有關於一種影像處理方法及裝置,且特別是有關於一種產生全景影像的方法及其影像擷取裝置。The present invention relates to an image processing method and apparatus, and more particularly to a method for generating a panoramic image and an image capturing device thereof.

習知產生全景影像的技術是利用相機連續平移以拍攝同一場景但不同視角的多張影像,並藉由估計相鄰影像間的移動據以拼接影像。由於相機在連續平移拍攝的過程具有時間差,因此若拍攝場景中具有正在移動的行人,將導致拼接後的影像產生殘影、疊影等問題。以下將以數個範例說明拼接後的全景影像可能產生的情況。Conventionally, a technique for generating a panoramic image is to use a camera to continuously pan to capture multiple images of the same scene but different viewing angles, and to stitch the images by estimating the movement between adjacent images. Since the camera has a time difference in the process of continuous panning, if there is a moving pedestrian in the shooting scene, the stitched image will cause problems such as afterimage and overlay. The following will illustrate the possible situation of the stitched panoramic image with several examples.

圖1(a)至圖1(d)繪示為拍攝場景中具有正在移動的行人所擷取的兩張待拼接影像的示意圖。首先請參照圖1(a),假設待拼接影像Img1中具有行人10,然而在待拼接影像Img2中卻沒有行人。習知將待拼接影像Img1、Img2進行拼接混合之後,將可能使行人10與影像中的背景融合而導致行人10的影像模糊甚至消失。接著請參照圖1(b),假設待拼接影像Img1中具有行人11,在待拼接影像Img2中亦具有相同的行人11,然而,行人11因移動位置的關係,將待拼接影像Img1、Img2進行拼接混合之後,將可能使拼接後的行人11產生多手多腳或者變成存在兩個行人的全景影像。1(a) to 1(d) are schematic diagrams showing two images to be stitched captured by a pedestrian who is moving in a shooting scene. First, please refer to FIG. 1(a), assuming that there is a pedestrian 10 in the image Img1 to be spliced, but there is no pedestrian in the image Img2 to be spliced. It is conventional to splicing and mixing the images to be stitched Img1 and Img2, which may cause the pedestrian 10 to merge with the background in the image to cause the image of the pedestrian 10 to be blurred or even disappear. Next, referring to FIG. 1(b), it is assumed that there is a pedestrian 11 in the image Img1 to be spliced, and the same pedestrian 11 is also present in the image Img2 to be spliced. However, the pedestrian 11 performs the image Img1 and Img2 to be spliced due to the position of the moving position. After splicing and mixing, it is possible to cause the spliced pedestrian 11 to generate multiple hands or feet or become a panoramic image with two pedestrians.

另請參照圖1(c),假設待拼接影像Img1中具有行人12,然而行人12僅擷取到部份的影像,而在待拼接影像Img2中並沒有行人12之影像。將待拼接影像Img1、Img2進行拼接混合之後,將可能出現半個行人12的全景影像。再請參照圖1(d),假設待拼接影像Img1中具有行人13,而在待拼接影像Img2中亦具有相同的行人13,然而行人13因移動速度較快而使得行人13在待拼接影像Img1、Img2中之位置相距較遠。將待拼接影像Img1、Img2進行拼接混合之後,可能使全景影像在不同的位置同時出現相同的行人13。Referring to FIG. 1(c), it is assumed that there is a pedestrian 12 in the image Img1 to be stitched, but the pedestrian 12 only captures part of the image, and there is no image of the pedestrian 12 in the image Img2 to be stitched. After splicing and mixing the images to be spliced Img1, Img2, a panoramic image of half a pedestrian 12 may appear. Referring to FIG. 1(d), it is assumed that there is a pedestrian 13 in the image Img1 to be spliced, and the same pedestrian 13 is also included in the image Img2 to be spliced. However, the pedestrian 13 is in the image to be spliced Img1 due to the faster moving speed. The positions in Img2 are far apart. After the spliced and mixed images Img1 and Img2 are spliced and mixed, it is possible to cause the panoramic image to simultaneously appear the same pedestrian 13 at different positions.

對於上述拍攝場景中具有正在移動的行人所產生的問題,實有必要提供一種能夠快速且正確地進行拼接處理的全景影像產生方法,以提高全景影像之品質。For the problem of having a moving pedestrian in the above shooting scene, it is necessary to provide a panoramic image generating method capable of quickly and correctly performing splicing processing to improve the quality of the panoramic image.

有鑑於此,本發明提供一種產生全景影像的方法及其影像擷取裝置,可減少因擷取時間差造成影像中的行人在接圖過程中產生殘影的問題,以提升全景影像之品質。In view of the above, the present invention provides a method for generating a panoramic image and an image capturing device thereof, which can reduce the problem that a pedestrian in the image generates a residual image during the connection process due to the time difference of the capturing, so as to improve the quality of the panoramic image.

本發明提出一種產生全景影像的方法,適用於影像擷取裝置,影像擷取裝置針對具有一或多個行人的拍攝場景進行拍攝。此方法包括下列步驟。先取得第一原始影像與第二原始影像,並且定義第一原始影像中的第一重疊區域與第二原始影像中的第二重疊區域。接著,針對第一原始影像與第二原始影像執行人形偵測程序與人形追蹤程序,以獲得關於上述行人的行人資訊。然後,依據行人資訊分別調整第一重疊區域與第二重疊區域中一或多個行人的權重設定值。再依據此權重設定值拼接第一與第二原始影像,以產生全景影像。The invention provides a method for generating a panoramic image, which is suitable for an image capturing device, and the image capturing device performs shooting for a shooting scene with one or more pedestrians. This method includes the following steps. First obtaining the first original image and the second original image, and defining a first overlapping area in the first original image and a second overlapping area in the second original image. Next, a humanoid detection program and a humanoid tracking program are executed for the first original image and the second original image to obtain pedestrian information about the pedestrian. Then, the weight setting values of one or more pedestrians in the first overlapping area and the second overlapping area are respectively adjusted according to the pedestrian information. The first and second original images are then stitched according to the weight setting value to generate a panoramic image.

在本發明之一實施例中,其中對第一原始影像與第二原始影像執行人形偵測程序的步驟包括分別判斷第一重疊區域與第二重疊區域中是否具有行人影像。若是,則分別記錄行人影像之位置與形狀,以作為行人資訊。In an embodiment of the invention, the step of executing the humanoid detection program on the first original image and the second original image includes determining whether the first overlapping area and the second overlapping area respectively have a pedestrian image. If so, the position and shape of the pedestrian image are recorded separately as pedestrian information.

在本發明之一實施例中,其中若判斷第一重疊區域與第二重疊區域中皆具有行人影像,則針對第一原始影像與第二原始影像執行人形追蹤程序的步驟包括利用特徵點追蹤法以判斷第一重疊區域中的行人影像與第二重疊區域中的行人影像是否為同一人,並將判斷結果作為行人資訊。In an embodiment of the present invention, if it is determined that both the first overlapping area and the second overlapping area have a pedestrian image, the step of performing a humanoid tracking procedure for the first original image and the second original image comprises using feature point tracking It is determined whether the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, and the judgment result is used as pedestrian information.

在本發明之一實施例中,其中若行人資訊顯示第一重疊區域中的行人影像與第二重疊區域中的行人影像並非為同一人,則調整第一重疊區域與第二重疊區域中一或多個行人的權重設定值的步驟包括分別提高第一重疊區域中的行人影像與第二重疊區域中的行人影像之權重設定值。In an embodiment of the present invention, if the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person, adjusting one of the first overlapping area and the second overlapping area The step of setting the weights of the plurality of pedestrians includes increasing the weight setting values of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area, respectively.

在本發明之一實施例中,其中若行人資訊顯示第一重疊區域中的行人影像與第二重疊區域中的行人影像為同一人,則調整第一重疊區域與第二重疊區域的權重設定值的步驟包括將第一重疊區域與第二重疊區域其中之一的行人影像之權重設定值設定為100%,並且將另一行人影像之權重設定值設定為0%。In an embodiment of the present invention, if the pedestrian information shows that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, adjusting the weight setting values of the first overlapping area and the second overlapping area The step of setting the weight setting value of the pedestrian image of one of the first overlapping area and the second overlapping area to 100%, and setting the weight setting value of another pedestrian image to 0%.

在本發明之一實施例中,其中若判斷第一重疊區域與第二重疊區域僅有其中之一具有行人影像,更包括判斷行人影像是否為完整的影像,若行人影像並非為完整的影像,則將行人影像之權重設定值設定為0%。In an embodiment of the present invention, if it is determined that only one of the first overlapping area and the second overlapping area has a pedestrian image, the method further includes determining whether the pedestrian image is a complete image, and if the pedestrian image is not a complete image, Then set the weight setting value of the pedestrian image to 0%.

本發明另提出一種產生全景影像的影像擷取裝置,適於針對具有一或多個行人的拍攝場景進行拍攝。此影像擷取裝置包括取圖單元、人形偵測與追蹤單元以及影像拼接單元。其中,取圖單元用以取得第一原始影像與第二原始影像,並且定義第一原始影像中的第一重疊區域與第二原始影像中的第二重疊區域。人形偵測與追蹤單元耦接至取圖單元,用以對第一原始影像與第二原始影像進行人形偵測與人形追蹤,以產生關於上述行人的行人資訊。影像拼接單元分別耦接至取圖單元及人形偵測與追蹤單元。影像拼接單元依據行人資訊分別調整第一重疊區域與第二重疊區域中一或多個行人的權重設定值,並且依據此權重設定值拼接第一原始影像與第二原始影像,以產生全景影像。The invention further provides an image capturing device for generating a panoramic image, which is suitable for shooting for a shooting scene with one or more pedestrians. The image capturing device comprises a drawing unit, a humanoid detection and tracking unit and an image splicing unit. The drawing unit is configured to obtain the first original image and the second original image, and define a first overlapping area in the first original image and a second overlapping area in the second original image. The humanoid detection and tracking unit is coupled to the image capturing unit for performing humanoid detection and humanoid tracking on the first original image and the second original image to generate pedestrian information about the pedestrian. The image splicing unit is respectively coupled to the drawing unit and the human figure detecting and tracking unit. The image splicing unit adjusts the weight setting values of the one or more pedestrians in the first overlapping area and the second overlapping area according to the pedestrian information, and splicing the first original image and the second original image according to the weight setting value to generate the panoramic image.

基於上述,本發明所提供之產生全景影像的方法及其影像擷取裝置,藉由人形的偵測與追蹤而獲得行人資訊,並依據此行人資訊調整權重值,再依照調整後的權重值進行影像拼接,可減少因擷取時間差造成影像中的行人在接圖過程中產生殘影的問題。Based on the above, the method for generating a panoramic image and the image capturing device thereof are provided by the present invention, and the pedestrian information is obtained by the human body detection and tracking, and the weight value is adjusted according to the pedestrian information, and then according to the adjusted weight value. Image stitching can reduce the problem of pedestrians in the image causing residual image during the connection process due to the time difference of the capture.

為讓本發明之上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明如下。The above described features and advantages of the present invention will be more apparent from the following description.

在拍攝全景圖的過程中,經常因為拍攝時間差導致拍攝場景中行人的位置因移動而出現錯位、殘影等問題,本發明提出一種藉由人形的偵測與追蹤而獲得行人資訊,並依據此行人資訊進行影像接合區域的權重值調整,依照調整後的權重值進行影像拼接可減少行人在全景接圖過程中產生不必要的瑕疵,藉以提供高品質的全景影像。為了使本發明之內容更為明瞭,以下列舉實施例作為本發明確實能夠據以實施的範例。In the process of shooting a panorama, the position of the pedestrian in the shooting scene often causes misalignment, residual image and the like due to the movement due to the difference in shooting time. The present invention provides a pedestrian information by means of humanoid detection and tracking, and according to this Pedestrian information is used to adjust the weight value of the image joint area, and the image stitching according to the adjusted weight value can reduce unnecessary defects in the panorama connection process, thereby providing high-quality panoramic images. In order to clarify the content of the present invention, the following examples are given as examples in which the present invention can be implemented.

圖2是依照本發明一實施例所繪示之產生全景影像的影像擷取裝置的方塊圖。請參照圖2,本實施例之影像擷取裝置200例如是數位相機、單眼相機、數位攝影機或是其他具有產生全景影像功能的智慧型手機、平板電腦等,不限於上述。影像擷取裝置200包括取圖單元210、人形偵測與追蹤單元220以及影像拼接單元230,分別可為硬體及/或軟體所實現的功能模塊,其功能分述如下:影像擷取裝置200適於針對具有一或多個行人的拍攝場景進行拍攝,並可將連續拍攝所獲得的多張影像儲存於記憶體(未繪示)中。當影像擷取裝置200欲執行全景影像合成功能時,取圖單元210用以自記憶體中取得第一原始影像Iori1與第二原始影像Iori2以提供給人形偵測與追蹤單元220進行人形偵測(human detection)與人形追蹤(human tracking),進而產生關於拍攝場景中的行人資訊。影像拼接單元230接收此行人資訊後據以調整權重設定值,利用權重設定值決定拼接後的全景影像Ipan是否保留或者移除原始影像中所出現的行人。據此,可避免影像中的行人在接圖過程中產生殘影或如先前技術所述不合理影像的問題。 FIG. 2 is a block diagram of an image capturing device for generating a panoramic image according to an embodiment of the invention. Referring to FIG. 2, the image capturing device 200 of the present embodiment is, for example, a digital camera, a monocular camera, a digital camera, or another smart phone, a tablet computer or the like having a function of generating a panoramic image, and is not limited to the above. The image capturing device 200 includes a drawing unit 210, a humanoid detecting and tracking unit 220, and an image splicing unit 230, which can be functional modules implemented by hardware and/or software, and the functions thereof are as follows: the image capturing device 200 It is suitable for shooting for shooting scenes with one or more pedestrians, and can store multiple images obtained by continuous shooting in a memory (not shown). When the image capturing device 200 is to perform the panoramic image synthesizing function, the drawing unit 210 is configured to obtain the first original image Iori1 and the second original image Iori2 from the memory for providing the humanoid detection and tracking unit 220 for humanoid detection. (human detection) and human tracking, which in turn produces information about pedestrians in the scene. The image splicing unit 230 receives the pedestrian information and adjusts the weight setting value, and uses the weight setting value to determine whether the spliced panoramic image Ipan retains or removes the pedestrian appearing in the original image. According to this, it is possible to avoid the problem that the pedestrian in the image generates a residual image or an unreasonable image as described in the prior art.

圖3是依照本發明一實施例所繪示之一種產生全景影像的方法流程圖。本實施例的方法適用於圖2的影像擷取裝置200,以下即搭配影像擷取裝置200中的各構件說明本實施例方法的詳細步驟:首先,於步驟S310中,藉由移動影像擷取裝置200以連續拍攝一場景中相鄰區域的多張影像,其中,該拍攝場景中具有靜止或移動中的一或多個行人。取圖單元210自該些影像中取得欲進行全景影像合成的第一原始影像Iori1與第二原始影像Iori2。並且取圖單元210定義第一原始影像Iori1中的第一重疊區域R1與第二原始影像Iori2中的第二重疊區域R2。其中第一重疊區域R1與第二重疊區域R2係代表第一原始影像Iori1與第二原始影像Iori2欲進行拼接的影像接合區域,其通常為背景幾近相同之區域。 FIG. 3 is a flow chart of a method for generating a panoramic image according to an embodiment of the invention. The method of the present embodiment is applicable to the image capturing device 200 of FIG. 2, and the detailed steps of the method of the embodiment are described below with the components in the image capturing device 200. First, in step S310, the image is captured by moving the image. The device 200 continuously captures a plurality of images of adjacent regions in a scene, wherein the scene has one or more pedestrians in still or moving. The image capturing unit 210 acquires the first original image Iori1 and the second original image Iori2 to be combined with the panoramic image from the images. And the drawing unit 210 defines a first overlapping area R1 in the first original image Iori1 and a second overlapping area R2 in the second original image Iori2. The first overlapping area R1 and the second overlapping area R2 represent image joining areas where the first original image Iori1 and the second original image Iori2 are to be stitched, which are generally the same area of the background.

接著,於步驟S320中,人形偵測與追蹤單元220針對第一原始影像Iori1與第二原始影像Iori2執行人形偵測與人形追蹤,以獲得關於上述行人的行人資訊。詳細而言,人形偵測與追蹤單元220先執行人形偵測,分別判斷第一原始影像Iori1與第二原始影像Iori2中是否具有行人影像。特別是,人形偵測與追蹤單元220會偵測第一重疊區域R1與第二重疊區域R2中是否具有行人影像。若是,則 分別記錄此行人影像在第一原始影像Iori1之位置與形狀以及在第二原始影像Iori2之位置與形狀,以作為行人資訊。舉例來說,可將第一與第二原始影像Iori1、Iori2分別視為二維直角座標系,藉由記錄行人影像之像素的座標位置,便可提供後續接圖處理的判斷依據。 Next, in step S320, the humanoid detection and tracking unit 220 performs humanoid detection and humanoid tracking for the first original image Iori1 and the second original image Iori2 to obtain pedestrian information about the pedestrian. In detail, the humanoid detection and tracking unit 220 first performs humanoid detection to determine whether there is a pedestrian image in the first original image Iori1 and the second original image Iori2, respectively. In particular, the humanoid detection and tracking unit 220 detects whether there is a pedestrian image in the first overlapping area R1 and the second overlapping area R2. If yes, then The position and shape of the pedestrian image in the first original image Iori1 and the position and shape of the second original image Iori2 are separately recorded as pedestrian information. For example, the first and second original images Iori1 and Iori2 can be regarded as a two-dimensional right-angle coordinate system respectively. By recording the coordinate position of the pixel of the pedestrian image, the judgment basis of the subsequent connection processing can be provided.

其中若偵測出第一原始影像Iori1與第二原始影像Iori2皆具有行人影像時,則人形偵測與追蹤單元220更包括利用特徵點追蹤法以判斷第一重疊區域R1中的行人影像與第二重疊區域R2中所出現的行人影像是否為同一人,並將判斷結果作為行人資訊。須說明的是,若影像擷取裝置200提供人臉辨識功能,則在此步驟中亦可利用人臉辨識輔以判斷第一原始影像Iori1與第二原始影像Iori2中所出現的行人影像是否為同一人。 If the first original image Iori1 and the second original image Iori2 are both detected as having a pedestrian image, the humanoid detection and tracking unit 220 further includes using the feature point tracking method to determine the pedestrian image in the first overlapping region R1. Whether the pedestrian image appearing in the two overlapping regions R2 is the same person, and the judgment result is used as pedestrian information. It should be noted that if the image capturing device 200 provides the face recognition function, the face recognition may be used in this step to determine whether the pedestrian image appearing in the first original image Iori1 and the second original image Iori2 is same person.

接下來,在步驟S330中,影像拼接單元230接收人形偵測與追蹤單元220所記錄的行人資訊,並依據包括行人位置、形狀等等的行人資訊分別調整第一重疊區域R1與第二重疊區域R2的權重設定值。權重設定值的調整將於稍候做更詳細的說明。 Next, in step S330, the image splicing unit 230 receives the pedestrian information recorded by the humanoid detection and tracking unit 220, and adjusts the first overlapping region R1 and the second overlapping region according to the pedestrian information including the pedestrian position, shape, and the like, respectively. R2 weight setting value. The adjustment of the weight setting will be explained in more detail later.

最後,在步驟S340中,影像拼接單元230依據調整後的權重設定值拼接第一原始影像Iori1與第二原始影像Iori2,以產生全景影像Ipan。 Finally, in step S340, the image splicing unit 230 splices the first original image Iori1 and the second original image Iori2 according to the adjusted weight setting value to generate the panoramic image Ipan.

接下來,圖4(a)至圖4(d)繪示為具有不同行人資訊的第一原始影像與第二原始影像的簡易示意圖,其係為了輔助說明如何依據不同行人資訊進而調整權重設定值。以下請配合參照圖4(a)至圖4(d)。Next, FIG. 4( a ) to FIG. 4( d ) are schematic diagrams showing a first original image and a second original image with different pedestrian information, which are used to help explain how to adjust the weight setting value according to different pedestrian information. . Please refer to FIG. 4(a) to FIG. 4(d) below.

若行人資訊顯示第一原始影像Iori1與第二原始影像Iori2僅有其中之一具有行人影像,且行人影像位於重疊區域之內。則人形偵測與追蹤單元220更包括偵測行人影像是否為完整的影像,並進一步記錄偵測結果以作為行人資訊之一部分。圖4(a)與圖4(c)所示之實施例即為第一原始影像Iori1與第二原始影像Iori2之中僅有其中之一具有行人影像。If the pedestrian information shows that only one of the first original image Iori1 and the second original image Iori2 has a pedestrian image, and the pedestrian image is located within the overlapping area. The humanoid detection and tracking unit 220 further includes detecting whether the pedestrian image is a complete image, and further recording the detection result as part of the pedestrian information. The embodiment shown in FIG. 4(a) and FIG. 4(c) is that only one of the first original image Iori1 and the second original image Iori2 has a pedestrian image.

其中,如圖4(c)所示之實施例,行人影像43在重疊區域R1中僅顯示一部分影像,則影像拼接單元230將行人影像43之權重設定值設定為0%。因行人影像43之資訊並不完整,將其權重設定值設定為0%後,可使行人影像43避免出現在拼接後的全景影像Ipan中。相反地,如圖4(a)所示之實施例,行人影像40為完整的影像,則影像拼接單元230可加強行人影像40之權重設定值,使其更能完整清楚的呈現在全景影像Ipan中。其中權重設定值之加強比例可由使用者依實際情況做設定,在此不加以限制。As shown in FIG. 4(c), when the pedestrian image 43 displays only a part of the image in the overlap region R1, the image splicing unit 230 sets the weight setting value of the pedestrian image 43 to 0%. Since the information of the pedestrian image 43 is not complete, after setting the weight setting value to 0%, the pedestrian image 43 can be prevented from appearing in the stitched panoramic image Ipan. Conversely, as shown in FIG. 4( a ), the pedestrian image 40 is a complete image, and the image splicing unit 230 can enhance the weight setting value of the pedestrian image 40 to make it more completely and clearly displayed in the panoramic image Ipan. in. The enhancement ratio of the weight setting value can be set by the user according to the actual situation, and is not limited herein.

若行人資訊顯示第一原始影像Iori1與第二原始影像Iori2皆具有行人影像,且行人影像皆位於重疊區域R1與R2之內。則藉由人形偵測與追蹤單元220執行人形追蹤程序以判斷第一重疊區域R1中的行人影像與第二重疊區域R2中的行人影像是否為同一人。If the pedestrian information shows that both the first original image Iori1 and the second original image Iori2 have a pedestrian image, and the pedestrian images are located within the overlapping regions R1 and R2. Then, the humanoid tracking program is executed by the humanoid detection and tracking unit 220 to determine whether the pedestrian image in the first overlapping region R1 and the pedestrian image in the second overlapping region R2 are the same person.

如圖4(b)所示之實施例,第一重疊區域R1中具有行人影像41,第二重疊區域R2中具有行人影像42。若行人影像41與行人影像42為同一人,則可選擇將行人影像41、42其中之一的權重設定值設定為100%,並將另一行人影像之權重設定值設定為0%。經此調整後,可避免拼接後的全景影像Ipan產生多手多腳或重疊的影像。相反地,若行人影像41與行人影像42並非為同一人,則可選擇分別提高行人影像41與行人影像42之權重設定值,使得行人影像41與42皆完整的呈現在拼接後的全景影像Ipan中。As shown in FIG. 4(b), the first overlapping area R1 has a pedestrian image 41, and the second overlapping area R2 has a pedestrian image 42. If the pedestrian image 41 and the pedestrian image 42 are the same person, the weight setting value of one of the pedestrian images 41, 42 may be set to 100%, and the weight setting value of another pedestrian image may be set to 0%. After this adjustment, it is possible to prevent the stitched panoramic image Ipan from generating multi-footed or overlapping images. Conversely, if the pedestrian image 41 and the pedestrian image 42 are not the same person, the weight setting values of the pedestrian image 41 and the pedestrian image 42 can be separately increased, so that the pedestrian images 41 and 42 are completely presented in the stitched panoramic image Ipan. in.

若行人資訊顯示第一原始影像Iori1與第二原始影像Iori2皆具有行人影像,且其中之一的行人影像位於重疊區域之內,另一行人影像位於重疊區域之外。則藉由人形偵測與追蹤單元220執行人形追蹤程序以判斷第一原始影像Iori1中的行人影像與第二原始影像Iori2中的行人影像是否為同一人。If the pedestrian information shows that both the first original image Iori1 and the second original image Iori2 have a pedestrian image, and one of the pedestrian images is located within the overlapping area, the other pedestrian image is located outside the overlapping area. Then, the humanoid tracking program is executed by the humanoid detection and tracking unit 220 to determine whether the pedestrian image in the first original image Iori1 and the pedestrian image in the second original image Iori2 are the same person.

如圖4(d)所示之實施例,行人影像44位於第一重疊區域R1之外,而行人影像45則位於第二重疊區域R2之內。若行人影像44與行人影像45為同一人,則選擇將行人影像44的權重設定值設定為100%,並將行人影像45之權重設定值設定為0%,以避免在同一全景影像Ipan中的不同位置產生相同的行人影像。反之,若行人影像44與行人影像45並非為同一人,則可選擇分別提高行人影像44與行人影像45之權重設定值,使得行人影像44與45皆完整的呈現在拼接後的全景影像Ipan中。As shown in the embodiment of FIG. 4(d), the pedestrian image 44 is located outside the first overlap region R1, and the pedestrian image 45 is located within the second overlap region R2. If the pedestrian image 44 and the pedestrian image 45 are the same person, the weight setting value of the pedestrian image 44 is selected to be 100%, and the weight setting value of the pedestrian image 45 is set to 0% to avoid the same panoramic image Ipan. The same pedestrian image is produced in different locations. On the other hand, if the pedestrian image 44 and the pedestrian image 45 are not the same person, the weight setting values of the pedestrian image 44 and the pedestrian image 45 can be separately increased, so that the pedestrian images 44 and 45 are completely presented in the stitched panoramic image Ipan. .

上述依據不同行人資訊調整權重設定值的實施範例對改善全景影像中行人錯位問題描繪出一個可能的型態,僅是一種選擇實施範例,不應限制本案的保護範圍,權重設定值不以100%與0%為限。換言之,凡是對行人影像進行獨立處理以產生行人資訊,並利用行人資訊作為調整權重值的依據,據以合成全景影像之技術即為本發明之精神所在。綜上所述,本發明可解決因為拍攝時間差導致拍攝場景中行人的位置因移動而出現錯位、殘影等問題,進而提供高品質的全景影像。The above-mentioned implementation example of adjusting the weight setting values according to different pedestrian information has drawn a possible pattern for improving the pedestrian misplacement problem in the panoramic image. It is only an example of selection and implementation, and should not limit the scope of protection of the case. The weight setting value is not 100%. Limited to 0%. In other words, all the pedestrian images are processed independently to generate pedestrian information, and pedestrian information is used as the basis for adjusting the weight value. The technique of synthesizing the panoramic image is the spirit of the invention. In summary, the present invention can solve the problem that the position of the pedestrian in the shooting scene is dislocated, residual, and the like due to the difference in shooting time, thereby providing a high-quality panoramic image.

雖然本發明已以實施例揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明之精神和範圍內,當可作些許之更動與潤飾,故本發明之保護範圍當視後附之申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

200...影像擷取裝置200. . . Image capture device

210...取圖單元210. . . Drawing unit

220...人形偵測與追蹤單元220. . . Humanoid detection and tracking unit

230...影像拼接單元230. . . Image splicing unit

Img1、Img2...待拼接影像Img1, Img2. . . Image to be stitched

Iori1、Iori2...原始影像Iori1, Iori2. . . Original image

Ipan...全景影像Ipan. . . Panoramic image

10~13、40~45...行人10~13, 40~45. . . pedestrian

R1、R2...重疊區域R1, R2. . . Overlapping area

S310~S340...產生全景影像的方法之各步驟S310~S340. . . Steps in the method of generating a panoramic image

圖1(a)至圖1(d)繪示為拍攝場景中具有正在移動的行人所擷取的兩張待拼接影像的示意圖。1(a) to 1(d) are schematic diagrams showing two images to be stitched captured by a pedestrian who is moving in a shooting scene.

圖2是依照本發明一實施例所繪示之產生全景影像的影像擷取裝置的方塊圖。FIG. 2 is a block diagram of an image capturing device for generating a panoramic image according to an embodiment of the invention.

圖3是依照本發明一實施例所繪示之一種產生全景影像的方法流程圖。FIG. 3 is a flow chart of a method for generating a panoramic image according to an embodiment of the invention.

圖4(a)至圖4(d)繪示為具有不同行人資訊的第一原始影像與第二原始影像的簡易示意圖。4(a) to 4(d) are diagrams showing simple first and second original images with different pedestrian information.

S310~S340...產生全景影像的方法之各步驟S310~S340. . . Steps in the method of generating a panoramic image

Claims (12)

一種產生全景影像的方法,適用於一影像擷取裝置,該影像擷取裝置針對具有至少一行人的拍攝場景進行拍攝,該方法包括下列步驟:取得一第一原始影像與一第二原始影像,定義該第一原始影像中的一第一重疊區域與該第二原始影像中的一第二重疊區域,其中該第一重疊區域對應於該第二重疊區域;針對該第一原始影像與該第二原始影像執行一人形偵測程序與一人形追蹤程序,以獲得關於該至少一行人的一行人資訊;依據該行人資訊分別調整該第一重疊區域與該第二重疊區域中該至少一行人的一權重設定值;以及依據該權重設定值拼接該第一與該第二原始影像,以產生一全景影像。 A method for generating a panoramic image, which is suitable for an image capturing device, wherein the image capturing device performs shooting for a shooting scene having at least one line of people, the method comprising the steps of: obtaining a first original image and a second original image, Defining a first overlapping area of the first original image and a second overlapping area of the second original image, wherein the first overlapping area corresponds to the second overlapping area; for the first original image and the first The original image performs a humanoid detection program and a humanoid tracking program to obtain information about the at least one pedestrian; and adjusts the first overlapping area and the at least one of the second overlapping areas according to the pedestrian information. a weight setting value; and splicing the first and second original images according to the weight setting value to generate a panoramic image. 如申請專利範圍第1項所述之產生全景影像的方法,其中對該第一原始影像與該第二原始影像執行該人形偵測程序的步驟包括:分別判斷該第一重疊區域與該第二重疊區域中是否具有一行人影像,若是,分別記錄該行人影像之位置與形狀以作為該行人資訊。 The method for generating a panoramic image according to the first aspect of the invention, wherein the step of performing the humanoid detection process on the first original image and the second original image comprises: respectively determining the first overlapping area and the second Whether there is a line of human images in the overlapping area, and if so, the position and shape of the pedestrian image are separately recorded as the pedestrian information. 如申請專利範圍第2項所述之產生全景影像的方法,其中若該第一重疊區域與該第二重疊區域中皆具有該行人影像,針對該第一原始影像與該第二原始影像執行該人形追蹤程序的步驟包括: 利用特徵點追蹤法以判斷該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像是否為同一人,並將判斷結果作為該行人資訊。 The method for generating a panoramic image according to claim 2, wherein if the first overlapping area and the second overlapping area both have the pedestrian image, performing the first original image and the second original image The steps of the humanoid tracking program include: The feature point tracking method is used to determine whether the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, and the judgment result is used as the pedestrian information. 如申請專利範圍第3項所述之產生全景影像的方法,其中若該行人資訊顯示該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像並非為同一人,調整該第一重疊區域與該第二重疊區域中該至少一行人的該權重設定值的步驟包括:分別提高該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像之權重設定值。 The method for generating a panoramic image according to claim 3, wherein if the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person, adjust the The step of setting the weight setting value of the at least one row in the first overlapping area and the second overlapping area comprises: respectively increasing the weight setting of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area value. 如申請專利範圍第3項所述之產生全景影像的方法,其中若該行人資訊顯示該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像為同一人,調整該第一重疊區域與該第二重疊區域的該權重設定值的步驟包括:將該第一重疊區域與該第二重疊區域其中之一的該行人影像之權重設定值設定為100%,將該另一行人影像之權重設定值設定為0%。 The method for generating a panoramic image according to claim 3, wherein if the pedestrian information indicates that the pedestrian image in the first overlapping area is the same person as the pedestrian image in the second overlapping area, adjust the first The step of setting the weight of the overlap region and the second overlap region includes: setting a weight setting value of the pedestrian image of one of the first overlap region and the second overlap region to 100%, and the other row The weight setting value of the human image is set to 0%. 如申請專利範圍第2項所述之產生全景影像的方法,其中若判斷該第一重疊區域與該第二重疊區域僅其中之一具有該行人影像,更包括下列步驟:判斷該行人影像是否為完整的影像,若該行人影像並非為完整的影像,將該行人影像之權重設定值設定為0%。 The method for generating a panoramic image according to claim 2, wherein if it is determined that only one of the first overlapping area and the second overlapping area has the pedestrian image, the method further includes the following steps: determining whether the pedestrian image is For a complete image, if the pedestrian image is not a complete image, set the weight setting of the pedestrian image to 0%. 一種產生全景影像的影像擷取裝置,適於針對具有 至少一行人的拍攝場景進行拍攝,該影像擷取裝置包括:一取圖單元,取得一第一原始影像與一第二原始影像,並定義該第一原始影像中的一第一重疊區域與該第二原始影像中的一第二重疊區域,其中該第一重疊區域對應於該第二重疊區域;一人形偵測與追蹤單元,耦接該取圖單元,對該第一原始影像與該第二原始影像進行人形偵測與人形追蹤,以產生關於該至少一行人的一行人資訊;以及一影像拼接單元,耦接該取圖單元與該人形偵測與追蹤單元,該影像拼接單元依據該行人資訊分別調整該第一重疊區域與該第二重疊區域中該至少一行人的一權重設定值,並依據該權重設定值拼接該第一與該第二原始影像,以產生一全景影像。 An image capturing device for generating a panoramic image, suitable for having Shooting a scene of at least one person, the image capturing device includes: a drawing unit, obtaining a first original image and a second original image, and defining a first overlapping area in the first original image and the a second overlapping area of the second original image, wherein the first overlapping area corresponds to the second overlapping area; a humanoid detecting and tracking unit coupled to the drawing unit, the first original image and the first The original image is subjected to humanoid detection and humanoid tracking to generate a pedestrian information about the at least one pedestrian; and an image mosaic unit coupled to the image capturing unit and the humanoid detection and tracking unit, wherein the image mosaic unit is configured according to the The pedestrian information adjusts a weight setting value of the at least one row in the first overlapping area and the second overlapping area, and splices the first and second original images according to the weight setting value to generate a panoramic image. 如申請專利範圍第7項所述之影像擷取裝置,其中:該人形偵測與追蹤單元分別偵測該第一重疊區域與該第二重疊區域中是否具有一行人影像,若是,該人形偵測與追蹤單元記錄該行人影像之位置與形狀以作為該行人資訊。 The image capturing device of claim 7, wherein the humanoid detecting and tracking unit respectively detects whether the first overlapping area and the second overlapping area have a pedestrian image, and if so, the humanoid detection The measurement and tracking unit records the position and shape of the pedestrian image as the pedestrian information. 如申請專利範圍第8項所述之影像擷取裝置,其中:該人形偵測與追蹤單元判斷該第一重疊區域與該第二重疊區域中皆具有該行人影像,該人形偵測與追蹤單元更包括利用特徵點追蹤法判斷該第一重疊區域中的該行人 影像與該第二重疊區域中的該行人影像是否為同一人,並記錄判斷結果以作為該行人資訊。 The image capturing device of claim 8, wherein the humanoid detecting and tracking unit determines that the first overlapping area and the second overlapping area both have the pedestrian image, the humanoid detecting and tracking unit Further including determining, by the feature point tracking method, the pedestrian in the first overlapping area Whether the image and the pedestrian image in the second overlapping area are the same person, and recording the judgment result as the pedestrian information. 如申請專利範圍第9項所述之影像擷取裝置,其中若該行人資訊顯示該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像並非為同一人:該影像拼接單元分別提高該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像之權重設定值,據以拼接該第一與該第二原始影像。 The image capturing device of claim 9, wherein the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person: the image stitching The unit respectively increases the weight setting value of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area, thereby splicing the first and second original images. 如申請專利範圍第9項所述之影像擷取裝置,其中若該行人資訊顯示該第一重疊區域中的該行人影像與該第二重疊區域中的該行人影像為同一人:該影像拼接單元將該第一重疊區域與該第二重疊區域其中之一的該行人影像之權重設定值設定為100%,並將該另一行人影像之權重設定值設定為0%,據以拼接該第一與該第二原始影像。 The image capturing device of claim 9, wherein the pedestrian information indicates that the pedestrian image in the first overlapping area is the same person as the pedestrian image in the second overlapping area: the image splicing unit Setting a weight setting value of the pedestrian image of one of the first overlapping area and the second overlapping area to 100%, and setting a weight setting value of the other pedestrian image to 0%, according to the first With the second original image. 如申請專利範圍第8項所述之產生全景影像的影像擷取裝置,其中若該行人資訊顯示該第一重疊區域與該第二重疊區域僅其中之一具有該行人影像:該人形偵測與追蹤單元更包括偵測該行人影像是否為完整的影像,並記錄偵測結果以作為該行人資訊,該影像拼接單元接收到偵測結果為否的該行人資訊後,據以將該行人影像之權重設定值設定為0%。 The image capturing device for generating a panoramic image according to claim 8, wherein the pedestrian information indicates that only one of the first overlapping area and the second overlapping area has the pedestrian image: the humanoid detection and The tracking unit further includes detecting whether the pedestrian image is a complete image, and recording the detection result as the pedestrian information. After the image splicing unit receives the pedestrian information with the detection result of the detection result, the pedestrian image is accordingly The weight setting value is set to 0%.
TW101111116A 2012-03-29 2012-03-29 Method for generating panoramic image and image capturing device thereof TWI516120B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW101111116A TWI516120B (en) 2012-03-29 2012-03-29 Method for generating panoramic image and image capturing device thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW101111116A TWI516120B (en) 2012-03-29 2012-03-29 Method for generating panoramic image and image capturing device thereof

Publications (2)

Publication Number Publication Date
TW201340706A TW201340706A (en) 2013-10-01
TWI516120B true TWI516120B (en) 2016-01-01

Family

ID=49771127

Family Applications (1)

Application Number Title Priority Date Filing Date
TW101111116A TWI516120B (en) 2012-03-29 2012-03-29 Method for generating panoramic image and image capturing device thereof

Country Status (1)

Country Link
TW (1) TWI516120B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107026973B (en) 2016-02-02 2020-03-13 株式会社摩如富 Image processing device, image processing method and photographic auxiliary equipment
TWI620147B (en) * 2016-11-17 2018-04-01 National Chunghsing University Image synthesis method for synthesizing people

Also Published As

Publication number Publication date
TW201340706A (en) 2013-10-01

Similar Documents

Publication Publication Date Title
US9325899B1 (en) Image capturing device and digital zooming method thereof
JP5592006B2 (en) 3D image processing
US9591237B2 (en) Automated generation of panning shots
US8509518B2 (en) Real-time image collage method and apparatus
TWI472224B (en) Image processing device and image processing method thereof
WO2016184131A1 (en) Image photographing method and apparatus based on dual cameras and computer storage medium
JP2013257686A5 (en)
TW200823797A (en) Face detector, image pickup device, and face detecting method
TW201024908A (en) Panoramic image auto photographing method of digital photography device
JP2009200560A (en) Imaging apparatus, imaging method, and program
CN106875341B (en) Distorted image correction method and positioning method thereof
TW201801516A (en) Image capturing apparatus and photo composition method thereof
WO2018196854A1 (en) Photographing method, photographing apparatus and mobile terminal
JP6656035B2 (en) Image processing apparatus, imaging apparatus, and control method for image processing apparatus
US8731327B2 (en) Image processing system and image processing method
TWI536829B (en) Electronic apparatus using image processing
TWI615808B (en) Image processing method for immediately producing panoramic images
JP5504990B2 (en) Imaging apparatus, image processing apparatus, and program
TWI516120B (en) Method for generating panoramic image and image capturing device thereof
JP2010114752A (en) Device and method of imaging and program
TWI445396B (en) Electronic apparatus, image capturing apparatus and method thereof
TWI502271B (en) Controlling method and electronic apparatus
CN103366351B (en) Produce method and the image acquiring device thereof of panoramic picture
JP2007316957A (en) Image trimming device, image trimming method and program
JP2007096480A (en) Object tracking apparatus and object tracking method

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees