TW201340706A - Method for generating panoramic image and image capturing device thereof - Google Patents

Abstract

A method for generating a panoramic image and an image capturing device thereof are provided. The method includes the following steps. First, a first original image and a second original image are obtained. A first overlapping area of the first original image and a second overlapping area of the second original image are defined. Next, a human detection procedure and a human tracking procedure are performed on the first and the second original image to obtain a pedestrian information in a shooting scene. Then, two weight setting values of the at least one pedestrian of the first and the second overlapping area are adjusted respectively according to the pedestrian information. Finally, the first and the second original images are stitched to generate the panoramic image according to the weight setting values.

Description

Method for generating panoramic image and image capturing device thereof

The present invention relates to an image processing method and apparatus, and more particularly to a method for generating a panoramic image and an image capturing device thereof.

Conventionally, a technique for generating a panoramic image is to use a camera to continuously pan to capture multiple images of the same scene but different viewing angles, and to stitch the images by estimating the movement between adjacent images. Since the camera has a time difference in the process of continuous panning, if there is a moving pedestrian in the shooting scene, the stitched image will cause problems such as afterimage and overlay. The following will illustrate the possible situation of the stitched panoramic image with several examples.

1(a) to 1(d) are schematic diagrams showing two images to be stitched captured by a pedestrian who is moving in a shooting scene. First, please refer to FIG. 1(a), assuming that there is a pedestrian 10 in the image Img1 to be spliced, but there is no pedestrian in the image Img2 to be spliced. It is conventional to splicing and mixing the images to be stitched Img1 and Img2, which may cause the pedestrian 10 to merge with the background in the image to cause the image of the pedestrian 10 to be blurred or even disappear. Next, referring to FIG. 1(b), it is assumed that there is a pedestrian 11 in the image Img1 to be spliced, and the same pedestrian 11 is also present in the image Img2 to be spliced. However, the pedestrian 11 performs the image Img1 and Img2 to be spliced due to the position of the moving position. After splicing and mixing, it is possible to cause the spliced pedestrian 11 to generate multiple hands or feet or become a panoramic image with two pedestrians.

Referring to FIG. 1(c), it is assumed that there is a pedestrian 12 in the image Img1 to be stitched, but the pedestrian 12 only captures part of the image, and there is no image of the pedestrian 12 in the image Img2 to be stitched. After splicing and mixing the images to be spliced Img1, Img2, a panoramic image of half a pedestrian 12 may appear. Referring to FIG. 1(d), it is assumed that there is a pedestrian 13 in the image Img1 to be spliced, and the same pedestrian 13 is also included in the image Img2 to be spliced. However, the pedestrian 13 is in the image to be spliced Img1 due to the faster moving speed. The positions in Img2 are far apart. After the spliced and mixed images Img1 and Img2 are spliced and mixed, it is possible to cause the panoramic image to simultaneously appear the same pedestrian 13 at different positions.

For the problem of having a moving pedestrian in the above shooting scene, it is necessary to provide a panoramic image generating method capable of quickly and correctly performing splicing processing to improve the quality of the panoramic image.

In view of the above, the present invention provides a method for generating a panoramic image and an image capturing device thereof, which can reduce the problem that a pedestrian in the image generates a residual image during the connection process due to the time difference of the capturing, so as to improve the quality of the panoramic image.

The invention provides a method for generating a panoramic image, which is suitable for an image capturing device, and the image capturing device performs shooting for a shooting scene with one or more pedestrians. This method includes the following steps. First obtaining the first original image and the second original image, and defining a first overlapping area in the first original image and a second overlapping area in the second original image. Next, a humanoid detection program and a humanoid tracking program are executed for the first original image and the second original image to obtain pedestrian information about the pedestrian. Then, the weight setting values of one or more pedestrians in the first overlapping area and the second overlapping area are respectively adjusted according to the pedestrian information. The first and second original images are then stitched according to the weight setting value to generate a panoramic image.

In an embodiment of the invention, the step of executing the humanoid detection program on the first original image and the second original image includes determining whether the first overlapping area and the second overlapping area respectively have a pedestrian image. If so, the position and shape of the pedestrian image are recorded separately as pedestrian information.

In an embodiment of the present invention, if it is determined that both the first overlapping area and the second overlapping area have a pedestrian image, the step of performing a humanoid tracking procedure for the first original image and the second original image comprises using feature point tracking It is determined whether the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, and the judgment result is used as pedestrian information.

In an embodiment of the present invention, if the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person, adjusting one of the first overlapping area and the second overlapping area The step of setting the weights of the plurality of pedestrians includes increasing the weight setting values of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area, respectively.

In an embodiment of the present invention, if the pedestrian information shows that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, adjusting the weight setting values of the first overlapping area and the second overlapping area The step of setting the weight setting value of the pedestrian image of one of the first overlapping area and the second overlapping area to 100%, and setting the weight setting value of another pedestrian image to 0%.

In an embodiment of the present invention, if it is determined that only one of the first overlapping area and the second overlapping area has a pedestrian image, the method further includes determining whether the pedestrian image is a complete image, and if the pedestrian image is not a complete image, Then set the weight setting value of the pedestrian image to 0%.

The invention further provides an image capturing device for generating a panoramic image, which is suitable for shooting for a shooting scene with one or more pedestrians. The image capturing device comprises a drawing unit, a humanoid detection and tracking unit and an image splicing unit. The drawing unit is configured to obtain the first original image and the second original image, and define a first overlapping area in the first original image and a second overlapping area in the second original image. The humanoid detection and tracking unit is coupled to the image capturing unit for performing humanoid detection and humanoid tracking on the first original image and the second original image to generate pedestrian information about the pedestrian. The image splicing unit is respectively coupled to the drawing unit and the human figure detecting and tracking unit. The image splicing unit adjusts the weight setting values of the one or more pedestrians in the first overlapping area and the second overlapping area according to the pedestrian information, and splicing the first original image and the second original image according to the weight setting value to generate the panoramic image.

Based on the above, the method for generating a panoramic image and the image capturing device thereof are provided by the present invention, and the pedestrian information is obtained by the human body detection and tracking, and the weight value is adjusted according to the pedestrian information, and then according to the adjusted weight value. Image stitching can reduce the problem of pedestrians in the image causing residual image during the connection process due to the time difference of the capture.

The above described features and advantages of the present invention will be more apparent from the following description.

In the process of shooting a panorama, the position of the pedestrian in the shooting scene often causes misalignment, residual image and the like due to the movement due to the difference in shooting time. The present invention provides a pedestrian information by means of humanoid detection and tracking, and according to this Pedestrian information is used to adjust the weight value of the image joint area, and the image stitching according to the adjusted weight value can reduce unnecessary defects in the panorama connection process, thereby providing high-quality panoramic images. In order to clarify the content of the present invention, the following examples are given as examples in which the present invention can be implemented.

FIG. 2 is a block diagram of an image capturing device for generating a panoramic image according to an embodiment of the invention. Referring to FIG. 2, the image capturing device 200 of the present embodiment is, for example, a digital camera, a monocular camera, a digital camera, or another smart phone, a tablet computer or the like having a function of generating a panoramic image, and is not limited to the above. The image capturing device 200 includes a drawing unit 210, a humanoid detecting and tracking unit 220, and an image splicing unit 230, which can be functional modules implemented by hardware and/or software, and the functions thereof are as follows: the image capturing device 200 It is suitable for shooting for shooting scenes with one or more pedestrians, and can store multiple images obtained by continuous shooting in a memory (not shown). When the image capturing device 200 is to perform the panoramic image synthesizing function, the drawing unit 210 is configured to obtain the first original image Iori1 and the second original image Iori2 from the memory for providing the humanoid detection and tracking unit 220 for humanoid detection. (human detection) and human tracking, which in turn produces information about pedestrians in the scene. The image splicing unit 230 receives the pedestrian information and adjusts the weight setting value, and uses the weight setting value to determine whether the spliced panoramic image Ipan retains or removes the pedestrian appearing in the original image. According to this, it is possible to avoid the problem that the pedestrian in the image generates a residual image or an unreasonable image as described in the prior art.

FIG. 3 is a flow chart of a method for generating a panoramic image according to an embodiment of the invention. The method of the present embodiment is applicable to the image capturing device 200 of FIG. 2, and the detailed steps of the method of the embodiment are described below with the components in the image capturing device 200. First, in step S310, the image is captured by moving the image. The device 200 continuously captures a plurality of images of adjacent regions in a scene, wherein the scene has one or more pedestrians in still or moving. The image capturing unit 210 acquires the first original image Iori1 and the second original image Iori2 to be combined with the panoramic image from the images. And the drawing unit 210 defines a first overlapping area R1 in the first original image Iori1 and a second overlapping area R2 in the second original image Iori2. The first overlapping area R1 and the second overlapping area R2 represent image joining areas where the first original image Iori1 and the second original image Iori2 are to be stitched, which are generally the same area of the background.

Next, in step S320, the humanoid detection and tracking unit 220 performs humanoid detection and humanoid tracking for the first original image Iori1 and the second original image Iori2 to obtain pedestrian information about the shooting scene. In detail, the humanoid detection and tracking unit 220 first performs humanoid detection to determine whether there is a pedestrian image in the first original image Iori1 and the second original image Iori2, respectively. In particular, the humanoid detection and tracking unit 220 detects whether there is a pedestrian image in the first overlapping area R1 and the second overlapping area R2. If so, the position and shape of the pedestrian image in the first original image Iori1 and the position and shape of the second original image Iori2 are recorded as pedestrian information. For example, the first and second original images Iori1 and Iori2 can be regarded as a two-dimensional right-angle coordinate system respectively. By recording the coordinate position of the pixel of the pedestrian image, the judgment basis of the subsequent connection processing can be provided.

If the first original image Iori1 and the second original image Iori2 are both detected as having a pedestrian image, the humanoid detection and tracking unit 220 further includes using the feature point tracking method to determine the pedestrian image in the first overlapping region R1. Whether the pedestrian image appearing in the two overlapping regions R2 is the same person, and the judgment result is used as pedestrian information. It should be noted that if the image capturing device 200 provides the face recognition function, the face recognition may be used in this step to determine whether the pedestrian image appearing in the first original image Iori1 and the second original image Iori2 is same person.

Next, in step S330, the image splicing unit 230 receives the pedestrian information recorded by the humanoid detection and tracking unit 220, and adjusts the first overlapping region R1 and the second overlapping region according to the pedestrian information including the pedestrian position, shape, and the like, respectively. The weight setting value of the pedestrian image in R2. The adjustment of the weight setting will be explained in more detail later.

Finally, in step S340, the image splicing unit 230 splices the first original image Iori1 and the second original image Iori2 according to the adjusted weight setting value to generate the panoramic image Ipan.

Next, FIG. 4( a ) to FIG. 4( d ) are schematic diagrams showing a first original image and a second original image with different pedestrian information, which are used to help explain how to adjust the weight setting value according to different pedestrian information. . Please refer to FIG. 4(a) to FIG. 4(d) below.

If the pedestrian information shows that only one of the first original image Iori1 and the second original image Iori2 has a pedestrian image, and the pedestrian image is located within the overlapping area. The humanoid detection and tracking unit 220 further includes detecting whether the pedestrian image is a complete image, and further recording the detection result as part of the pedestrian information. The embodiment shown in FIG. 4(a) and FIG. 4(c) is that only one of the first original image Iori1 and the second original image Iori2 has a pedestrian image.

As shown in FIG. 4(c), when the pedestrian image 43 displays only a part of the image in the overlap region R1, the image splicing unit 230 sets the weight setting value of the pedestrian image 43 to 0%. Since the information of the pedestrian image 43 is not complete, after setting the weight setting value to 0%, the pedestrian image 43 can be prevented from appearing in the stitched panoramic image Ipan. Conversely, as shown in FIG. 4( a ), the pedestrian image 40 is a complete image, and the image splicing unit 230 can enhance the weight setting value of the pedestrian image 40 to make it more completely and clearly displayed in the panoramic image Ipan. in. The enhancement ratio of the weight setting value can be set by the user according to the actual situation, and is not limited herein.

If the pedestrian information shows that both the first original image Iori1 and the second original image Iori2 have a pedestrian image, and the pedestrian images are located within the overlapping regions R1 and R2. Then, the humanoid tracking program is executed by the humanoid detection and tracking unit 220 to determine whether the pedestrian image in the first overlapping region R1 and the pedestrian image in the second overlapping region R2 are the same person.

As shown in FIG. 4(b), the first overlapping area R1 has a pedestrian image 41, and the second overlapping area R2 has a pedestrian image 42. If the pedestrian image 41 and the pedestrian image 42 are the same person, the weight setting value of one of the pedestrian images 41, 42 may be set to 100%, and the weight setting value of another pedestrian image may be set to 0%. After this adjustment, it is possible to prevent the stitched panoramic image Ipan from generating multi-footed or overlapping images. Conversely, if the pedestrian image 41 and the pedestrian image 42 are not the same person, the weight setting values of the pedestrian image 41 and the pedestrian image 42 can be separately increased, so that the pedestrian images 41 and 42 are completely presented in the stitched panoramic image Ipan. in.

If the pedestrian information shows that both the first original image Iori1 and the second original image Iori2 have a pedestrian image, and one of the pedestrian images is located within the overlapping area, the other pedestrian image is located outside the overlapping area. Then, the humanoid tracking program is executed by the humanoid detection and tracking unit 220 to determine whether the pedestrian image in the first original image Iori1 and the pedestrian image in the second original image Iori2 are the same person.

As shown in the embodiment of FIG. 4(d), the pedestrian image 44 is located outside the first overlap region R1, and the pedestrian image 45 is located within the second overlap region R2. If the pedestrian image 44 and the pedestrian image 45 are the same person, the weight setting value of the pedestrian image 44 is selected to be 100%, and the weight setting value of the pedestrian image 45 is set to 0% to avoid the same panoramic image Ipan. The same pedestrian image is produced in different locations. On the other hand, if the pedestrian image 44 and the pedestrian image 45 are not the same person, the weight setting values of the pedestrian image 44 and the pedestrian image 45 can be separately increased, so that the pedestrian images 44 and 45 are completely presented in the stitched panoramic image Ipan. .

The above-mentioned implementation example of adjusting the weight setting values according to different pedestrian information has drawn a possible pattern for improving the pedestrian misplacement problem in the panoramic image. It is only an example of selection and implementation, and should not limit the scope of protection of the case. The weight setting value is not 100%. Limited to 0%. In other words, all the pedestrian images are processed independently to generate pedestrian information, and pedestrian information is used as the basis for adjusting the weight value. The technique of synthesizing the panoramic image is the spirit of the invention. In summary, the present invention can solve the problem that the position of the pedestrian in the shooting scene is dislocated, residual, and the like due to the difference in shooting time, thereby providing a high-quality panoramic image.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

200. . . Image capture device

210. . . Drawing unit

220. . . Humanoid detection and tracking unit

230. . . Image splicing unit

Img1, Img2. . . Image to be stitched

Iori1, Iori2. . . Original image

Ipan. . . Panoramic image

10~13, 40~45. . . pedestrian

R1, R2. . . Overlapping area

S310~S340. . . Steps in the method of generating a panoramic image

1(a) to 1(d) are schematic diagrams showing two images to be stitched captured by a pedestrian who is moving in a shooting scene.

FIG. 2 is a block diagram of an image capturing device for generating a panoramic image according to an embodiment of the invention.

FIG. 3 is a flow chart of a method for generating a panoramic image according to an embodiment of the invention.

4(a) to 4(d) are diagrams showing simple first and second original images with different pedestrian information.

S310~S340. . . Steps in the method of generating a panoramic image

Claims (12)

A method for generating a panoramic image, which is suitable for an image capturing device, wherein the image capturing device performs shooting for a shooting scene having at least one line of people, the method comprising the steps of: obtaining a first original image and a second original image, Defining a first overlapping area of the first original image and a second overlapping area of the second original image; performing a humanoid detection program and a humanoid tracking program for the first original image and the second original image Obtaining a pedestrian information about the at least one pedestrian; adjusting a weight setting value of the at least one row in the first overlapping area and the second overlapping area according to the pedestrian information; and splicing the first according to the weight setting value And the second original image to generate a panoramic image. The method for generating a panoramic image according to the first aspect of the invention, wherein the step of performing the humanoid detection process on the first original image and the second original image comprises: respectively determining the first overlapping area and the second Whether there is a line of human images in the overlapping area, and if so, the position and shape of the pedestrian image are separately recorded as the pedestrian information. The method for generating a panoramic image according to claim 2, wherein if the first overlapping area and the second overlapping area both have the pedestrian image, performing the first original image and the second original image The step of the humanoid tracking program includes: using the feature point tracking method to determine whether the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, and using the determination result as the pedestrian information. The method for generating a panoramic image according to claim 3, wherein if the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person, adjust the The step of setting the weight setting value of the at least one row in the first overlapping area and the second overlapping area comprises: respectively increasing the weight setting of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area value. The method for generating a panoramic image according to claim 3, wherein if the pedestrian information indicates that the pedestrian image in the first overlapping area is the same person as the pedestrian image in the second overlapping area, adjust the first The step of setting the weight of the overlap region and the second overlap region includes: setting a weight setting value of the pedestrian image of one of the first overlap region and the second overlap region to 100%, and the other row The weight setting value of the human image is set to 0%. The method for generating a panoramic image according to claim 2, wherein if it is determined that only one of the first overlapping area and the second overlapping area has the pedestrian image, the method further comprises: determining whether the pedestrian image is complete Image, if the pedestrian image is not a complete image, set the weight setting value of the pedestrian image to 0%. An image capturing device for generating a panoramic image, which is suitable for shooting a shooting scene having at least one line of people, the image capturing device comprising: a drawing unit, obtaining a first original image and a second original image, and defining a first overlapping area of the first original image and a second overlapping area of the second original image; a humanoid detecting and tracking unit coupled to the drawing unit, the first original image and the first The original image is subjected to humanoid detection and humanoid tracking to generate a pedestrian information about the at least one pedestrian; and an image mosaic unit coupled to the image capturing unit and the humanoid detection and tracking unit, wherein the image mosaic unit is configured according to the The pedestrian information adjusts a weight setting value of the at least one row in the first overlapping area and the second overlapping area, and splices the first and second original images according to the weight setting value to generate a panoramic image. The image capturing device of claim 7, wherein the humanoid detecting and tracking unit respectively detects whether the first overlapping area and the second overlapping area have a pedestrian image, and if so, the humanoid detection The measurement and tracking unit records the position and shape of the pedestrian image as the pedestrian information. The image capturing device of claim 8, wherein the humanoid detecting and tracking unit determines that the first overlapping area and the second overlapping area both have the pedestrian image, the humanoid detecting and tracking unit The method further includes determining, by the feature point tracking method, whether the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are the same person, and recording the determination result as the pedestrian information. The image capturing device of claim 9, wherein the pedestrian information indicates that the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area are not the same person: the image stitching The unit respectively increases the weight setting value of the pedestrian image in the first overlapping area and the pedestrian image in the second overlapping area, thereby splicing the first and second original images. The image capturing device of claim 9, wherein the pedestrian information indicates that the pedestrian image in the first overlapping area is the same person as the pedestrian image in the second overlapping area: the image splicing unit Setting a weight setting value of the pedestrian image of one of the first overlapping area and the second overlapping area to 100%, and setting a weight setting value of the other pedestrian image to 0%, according to the first With the second original image. The method for generating a panoramic image according to claim 8, wherein the pedestrian information indicates that only one of the first overlapping area and the second overlapping area has the pedestrian image: the humanoid detection and tracking unit The method includes: detecting whether the pedestrian image is a complete image, and recording the detection result as the pedestrian information. After the image splicing unit receives the pedestrian information with the detection result of the detection result, the weight of the pedestrian image is set accordingly. Set to 0%.