JP6653310B2 - Method and terminal for acquiring panoramic image - Google Patents

Description

  The present invention relates to the field of image processing technology, and more particularly, to a method and a terminal for acquiring a panoramic image.

  The back panorama shooting mode is a shooting mode usually used in existing smartphones. Back panoramic photography is to move a photographing terminal in a certain direction to photograph a plurality of photos and join the plurality of photos to an image having a larger viewing angle.

  When one person goes out for a trip and wants to take a picture with a scene, that person usually needs to seek help from another person, which is inconvenient. If the person wants to take a selfie photo, usually a photo containing only the person's face can be taken, but it is difficult to take a photo that includes the entire background of the scene.

  When multiple people together want to take a group photo, a photo that includes everyone cannot be taken even by utilizing the selfie mode, which is troublesome for the user.

  The inventor of the present invention aims to obtain a photograph having a large viewing angle by photographing a plurality of photographs even when the front panorama mode is not applied in the prior art and the rear panorama mode in the prior art is applied to the front photography mode. Noticed that the shooting terminal was moved in only one direction during shooting. In this way, when shooting starts from a certain person, a scene on only one side of the person can be shot, and when starting from a certain scene, the person cannot be emphasized. Reduce the effect.

  Embodiments of the present invention provide a method for acquiring a panoramic image that can enhance the image at the center position during panoramic shooting, thereby improving the overall effect of the panoramic image. Embodiments of the present invention further provide a corresponding terminal.

According to a first aspect, the present invention is a method for obtaining a panoramic image,
Capturing a first image after obtaining a panoramic shooting instruction triggered by a user, and obtaining shooting parameters of the first image;
Determining travel guide information according to a predetermined travel guide policy, and displaying the travel guide information on the terminal to instruct the user to move the terminal according to the travel guide information;
After detecting that the terminal has moved, capturing a predetermined number of images according to a capturing parameter of the first image, wherein the predetermined number of images are on a left side of a background corresponding to the first image. Shooting a background image on both the right side and the right side;
Using the first image as a center, performing a seamless connection to the first image and the predetermined number of images by using a predetermined connection method, and acquiring a panoramic image;
Is provided.

Referring to the first aspect, in a first possible realization scheme, the mobile terminal determines travel guide information according to the predetermined travel guide policy so that the user moves the terminal according to the travel guide information, The step of displaying the movement guide information above,
Determining a shooting position of the next image according to the predetermined movement guide policy, and displaying the shooting position of the next image on the terminal;
Correspondingly, after detecting that the terminal has moved, capturing a predetermined number of images according to the capturing parameters of the first image, wherein the predetermined number of images correspond to the first image. Taking a background image on both the left and right sides of the background
After the terminal moves to the shooting position of the next image, shooting the next image according to shooting parameters of the first image;
Until the predetermined number of images are captured, repeating the above-described determination, display and capturing steps,
Having.

With reference to the first possible realization of the first aspect, in a second possible realization, after displaying the shooting position of the next image on the terminal, the method further comprises:
When the terminal is moving, has a step of obtaining and displaying the position of the terminal in real time,
Correspondingly, after the terminal has moved to the shooting position of the next image, the step of shooting the next image according to the shooting parameters of the first image,
When the displayed position of the terminal is the same as the photographing position of the next image, a step of photographing the next image according to the photographing parameters of the first image is included.

With reference to the second possible realization of the first aspect, in a third possible realization, when the terminal is moving, obtaining and displaying the position of the terminal in real time comprises:
When the terminal is moving, moving the terminal by detecting the relative movement of images in adjacent frames to instruct the user to move the terminal to a position for acquiring the next image. Acquiring and displaying the position of the terminal in the process in real time.

Referring to the second or third possible realization method of the first aspect, in a fourth possible realization method, photographing the first image and acquiring photographing parameters of the first image. After, the method further comprises:
Performing a center position calibration at predetermined image frame intervals, the center position calibration comparing a current preview image with the first image, and comparing the current preview image with the first image. When the matching similarity between is higher than a predetermined threshold, and when the position of the terminal cannot be detected, shooting is restarted according to the position where the first image is shot. Determining that it has returned to the position of the terminal at which the image is taken.

With reference to any one of the first to fourth possible realizations of the first aspect, in a fifth possible realization, the first image is captured and the first image is captured. Prior to the step of obtaining the parameters, the method further comprises:
Locking the automatic white balance AWB control and the automatic exposure AE control so that the shooting parameters of the predetermined number of shot images continue to match the shooting parameters of the first image.

With reference to any one of the first to fifth possible implementation methods of the first aspect, in a sixth possible implementation method, the first image is used as a center, and a predetermined joining method is used. Performing a seamless join to the first image and the predetermined number of images to obtain a panoramic image,
Detecting the position of the human image in both the first image and the predetermined number of images;
Performing image joining from the position of the non-human image in each image when image joining is performed by utilizing the predetermined joining method;
Having.

According to a second aspect, the invention relates to a terminal,
An obtaining unit configured to obtain a panoramic shooting instruction triggered by a user;
A photographing unit configured to photograph a first image after the acquiring unit acquires a panoramic photographing instruction triggered by the user;
Here, the obtaining unit is further configured to obtain a shooting parameter of the first image when the shooting unit shoots the first image,
A determining unit configured to determine moving guide information according to a predetermined moving guide policy after the obtaining unit has obtained the first image;
A display unit configured to display the travel guide information determined by the determination unit on the terminal to instruct the user to move the terminal according to the travel guide information;
A detection unit configured to detect a movement state of the terminal;
Here, the photographing unit is further configured to photograph a predetermined number of images according to a photographing parameter of the first image after the detection unit detects that the terminal has moved, and the predetermined number of images is Background images on both the left and right sides of the background corresponding to the first image,
Using the first image as a center and using a predetermined joining method, seamless joining is performed on the first image and the predetermined number of images photographed by the photographing unit, and the panorama is performed. A joining unit configured to acquire an image;
Is provided.

Referring to the second aspect, in a first possible realization scheme:
The determining unit is configured to determine a shooting position of a next image according to the predetermined movement guide policy,
The display unit is configured to display a shooting position of the next image on the terminal,
After the terminal moves to the shooting position of the next image, the shooting unit is configured to shoot the next image according to shooting parameters of the first image,
The determination unit, the display unit, and the photographing unit repeat the above-described determination, display, and photographing steps until the predetermined number of images are photographed.

With reference to the first possible implementation of the second aspect, in a second possible implementation,
The obtaining unit is further configured to obtain a position of the terminal in real time when the terminal is moving,
The display unit is further configured to display the position of the terminal obtained by the obtaining unit in real time,
When the position of the terminal displayed by the display unit is the same as the shooting position of the next image, the shooting unit is configured to shoot the next image according to the shooting parameters of the first image. .

With reference to a second possible implementation of the second aspect, in a third possible implementation,
The obtaining unit detects a relative movement of an image of an adjacent frame to instruct the user to move the terminal to a position for obtaining the next image when the terminal is moving. By doing so, it is configured to obtain the position of the terminal in a moving process in real time,
The display unit is configured to display a position of the terminal in the movement process acquired by the acquisition unit.

With reference to the second or third possible implementation of the second aspect, in a fourth possible implementation,
The terminal further comprises:
A calibration unit configured to perform a center position calibration at predetermined image frame intervals after the imaging unit captures the first image, wherein the center position calibration is performed based on a current preview image and the second preview image. The first image is compared with the first image when the matching similarity between the current preview image and the first image is higher than a predetermined threshold, and when the position of the terminal cannot be detected. A calibration unit for determining that the position of the terminal has returned to the position of the terminal at which the first image is captured, in order to resume shooting according to the position where the image is captured.

With reference to the second aspect and any one of the first to fourth possible implementations of the second aspect, in a fifth possible implementation,
The terminal further comprises:
Before the photographing unit photographs the first image, automatic white balance AWB control and automatic exposure AE are performed so that the photographing parameters of the predetermined number of photographed images continue to match the photographing parameters of the first image. A locking unit configured to lock control.

With reference to the second aspect and any one of the first to fifth possible implementations of the second aspect, in a sixth possible implementation,
The joining unit is
A detection subunit configured to detect a position of a human image in both the first image and the predetermined number of images;
A joining subunit configured to perform joining from the position of the non-human image in each image detected by the detection subunit when image joining is performed by utilizing the predetermined joining method;
Having.

According to a third aspect, the present invention is a terminal having a receiver, a transmitter, a camera, a processor and a memory,
The camera is configured to capture an image,
The receiver is configured to receive a panorama shooting instruction input by a user,
The processor comprises the following steps:
Capturing a first image after obtaining a panoramic shooting instruction triggered by a user, and obtaining shooting parameters of the first image;
Determining travel guide information according to a predetermined travel guide policy, displaying the travel guide information on the terminal to instruct the user to move the terminal according to the travel guide information,
After detecting that the terminal has moved, capturing a predetermined number of images according to a capturing parameter of the first image, wherein the predetermined number of images are on a left side of a background corresponding to the first image. Shooting a background image on both the right side and the right side;
Using the first image as a center, performing a seamless connection to the first image and the predetermined number of images by using a predetermined connection method, and acquiring a panoramic image;
Providing a terminal configured to execute

  According to an embodiment of the present invention, after a panoramic shooting instruction triggered by a user is obtained, a first image is shot, shooting parameters of the first image are obtained, and the moving guide information is a predetermined moving guide policy. The travel guide information is displayed on the terminal to instruct the user to move the terminal according to the travel guide information. After it is detected that the terminal has moved, a predetermined number of images are captured according to the capturing parameters of the first image, wherein the predetermined number of images correspond to the left and right sides of the background corresponding to the first image. A background image on both sides, using the first image as a center and using a predetermined joining method, seamless joining is performed on the first image and a predetermined number of images, and a panoramic image is obtained. I do. Compared with the prior art where the image at the center position cannot be enhanced during panoramic shooting, the method provided in the embodiment of the present invention allows the image at the center position to be enhanced during panoramic shooting, thereby Improve the overall effect of

In order to more clearly explain the technical solutions in the embodiments of the present invention, the following briefly introduces the accompanying drawings necessary for describing the embodiments. Apparently, the accompanying drawings in the following description show merely some embodiments of the present invention, and those skilled in the art may still derive other drawings from these accompanying drawings without creative efforts.
FIG. 1 is a schematic diagram of a method for acquiring a panoramic image according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a method for acquiring a panoramic image according to another embodiment of the present invention. FIG. 3A is a schematic diagram of a method for acquiring a panoramic image according to another embodiment of the present invention. FIG. 3B is a schematic diagram of a method for acquiring a panoramic image according to another embodiment of the present invention. FIG. 3C is a schematic diagram of a method for acquiring a panoramic image according to another embodiment of the present invention. FIG. 4 is a schematic diagram of a terminal according to an embodiment of the present invention. FIG. 5 is a schematic diagram of a terminal according to another embodiment of the present invention. FIG. 6 is a schematic diagram of a terminal according to another embodiment of the present invention. FIG. 7 is a schematic diagram of a terminal according to another embodiment of the present invention. FIG. 8 is a schematic diagram of a terminal according to another embodiment of the present invention. FIG. 9 is a schematic diagram of a terminal according to another embodiment of the present invention.

  Embodiments of the present invention provide a method for acquiring a panoramic image that can enhance the image at the center position during panoramic shooting, thereby improving the overall effect of the panoramic image. Embodiments of the present invention further provide a corresponding terminal. A detailed description is given below.

  The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are merely a part rather than all of the embodiments of the present invention. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

  The solution for obtaining a panoramic image provided in the embodiment of the present invention is applicable to a rear shooting mode and a front shooting mode, and particularly applicable to a front shooting mode.

  Referring to FIG. 1, a method for obtaining a panoramic image provided in an embodiment of the present invention includes the following steps.

  101. After the panorama shooting instruction triggered by the user is obtained, the first image is shot and the shooting parameters of the first image are obtained.

  When panoramic shooting is required, the terminal shoots the first image after the panoramic shooting instruction triggered by the user is acquired, and sets parameters such as exposure control and white balance control of other subsequent images. In order to ensure that they are the same as those of the first image, the shooting parameters of the first image are obtained at the same time.

  102. The travel guide information is displayed on the terminal to determine travel guide information according to a predetermined travel guide policy and to instruct a user to move the terminal according to the travel guide information.

  The predetermined movement guide policy in this embodiment of the present invention is to move or rotate first to the left of the initial position of the terminal, and then move or rotate to the right of the initial position of the terminal. Is also good.

  The movement guide information may be icon guide information or text guide information. The position to which the terminal should move may be displayed on the terminal.

  103. After it is detected that the terminal has moved, a predetermined number of images are photographed according to the photographing parameters of the first image, wherein the predetermined number of images are both on the left and right sides of the background corresponding to the first image. 3 is a background image.

  During front panoramic shooting, when the terminal rotates to the left of the initial position, an image on the right side of the background of the first image may be captured, and when the terminal rotates to the right of the initial position, the background of the first image may be captured. May be taken.

  The predetermined number may be 4, and when the terminal rotates to the left of the initial position, the terminal captures two images, and when the terminal rotates to the right of the initial position, the terminal captures two images. Alternatively, the predetermined number may be two, that is, one image on the left side and the other image on the right side. The predetermined number is not limited, and may be set as needed.

  For example, you want to take a selfie photo, the user holds the mobile phone, focuses on the user, takes an image, and the image is labeled "C". The user rotates the mobile phone toward the left side of the position where the image C was taken, continuously takes two images, that is, the image B and the image A, and then takes the right side of the position where the image C was taken. , The mobile phone is rotated to capture two images, that is, an image D and an image E. In this manner, five images, that is, the image E to the image A are obtained.

  104. Using the first image as a center and using a predetermined joining method, seamless joining is performed on the first image and a predetermined number of images to obtain a panoramic image.

  The image joining technique is to join a plurality of images into an image having a large viewing angle. The example in step 103 is still used. Images A to E are joined according to a sequence of A, B, C, D and E or a sequence of E, D, C, B and A such that a panoramic image with image C in the center is obtained.

  The image joining solution in this embodiment of the present invention belongs to the prior art. For example, the joining may be performed by using a method such as a template matching method, an adjacent column ratio matching method, a fast grid-based matching method, or a local entropy difference-based image matching method.

  According to the present embodiment of the present invention, after the panorama shooting instruction triggered by the user is obtained, the first image is shot, the shooting parameters of the first image are obtained, and the moving guide information is set to the predetermined moving guide. The travel guide information is displayed on the terminal to be determined according to the policy and instruct the user to move the terminal according to the travel guide information. After it is detected that the terminal has moved, a predetermined number of images are captured according to the capturing parameters of the first image, wherein the predetermined number of images correspond to the left and right sides of the background corresponding to the first image. A background image on both sides, using the first image as a center and using a predetermined joining method, seamless joining is performed on the first image and a predetermined number of images, and a panoramic image is obtained. I do. Compared with the prior art where the image at the center position cannot be enhanced during panoramic shooting, the method provided in the embodiment of the present invention allows the image at the center position to be enhanced during panoramic shooting, thereby Improve the overall effect of

Optionally, based on the embodiment corresponding to FIG. 1 above, in another embodiment of the method for obtaining a panoramic image provided in the embodiment of the present invention, the moving guide information is determined according to a predetermined moving guide policy. Displaying the travel guide information on the terminal to instruct the user to move the terminal according to the travel guide information,
Determining the shooting position of the next image according to a predetermined movement guide policy, and displaying the shooting position of the next image on the terminal;
Correspondingly, photographing a predetermined number of images according to the photographing parameters of the first image after detecting that the terminal has moved, wherein the predetermined number of images is a background image corresponding to the first image. Background images on both the left and right sides,
After the terminal moves to the shooting position of the next image, shooting the next image according to the shooting parameters of the first image;
Until a predetermined number of images are taken, repeating the above determination, display and photographing steps,
Having.

  In this embodiment of the present invention, if the predetermined movement guide policy instructs to move or rotate first to the left of the initial position, the terminal determines that the shooting position of the next image is on the left of the initial position. Then, the photographing position of the next image is displayed on the display interface of the terminal.

  The user may move or rotate the terminal according to the prompt on the display interface of the terminal, and when the terminal rotates to the shooting position of the next image, the terminal automatically shoots the next image. In this way, each time an image is taken, the location of the next image is displayed on the terminal interface and corresponds to the first image as long as the user moves or rotates the terminal according to the prompts on the display interface of the terminal. Images can be captured until a predetermined number of background images are captured on both the left and right sides of the background.

  For example, referring to FIG. 2, the user takes a selfie photograph in the panoramic shooting mode, the user holds the mobile phone, focuses on the user, and shoots the first image. The terminal instructs the mobile phone to rotate to the left according to a predetermined movement guide policy in order to complete the capturing of the second image. In addition, a small ball and a small circle are displayed on the display interface of the mobile phone, where the arrow points from the small circle to the small ball, and the small ball is used to indicate the shooting position of the next image, and the small circle Is used to indicate the initial position of the mobile phone where the first image was taken. The direction of the arrow indicates that the mobile phone needs to rotate to the left. In this way, the user can rotate the mobile phone to the left according to the prompt in the interface. When the small circle rotates to the position of the small ball, the small circle overlaps with the small ball and the mobile phone automatically captures the second image. After the second image is shot, the mobile phone further displays the shooting position of the third image. The user only needs to rotate the mobile phone according to the prompts on the display interface, and then the required image capture can be completed.

  Referring to FIG. 3A, the circle shown in FIG. 3A indicates the body of the user who wants to take a selfie photograph using the terminal, the vertical line indicates the user's arm, and the horizontal line indicates the terminal. FIG. 3A is a schematic diagram of a relationship between the terminal and the user when the user captures the first image. After the first image is captured, the terminal needs to rotate left to capture the second image, at which time the user may rotate the terminal left but need to move his arm There is no. As illustrated in FIG. 3B, the second image may be captured after the terminal rotates to the capturing position of the second image. After the second image is taken, the terminal needs to rotate rightward and take a third image, at which time the user may rotate the terminal rightward. When the terminal is rotated to the shooting position of the third image shown in FIG. 3C, the terminal may shoot the third image.

Optionally, based on the optional embodiment corresponding to FIG. 1 above, in another embodiment of the method of obtaining a panoramic image provided in the embodiment of the present invention, the following image capture on the terminal After indicating the location, the method further comprises:
Comprising obtaining and displaying the location of the terminal in real time when the terminal is moving,
Correspondingly, photographing the next image according to the photographing parameters of the first image after the terminal moves to the photographing position of the next image,
When the displayed position of the terminal is the same as the photographing position of the next image, photographing the next image according to the photographing parameters of the first image may be included.

  In this embodiment of the present invention, referring to FIG. 2, when the user rotates the mobile phone, the mobile phone obtains and displays the position of the mobile phone in the rotating process in real time, that is, a small circle corresponds to a small ball. To help the user determine how much the mobile phone should be rotated to allow them to overlap, changes in the position of the small circles are made on the mobile phone in real time during the mobile phone's movement. Is displayed. When the positions of the small circle and the small ball overlap each other, the mobile phone automatically acquires the next image.

Optionally, based on the optional embodiment corresponding to FIG. 1 above, in another embodiment of the method for obtaining a panoramic image provided in the embodiment of the present invention, when the terminal is moving, Obtaining and displaying the location of the terminal in real time
When the terminal is moving, detecting the relative movement of the images of adjacent frames to instruct the user to move the terminal to a position for acquiring the next image, Including obtaining and displaying the position in real time.

  In this embodiment of the present invention, after the first image is captured, the mobile phone needs to rotate to the position of the next image to capture the next image. In fact, from the time when the shooting mode is enabled, the user cannot capture an image according to the frame rate. For example, when the frame rate is 30 frames / sec, the terminal captures one frame at an interval of 1/30 second, and the terminal moves according to the difference between two images of adjacent frames. The determined position can be displayed on the terminal in order to determine the position of the terminal in the moving process and instruct the user to move the terminal to the shooting position of the next image.

Optionally, based on the optional embodiment corresponding to FIG. 1 above, in another embodiment of the method for obtaining a panoramic image provided in the embodiment of the present invention, capturing the first image, After obtaining the shooting parameters of the first image, the method further comprises:
Performing a center position calibration at predetermined image frame intervals, wherein the center position calibration compares the current preview image with the first image, and compares the current preview image with the first image. Is higher than a predetermined threshold, and when the position of the terminal cannot be detected, the photographing is restarted according to the position where the first image is photographed. Is determined to have returned to the position of the terminal.

  In this embodiment of the invention, the predetermined image frame may be ten frames, ie, the center position calibration is performed at intervals of ten frames. The purpose of the center position calibration is to determine the position where the first image was taken when the terminal cannot detect the position of the terminal. In this way, shooting can be resumed.

  Since the first image is typically taken by focusing on a human face, the predetermined threshold may be 80% or 90%, or another value. When the similarity between the preview image and the first image is higher than a predetermined threshold, the terminal may be determined to have returned to the position where the first image was captured.

Optionally, based on the embodiment corresponding to FIG. 1 or any optional embodiment corresponding to FIG. 1 above, another implementation of the method for obtaining a panoramic image provided in an embodiment of the present invention In the example,
Prior to capturing the first image and obtaining the capturing parameters of the first image, the method further includes:
Lock the automatic white balance (Auto White Balance, AWB) control and the automatic exposure (Auto-Exposure, AE) control so that the shooting parameters of the predetermined number of images to be shot continue to match the shooting parameters of the first image. It may have that.

  In order to collect multiple images during panoramic photography, the exposure settings and white balance settings need to be locked according to the exposure and white balance settings corresponding to the first image. Taking an image first in the middle can ensure that locked exposure and white balance settings render a human face better. If the image is taken first on the left, the luminance of the outdoor background is usually much greater than the luminance of the human face, and the human face is too dark.

Optionally, based on the embodiment corresponding to FIG. 1 or any optional embodiment corresponding to FIG. 1 above, another implementation of the method for obtaining a panoramic image provided in an embodiment of the present invention In the example, using the first image as a center and using a predetermined joining method, performing seamless joining on the first image and a predetermined number of images to obtain a panoramic image,
Detecting the position of the human image in both the first image and the predetermined number of images;
Performing image joining from the position of the non-human image in each image when image joining is performed by utilizing a predetermined joining method;
May be provided.

  In this embodiment of the present invention, in order to guarantee the joining effect, the person image should be prevented from being joined as much as possible, the image point with low gradient in the image is selected, and the area where the human face is located Should be avoided as much as possible.

  During image joining, feature points in each image may be extracted, wherein the scale-invariant feature transform (SIFT) extraction method, feature point extraction method, or corner point extraction method extracts feature points. It may be used to do so. The SIFT extraction method and the corner point extraction method belong to a well-known technique in the field of image processing, and will not be described in detail in this embodiment.

  After the feature points are extracted, feature points extracted from different images are compared, and feature points representing the same image character are matched. SIFT feature point matching may be used, or image block matching may be used. SIFT feature point matching and image block matching belong to well-known techniques in the field of image processing, and will not be described in detail in this embodiment.

  After the feature points are matched, feature point pairs are selected, feature point pairs representing the same image transformation are retained, feature point pairs pointing to other image transformations are deleted, and a random sample consensus (RANdom Sample Consensus, RANSAC). An algorithm may be used. A transformation matrix between the two images may also be calculated according to the feature point pairs selected by transforming the matrix.

  After the image is rotated according to the transformation matrix, the position of the human face is detected to prevent the human faces from being joined. The iterative algorithm ADABOOST algorithm may be used, or the FLOATBOOST algorithm may be used. These algorithms belong to well-known techniques in the field of image processing and will not be described in detail here.

  The joint seam between the two images is searched and the images are seamed. Images may be seamed by utilizing seam carving techniques. The seam carving technique belongs to a well-known technique and will not be described in detail here. During the selection of the joint seam, image points with a low gradient in the image are selected and the area where the human face is should be avoided as much as possible.

Referring to FIG. 4, the terminal provided in the embodiment of the present invention includes:
An obtaining unit 201 configured to obtain a panoramic shooting instruction triggered by a user;
A photographing unit 202 configured to photograph a first image after the acquiring unit 201 acquires a panoramic photographing instruction triggered by a user;
Here, the obtaining unit 201 is further configured to obtain a shooting parameter of the first image when the shooting unit shoots the first image;
A determining unit 203 configured to determine the moving guide information according to a predetermined moving guide policy after the obtaining unit 201 has obtained the first image;
A display unit 204 configured to display the travel guide information determined by the determination unit 203 on the terminal to instruct the user to move the terminal according to the travel guide information;
A detection unit 205 configured to detect a movement state of the terminal after the display unit 204 displays the movement guide information;
Here, the photographing unit 202 is further configured to photograph a predetermined number of images in accordance with the photographing parameters of the first image after the detection unit 205 detects that the terminal has moved, wherein the predetermined number of images is Background images on both the left and right sides of the background corresponding to the first image,
Using the first image as a center and using a predetermined joining method, seamless joining is performed on the first image and a predetermined number of images taken by the imaging unit 202 to obtain a panoramic image. A joining unit 206 configured to
Having.

  According to this embodiment of the present invention, the obtaining unit 201 obtains a panoramic shooting instruction triggered by the user, and the shooting unit 202 performs the first panoramic shooting instruction after the obtaining unit 201 obtains the panoramic shooting instruction triggered by the user. Photographing the image, the acquiring unit 201 further acquires a photographing parameter of the first image when the photographing unit photographs the first image, and after the acquiring unit 201 acquires the first image, the determining unit 203 Determines the travel guide information according to a predetermined travel guide policy, and the display unit 204 displays the travel guide information determined by the determination unit 203 on the terminal to instruct the user to move the terminal according to the travel guide information. Then, the detection unit 205 moves the terminal after the display unit 204 displays the movement guide information. State, and the photographing unit 202 further photographs a predetermined number of images according to the photographing parameters of the first image after the detection unit 205 detects that the terminal has moved, where the predetermined number of images is This is a background image on both the left side and the right side of the background corresponding to the first image. The joining unit 206 uses the first image as a center and uses a predetermined joining method to capture images by the imaging unit 202. A seamless joining is performed on the obtained first image and a predetermined number of images to obtain a panoramic image. Compared with the prior art in which the image at the center position cannot be enhanced during panoramic shooting, the terminal provided in the present embodiment of the present invention can enhance the image at the center position during panoramic shooting, and thereby the panoramic image Improve the overall effect of

Optionally, based on the embodiment corresponding to FIG. 4 above, in another embodiment of the terminal provided in the embodiment of the present invention,
The determining unit 203 is configured to determine a shooting position of a next image according to a predetermined movement guide policy,
The display unit 204 is configured to display a shooting position of a next image on the terminal,
The detection unit 205 is configured to detect whether the terminal is for the next image and moves to a shooting position displayed by the display unit 204,
After the detection unit 205 detects that the terminal has moved to the shooting position of the next image, the shooting unit 202 is configured to shoot the next image according to the shooting parameters of the first image,
The determination unit 203, the display unit 204, and the imaging unit 202 repeat the above-described determination, display, and imaging steps until a predetermined number of images are captured.

Optionally, based on the embodiment corresponding to FIG. 4 above, and referring to FIG. 5, in another embodiment of the terminal provided in the embodiment of the present invention,
The obtaining unit 201 is further configured to obtain the position of the terminal in real time when the terminal is moving,
The display unit 204 is further configured to display the position of the terminal obtained by the obtaining unit 201 in real time,
The detection unit 205 is configured to detect whether the position of the terminal displayed by the display unit is the same as the shooting position of the next image,
When the detection unit 205 detects that the position of the terminal displayed by the display unit is the same as the shooting position of the next image, the shooting unit 202 is configured to shoot the next image according to the shooting parameters of the first image. You.

Optionally, based on the embodiment corresponding to FIG. 5 above, in another embodiment of the terminal provided in the embodiment of the present invention,
The acquiring unit 201 detects a relative movement of an image of an adjacent frame to instruct a user to move the terminal to a position for acquiring a next image when the terminal is moving, It is configured to obtain the position of the terminal in the moving process in real time,
The display unit 204 is configured to display the position of the terminal during the movement process acquired by the acquisition unit 201.

Optionally, based on the embodiment corresponding to FIG. 5 above, and referring to FIG. 6, in another embodiment of the terminal provided in the embodiment of the present invention, the terminal 20 further includes:
After the photographing unit 202 has photographed the first image, it has a calibration unit 207 configured to perform a center position calibration at predetermined image frame intervals, wherein the center position calibration is performed by comparing the current preview image with the current preview image. The first image is compared with the first image, and when the matching similarity between the current preview image and the first image is higher than a predetermined threshold, and when the terminal position cannot be detected, the first image is captured. In order to resume shooting according to the position, it is determined that the terminal position has returned to the terminal position where the first image is shot.

Optionally, based on the embodiment corresponding to FIG. 4 above and referring to FIG. 7, in another embodiment of the terminal provided in the embodiment of the present invention, the terminal 20 further comprises:
Before the shooting unit 202 shoots the first image, the automatic white balance AWB control and the automatic exposure AE control are locked so that the shooting parameters of a predetermined number of shot images continue to match the shooting parameters of the first image. And a lock unit 208 configured to

Optionally, based on the embodiment corresponding to FIG. 4 above, and referring to FIG. 8, in another embodiment of the terminal provided in the embodiment of the present invention, the joining unit 206 further includes:
A detection sub-unit 2061 configured to detect the position of the human image in both the first image and the predetermined number of images;
A joining sub-unit 2062 configured to perform joining from the position of the non-human image in each image detected by the detection sub-unit 2061, when the image joining is performed by using a predetermined joining method;
Having.

  FIG. 9 shows a configuration of a terminal provided in an embodiment of the present invention. The terminal provided in this embodiment of the present invention may be configured to implement the method for acquiring a panoramic image in the embodiment of the present invention. For simplicity of description, only those parts of the present invention that are relevant to this embodiment are shown. For specific technical details not disclosed, reference is made to parts in the method embodiments of the present invention for understanding.

  The terminal may include a terminal such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant, personal digital assistant), a POS (Point of Sales, point of sales), and a vehicle-mounted computer. For example, the terminal is a mobile phone. FIG. 9 is a block diagram of a partial configuration of a mobile phone 400 related to a terminal provided in this embodiment of the present invention. Referring to FIG. 9, a mobile phone 400 includes an RF (Radio Frequency, radio frequency) circuit 410, a memory 420, an input unit 430, a display device 440, a sensor 450, an output unit 451, an audio frequency circuit 460, and a wireless fidelity (wireless). (WiFi) module 470, a processor 480, and a power supply 490. One skilled in the art will appreciate that the configuration of the mobile phone shown in FIG. 9 does not constitute a limitation on the mobile phone, and that the mobile phone may have more or less components, some combinations of components or differently arranged components than those shown in FIG. It may be understood that components may be included.

  The components of the mobile phone 400 are described in detail below with reference to FIG.

  The RF circuit 410 transmits and receives signals in information transmission and reception processing or call processing, and in particular, after receiving downlink information of the base station, transmits downlink information to the processor 480 for processing, and further transmits uplink data to the base station. It may be configured to transmit to a station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, and the like. Further, the RF circuit 410 may further communicate with other devices by wireless communication. Wireless communications include, without limitation, Global System of Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (GPRS), Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (Long Term Ession, LTE e-mailing, Email Service, Long Term E-mail, Short Term Email Service, Long Term Email Service, Long Term Email Service, Long Term Email Service, Long Term Email Service) SMS) may be used either communication standards or protocols, including.

  The memory 420 may be configured to store software programs and software modules, and the processor 480 executes the software programs and software modules stored in the memory 420 to execute various function applications and data processing of the mobile phone 400. Execute The memory 420 may mainly include a program storage area and a data storage area, where an operating system, application programs required by at least one function (such as a sound reproduction function and an image reproduction function), and the like are stored in the program storage area. It may be stored in an area. Data created according to the use of the mobile phone 400 (such as audio data and an address book) may be stored in a data storage area. Further, memory 420 may include high speed random access memory, or may include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state memory device. Good.

  The input unit 430 may be configured to receive input digital or textual information and generate key signal inputs related to function control of the mobile phone 400 and user settings. Specifically, the input unit 430 may include a touch panel 431 and another input device 432. Touch panel 431 is also referred to as a touch screen, and may be touched on or near touch screen by a user (eg, by utilizing any suitable object or accessory, such as a finger or stylus, on touch panel 431 or An operation performed by the user near the touch panel 431 is collected, and the corresponding connection device can be driven according to a predetermined program. Optionally, the touch panel 431 may have two parts, a touch detection device and a touch controller. The touch detection device detects a touch direction of the user, detects a signal generated by a touch operation, and transmits the signal to the touch controller. The touch controller can receive touch information from the touch detection device, convert the touch information into contact coordinates, send the contact coordinates to the processor 480, receive a command from the processor 480, and execute the command. Further, the touch panel 431 may be realized by various types such as a resistance type, a capacitance type, an infrared type, and a surface acoustic wave. In addition to the touch panel 431, the input unit 430 may further include another input device 432. In particular, other input devices 432 include, without limitation, one or more of a physical keyboard, function keys (eg, volume control keys, switch keys, etc.), a trackball, a mouse, and an operating rod. Is also good.

  The display device 440 may be configured to display information input by or provided to the user and various menus of the mobile phone 400. The display device 440 may have a display panel 441, and optionally, the display panel 441 in the form of a liquid crystal display (LCD), an organic light emitting diode (Organic Light-Emitting Diode, OLED), or the like. You may comprise. Further, the touch panel 431 may cover the display panel 441, and when the touch panel 431 detects a touch operation performed on the touch panel 431, the touch operation may be transmitted to the processor 480 to determine a type of a touch event. At this time, the processor 480 provides a corresponding visual output on the display panel 441 according to the type of the touch event. In FIG. 9, the touch panel 431 and the display panel 441 function as two independent components for realizing the input / output function of the mobile phone 400, but in some embodiments, the touch panel 431 and the display panel 441 are It may be integrated to realize the input / output function of the telephone 400.

  Cell phone 400 may further include at least one type of sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, where the ambient light sensor adjusts the brightness of the display panel 441 according to the brightness and darkness of the ambient light, and the proximity sensor When the mobile phone 400 moves closer to the ear, the display panel 441 and / or the backlight may be turned off. As one type of motion sensor, the acceleration sensor can detect acceleration in various directions (usually three axes), and can detect the value and direction of gravity when the mobile phone 400 is stationary. It can be used in applications to determine the orientation of the mobile phone (switch between landscape and portrait orientation, related games and magnetometer attitude measurement, etc.), vibration identification related functions (pedometer, knocking, etc.), etc. . Other sensors such as gyroscopes, barometers, hygrometers, thermometers and infrared sensors for the mobile phone 400 may be further configured and will not be described in detail herein.

  The output unit 451 may be configured to transmit a signal.

  Audio frequency circuit 460, loudspeaker 461, and microphone 462 can provide an audio interface between the user and mobile phone 400. The audio frequency circuit 460 can transmit an electric signal obtained by converting the received audio data to the loudspeaker 461, and then the loudspeaker 461 converts the electric signal into an audio signal for output. In another aspect, microphone 462 converts the collected audio signals to electrical signals, and then, after receiving the electrical signals, audio frequency circuit 460 converts the electrical signals to audio data and processes the audio data for processing. Output to processor 480. The audio data is then transmitted by utilizing the RF circuit 410, for example, to another mobile phone, or output to the memory 420 for further processing.

  WiFi belongs to the short-range wireless transmission technology, and the mobile phone 400 can use the WiFi module 470 to help the user send and receive e-mails, browse web pages, and access streaming media. WiFi provides users with wireless broadband Internet access. Although FIG. 9 illustrates the WiFi module 470, the WiFi mode 470 is not a necessary part of the mobile phone 400 and may be omitted as necessary without departing from the scope of the present invention.

  The processor 480 is a control center of the mobile phone 400 and is connected to various parts of the entire mobile phone by using various interfaces and lines, and performs overall monitoring on the mobile phone. , Execute software programs and / or modules stored in the memory 420, and process data by calling the data stored in the memory 420. Optionally, processor 480 may have one or more processing units. Preferably, the processor 480 may be integrated with an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application program, and the like. The modem processor mainly processes wireless communication. It may be appreciated that the modem processor described above may not be integrated with processor 480.

  The mobile phone 400 further includes a power supply 490 (for example, a battery) that supplies power to various components. Preferably, the power supply may be logically connected to the processor 480 by utilizing the power management system to achieve functions such as charging, discharging and power consumption management by utilizing the power management system.

  The mobile phone 400 may further include a camera 491, where the camera 491 captures an image.

  The mobile phone 400 may further include a Bluetooth module (not shown), and details are not described herein.

  In this embodiment of the present invention, the processor 480 included in the terminal further has the following functions.

  The processor 480, after the panoramic shooting instruction triggered by the user is obtained, shoots the first image, obtains the shooting parameters of the first image, determines the moving guide information according to a predetermined moving guide policy, In order to instruct the user to move the terminal according to the movement guide information, the movement guide information is displayed on the terminal, and after detecting that the terminal has moved, a predetermined number of images are photographed according to the photographing parameters of the first image. Here, the predetermined number of images are background images on both the left side and the right side of the background corresponding to the first image. By using the first image as a center and using a predetermined joining method, , Performing seamless joining on the first image and the predetermined number of images to obtain a panoramic image.

  In this embodiment of the present invention, when a panoramic image is taken, the middle image may be taken first, then the two side images may be taken, and the image at the center position can be enhanced. This improves the overall effect of the panoramic image.

  In some embodiments of the present invention, processor 480 is further configured to determine a next image capture location according to a predetermined travel guide policy.

  The display device 440 may display the shooting position of the next image on the terminal.

  The camera 491 is further configured to shoot the next image according to the shooting parameters of the first image after the terminal moves to the shooting position of the next image.

  The processor 480, the display device 440, and the camera 491 repeat the above determination, display, and shooting steps until a predetermined number of images have been shot.

  In some embodiments of the present invention, processor 480 is further configured to obtain the location of the terminal in real time as the terminal is moving.

  The display device 440 may further display the location of the terminal.

  Processor 480 is further configured to trigger the camera to capture the next image when the displayed position of the terminal is the same as the capture position of the next image.

  Camera 491 is further configured to capture the next image according to the capture parameters of the first image.

  In some embodiments of the present invention, the processor 480 may further include, when the terminal is moving, the relative position of the image of the adjacent frame to instruct the user to move the terminal to a position for acquiring the next image. It is configured to obtain the position of the terminal in the moving process in real time by detecting a dynamic movement.

  The display device 440 may display the position of the terminal in the moving process.

  In some embodiments of the present invention, processor 480 is further configured to perform a center position calibration at predetermined image frame intervals, where the center position calibration is performed with the current preview image and the first image. When the matching similarity between the current preview image and the first image is higher than a predetermined threshold, and when the position of the terminal cannot be detected, shooting is restarted according to the position where the first image is shot. Therefore, it is determined that the position of the terminal has returned to the position of the terminal where the first image is captured.

  In some embodiments of the present invention, processor 480 further includes an automatic white balance AWB control and an automatic exposure AE control such that the imaging parameters of the predetermined number of captured images continue to match the imaging parameters of the first image. Is configured to lock.

  In some embodiments of the invention, the processor 480 further detects the position of the human image in both the first image and the predetermined number of images, and the image combining is performed by utilizing a predetermined combining scheme. Then, it is configured to execute the joining from the position of the non-human image in each image.

  Those skilled in the art may understand that all or some of the steps of the method in the above embodiments may be realized by a program instructing relevant hardware. The program may be stored on a computer-readable storage medium. The storage medium may include a ROM, a RAM, a magnetic disk, or an optical disk.

  The method and terminal for obtaining a panoramic image provided in the embodiment of the present invention will be described in detail below. This specification describes the principles and implementations of the present invention by way of specific examples. The above description of the embodiments is merely intended to assist in understanding the method of the present invention and its core ideas. On the other hand, those skilled in the art may make modifications to the specific implementation and scope according to the idea of the present invention. In conclusion, the contents herein should not be construed as limitations on the invention.

Claims (16)

A method for acquiring a panoramic image by a mobile terminal having a front camera ,
Obtaining a shadow instruction Ta triggered by the user,
In response to the photographing instruction, photographing a center image by the front camera ,
After capturing the center image, displaying on a display moving guide information for the user to instruct the first side of the user to rotate the mobile terminal;
Detecting that the mobile terminal has been rotated toward a first side of the user;
Automatically capturing an image of the first side by the front camera when the front camera faces the background on the first side of the user;
Displaying the movement guide information on the display for instructing the user to rotate the mobile terminal on the second side of the user after capturing the image of the first side, Displaying the second side is opposite to the first side;
Detecting that the mobile terminal has been rotated toward a second side of the user;
Automatically capturing an image of a second side by the front camera when the front camera faces the background on the second side of the user;
Joining the center image, the first side image, and the second side image to obtain a panoramic image based on the center image at the center of the panoramic image;
Having a method.   The method of claim 1, wherein the travel guide information comprises text guide information. The moving guide information further includes an icon guide information, The method of claim 1, wherein. The icon guide information, said towards the first side and the second side of the user is shown only in order to instruct the user to capture an image, The method of claim 3. Those method further the comprising the step of displaying the real-time location of the mobile terminal, the real-time position Ri relative der against first-term position of the portable terminal, the initial position of the portable terminal the portable terminal Ru place der to shoot the central image, claims 1 to 4 or the method of one claim. Displaying a real-time location of said mobile terminal when said mobile terminal is rotating, by detecting the relative motion of the image of the adjacent frames, said real-time location of the mobile terminal in rotation process 6. The method of claim 5, comprising displaying. 6. The method of claim 5, further comprising locking at least one of an automatic white balance AWB or an automatic exposure AE after capturing the central image and before capturing the first side image. The described method. A mobile terminal having a touch screen, a front camera, one or more processors, and a memory , wherein the memory stores a program, and the program is executed by the one or more processors.
Obtaining a shadow instruction Ta triggered by the user,
In response to the photographing instruction, photographing a center image by the front camera ,
After capturing the center image, displaying on a display moving guide information for the user to instruct the first side of the user to rotate the mobile terminal;
Detecting that the mobile terminal has been rotated toward a first side of the user;
Automatically capturing an image of the first side by the front camera when the front camera faces the background on the first side of the user;
Displaying the movement guide information on the display for instructing the user to rotate the mobile terminal on the second side of the user after capturing the image of the first side, Displaying the second side is opposite to the first side;
Detecting that the mobile terminal has been rotated toward a second side of the user;
Automatically capturing an image of a second side by the front camera when the front camera faces the background on the second side of the user;
Joining the center image, the first side image, and the second side image to obtain a panoramic image of the center image at the center of the panoramic image;
A portable terminal that causes the portable terminal to execute . The mobile terminal according to claim 8 , wherein the travel guide information includes text guide information. The mobile terminal according to claim 8 , wherein the movement guide information further includes icon guide information. The icon guide information, first only towards the side and the second side is shown to indicate to the user to capture an image, the portable terminal according to claim 10, wherein the user. When the program is executed by the one or more processors, said step of displaying a real-time location of the mobile terminal further executes the portable terminal, wherein the real-time location relative with respect to the first-term position of said mobile terminal specifically der it is, the initial position of the mobile terminal Ru place der which the mobile terminal captures the central image, the portable terminal of any one of claims 9 to 11. When the program is executed by the one or more processors, when said mobile terminal is rotating, by detecting the relative motion of the image of the adjacent frames, the said mobile terminal in rotation process the step of displaying the real-time location Ru further executed to the portable terminal, the portable terminal according to claim 12, wherein. When the program is executed by the one or more processors, even after taken the central image, before capturing an image of said first side, at least one of the self Dohakushoku balance AWB or automatic exposure AE 13. The mobile terminal according to claim 12 , further comprising the step of locking the mobile terminal. A computer-readable storage medium storing one or more programs, wherein the one or more programs are executed by a mobile terminal having a front camera,
Obtaining a shadow instruction Ta triggered by the user,
In response to the photographing instruction, photographing a center image by the front camera ,
After capturing the center image, displaying on a display moving guide information for the user to instruct the first side of the user to rotate the mobile terminal;
Detecting that the mobile terminal has been rotated toward a first side of the user;
Automatically capturing an image of the first side by the front camera when the front camera faces the background on the first side of the user;
Displaying the movement guide information on the display for instructing the user to rotate the mobile terminal on the second side of the user after capturing the image of the first side, Displaying the second side is opposite to the first side;
Detecting that the mobile terminal has been rotated toward a second side of the user;
Automatically capturing an image of a second side by the front camera when the front camera faces the background on the second side of the user;
Joining the center image, the first side image, and the second side image to obtain a panoramic image of the center image at the center of the panoramic image;
A storage medium containing instructions for causing the portable terminal to execute the following. A computer program having computer-executable instructions, wherein the instructions are executed by a processor,
A process of acquiring the shadow instruction Ta triggered by the user,
In response to the shooting instruction, a process of shooting a center image by a front camera ,
After capturing the center image, a process of displaying, on a display, movement guide information for the user to instruct the first side of the user to rotate the mobile terminal;
Detecting that the mobile terminal has been rotated toward a first side of the user;
Automatically capturing an image of the first side by the front camera when the front camera faces the background on the first side of the user;
After capturing the image of the first side, a process of displaying on the display the movement guide information for the user to instruct the second side of the user to rotate the mobile terminal, Displaying the second side is opposite to the first side;
Detecting that the mobile terminal has been rotated toward the second side of the user;
When the front camera faces the background on the second side of the user, a process of automatically capturing an image of a second side by the front camera;
A process of joining the center image, the image on the first side, and the image on the second side to obtain a panoramic image based on the center image at the center of the panoramic image;
A computer program that executes.

Images