CN106101552B - A kind of panoramic photographing method and mobile terminal - Google Patents

Abstract

The embodiment of the invention discloses a panoramic photographing method and a mobile terminal. The method is applied to a mobile terminal, and the method includes: when receiving a panoramic photographing instruction input by a user of the mobile terminal, controlling the camera to continuously capture at least one frame of images; during the process of continuously capturing images by the camera, recording the orientation information of the mobile terminal when capturing each frame of the at least one frame of image; performing image splicing on the at least one frame of image to generate a panoramic photo, each frame of the panoramic photo carrying a corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of images carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Description

A kind of panoramic photographing method and mobile terminal

technical field

Embodiments of the present invention relate to the technical field of image processing, and in particular, to a panoramic photographing method and a mobile terminal.

Background technique

With the continuous development of terminal technology, the camera has become one of the basic configurations of the terminal. Based on panoramic photography, images with a wider field of view can be obtained. Therefore, many terminals are equipped with a panoramic photography function.

When the user takes a panoramic photo, the camera of the fan-shaped mobile terminal generally takes a certain point as the center to take multiple partial pictures, and the terminal stitches the multiple partial pictures into one panoramic picture and displays it to the user. Since the panorama picture is taken by the user in a fixed-point fan shape, rather than moving in parallel with the scene, the resulting panorama picture has image distortion and cannot obtain the real scene shooting effect.

Contents of the invention

An embodiment of the present invention provides a panoramic photographing method to solve the problem in the background art that there is image curvature and distortion in the obtained panoramic picture, and a real scene photographing effect cannot be obtained.

In a first aspect, a panoramic photographing method is provided, the method comprising:

When receiving a panoramic photographing instruction input by the user of the mobile terminal, controlling the camera to continuously capture at least one frame of images;

During the process of the camera continuously capturing images, recording the orientation information of the mobile terminal when capturing each frame of the at least one frame of images;

Image splicing is performed on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information.

In a second aspect, a mobile terminal is provided, the mobile terminal has a camera, including:

An image acquisition module, configured to control the camera to continuously acquire at least one frame of images when a panoramic photographing instruction input by the user of the mobile terminal is received;

An azimuth information recording module, configured to record the azimuth information of the mobile terminal when each frame of at least one frame of images collected by the image acquisition module is captured during the process of the image acquisition module controlling the camera to continuously acquire images;

A panoramic photo generation module, configured to stitch the at least one frame of images collected by the image acquisition module to generate a panoramic photo, where each frame of the panoramic photo carries the corresponding information recorded by the orientation information recording module. orientation information.

According to an embodiment of the present invention, during the panoramic photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and perform image splicing on the at least one frame of images to generate a panoramic photo, Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained according to these drawings without paying creative labor.

Fig. 1 is the flow chart of the panoramic photographing method of an embodiment of the present invention;

Fig. 2 is a flow chart of a panoramic photographing method according to another embodiment of the present invention;

Fig. 3 is a block diagram of a mobile terminal according to an embodiment of the present invention;

Fig. 4 is a block diagram of a mobile terminal according to another embodiment of the present invention;

Fig. 5 is a block diagram of a mobile terminal according to another embodiment of the present invention;

FIG. 6 is a block diagram of a mobile terminal according to another embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Example 1

Fig. 1 is a flow chart of a panoramic photographing method according to an embodiment of the present invention. The panoramic photographing method shown in Fig. 1 comprises:

Step 101. When receiving a panoramic photographing instruction input by a user of the mobile terminal, control the camera to continuously capture at least one frame of images.

In the embodiment of the present invention, the mobile terminal includes a mobile phone, a tablet, a notebook computer, and the like.

The mobile terminal is equipped with a camera, which can be used to take photos and record videos. A mobile terminal with a higher configuration can take panoramic shots through the camera, and stitch multiple frames of images to obtain a panoramic photo with a wider viewing angle.

The user can select a panorama photography entry in a certain menu of the camera software loaded on the mobile terminal, so that the mobile terminal enters the panorama photography mode. After the mobile terminal receives the panoramic photographing instruction input by the user, it controls the camera to continuously collect at least one frame of images for the target scene.

Step 102. During the process of the camera continuously capturing images, record the orientation information of the mobile terminal when capturing each frame of the at least one frame of images.

In the embodiment of the present invention, in the process of using the camera to continuously capture images, related equipment installed inside the mobile terminal is used to record the orientation information of the mobile terminal when capturing each frame of image, wherein the orientation information of the mobile terminal can include various forms, such as Direction information, displacement information, location information, etc.

The mobile terminal can take pictures of panoramas at a fixed point, and can also take pictures of panoramas while moving. When the mobile terminal takes a panorama at a fixed point, the compass placed inside the mobile terminal can be used to collect the direction information of the mobile terminal; The GPS positioning device collects displacement information or location information of the mobile terminal, and other applicable devices can also be used to collect orientation information of the mobile terminal.

Further, each frame of image collected and the orientation information of the mobile terminal when each frame of image is collected are recorded, and each frame of image and the corresponding recorded orientation information are combined and saved.

Step 103: Perform image splicing on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information.

After the panorama is photographed, at least one frame of image is collected, and at least one frame of the collected image is stitched to generate a panoramic photo. In the generated panoramic photo, each frame of image carries corresponding orientation information. The splicing operation of multi-frame images is a prior art, and the present invention will not repeat it here.

Because in the resulting panoramic photo, each frame of image carries corresponding orientation information, it is possible to control the display effect of the panoramic photo according to the orientation information carried by each frame of image, that is, the orientation information of the mobile terminal when each frame of image is taken, and make the panoramic photo The display effect is more in line with the actual scene.

According to an embodiment of the present invention, during the panoramic photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and perform image splicing on the at least one frame of images to generate a panoramic photo, Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Example 2

Fig. 2 is a flow chart of a panoramic photographing method according to another embodiment of the present invention. The panoramic photographing method shown in Figure 2 includes:

Step 201. When receiving a panoramic photographing instruction input by the user of the mobile terminal, control the camera to continuously capture at least one frame of images.

Step 202. During the process of the camera continuously capturing images, record the orientation information of the mobile terminal when capturing each frame of the at least one frame of images.

The orientation information may include one or more of direction information and displacement information. In the process of using the camera to continuously capture images, the direction information of the mobile terminal when capturing each frame of image, or the direction information and displacement information of the mobile terminal can be selectively recorded according to whether the mobile terminal moves.

Step 203: Perform image stitching on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information.

In the process of collecting images, if the position of the mobile terminal does not move, each frame of image in the panoramic photo carries the direction information of the mobile terminal when the frame of image is taken; if the mobile terminal moves, usually the mobile terminal keeps moving parallel to the scene , each frame of image in the panorama photo carries direction information and displacement information of the mobile terminal when the frame of image is captured.

Step 204, receiving an instruction for viewing the panoramic photo input by the user of the mobile terminal.

Users can view the panorama by clicking the thumbnail of the panorama in the gallery; or view the panorama in other locations.

Step 205, acquire the screen type of the display screen of the mobile terminal.

In the embodiment of the present invention, the screen type includes a flexible screen and a non-flexible screen, and the non-flexible screen is a plane non-deformable screen.

After receiving the instruction to view the panoramic photo, it is necessary to first determine whether the display screen of the mobile terminal is a flexible screen or a non-deformable screen.

Step 206. Based on the screen type, determine a display mode of the panoramic photo, and display the panoramic photo according to the display mode.

Panoramas are displayed differently depending on the type of screen on which the display is based.

If the screen type of the display screen is a flexible screen, each screen area of the flexible screen is determined based on the orientation information carried by each frame of image in at least one frame of image collected, that is, based on the orientation information of the mobile terminal when each frame of image is collected The degree of bending; further control the flexible screen to bend according to the degree of bending of each screen area; finally display the panoramic photo on the curved flexible screen.

Specifically, multiple driving devices can be installed inside the flexible screen, and according to the degree of curvature of each screen area determined by the orientation information carried by each frame of image, the movement of the corresponding driving device is controlled according to the determined degree of curvature, such as forward and backward movement , so as to realize the bending control of the flexible screen. The panoramic photos displayed on the curved flexible screen are closer to the actual scene.

After the bending of the flexible screen is controlled, the center position of the bent flexible screen can be calculated. Compared with other positions, mobile terminal users can obtain a more realistic panoramic display effect when watching at the center position.

If the screen type of the display screen is a plane non-deformable screen, then based on the orientation information carried by each frame of image in at least one frame of image collected, determine the stereo model for displaying the panoramic image; further display the panoramic photo in the determined stereo on the model.

In the prior art, in order to make the photographed photos have a three-dimensional display effect, the photographs are displayed on a three-dimensional model and further displayed to the user. The three-dimensional model is generally a spatial structure, including an X axis indicating a horizontal direction, a Y axis indicating a height direction, and a Z axis indicating a front-to-back direction. The embodiment of the present invention can adjust the size of the Z-axis data according to the orientation information carried by each frame of image to realize the adjustment of the three-dimensional model, thereby realizing the control of the display effect of the panoramic image.

In the process of continuously capturing images by the camera, the depth of field information of each frame of images in at least one frame of continuously captured images may also be recorded. Correspondingly, this step may include: if the screen type of the display screen is a flexible screen, then based on the depth of field information of each frame of image in at least one frame of image continuously collected, determine the level information of the image area corresponding to the depth of field information in the panoramic photo, That is, the front-to-back positional relationship of different image areas; further, in the process of displaying the panoramic photo, based on the determined layer information, the flexible screen is controlled to deform. The embodiments of the present invention can control the bending of the flexible screen according to the field depth information of each frame of image, so that the displayed panoramic image is closer to the actual scene.

In the process of displaying the panoramic photo, it is also possible to receive the click operation of the mobile terminal user on the panoramic photo, such as a click operation for a certain image area. If the image area where the click operation is located contains depth information, based on the depth information, the Click the image area where the operation is performed to perform depth-of-field effect processing corresponding to the depth-of-field information. Specifically, a click operation of the mobile terminal user on the image area is received, based on the depth of field information, the image in the image area becomes clear, and correspondingly, the images in other image areas become blurred. The above method can satisfy the user's requirement for viewing partial images, and makes the viewed partial images clearer.

According to an embodiment of the present invention, during the panoramic photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and perform image splicing on the at least one frame of images to generate a panoramic photo, Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Example 3

FIG. 3 is a block diagram of a mobile terminal according to an embodiment of the present invention. The mobile terminal shown in Figure 3 includes:

The image acquisition module 301 is configured to control the camera to continuously acquire at least one frame of images when receiving a panoramic photographing instruction input by the user of the mobile terminal.

The orientation information recording module 302 is configured to record the orientation information of the mobile terminal when the image acquisition module controls each frame of image in at least one frame of images acquired by the image acquisition module during the process of the image acquisition module controlling the continuous acquisition of images by the camera .

The panoramic photo generation module 303 is configured to stitch the at least one frame of images collected by the image acquisition module to generate a panoramic photo, and each frame of the panoramic photo carries the corresponding information recorded by the orientation information recording module. location information.

Referring to FIG. 4 , on the basis of FIG. 3 , other modules of the mobile terminal, submodules included in the modules and their functions are described.

Fig. 4 is a block diagram of a mobile terminal according to another embodiment of the present invention. The mobile terminal shown in Figure 4 includes:

The image acquisition module 401 is configured to control the camera to continuously acquire at least one frame of images when receiving a panoramic photographing instruction input by the user of the mobile terminal.

The orientation information recording module 402 is configured to record the orientation information of the mobile terminal when the image acquisition module controls each frame of image in at least one frame of images acquired by the image acquisition module during the process of the image acquisition module controlling the continuous acquisition of images by the camera .

The panoramic photo generating module 403 is configured to perform image splicing on the at least one frame of images collected by the image acquisition module to generate a panoramic photo, and each frame of the image in the panoramic photo carries a corresponding location information.

The viewing instruction receiving module 404 is configured to receive a viewing instruction for the panoramic photo input by the user of the mobile terminal after the step of stitching the at least one frame of images by the panoramic photo stitching module to generate a photo.

The screen type acquiring module 405 is configured to acquire the screen type of the display screen of the mobile terminal.

The panoramic photo display module 406 is configured to determine the display mode of the panoramic photo based on the screen type acquired by the screen type acquisition module, and display the panoramic photo according to the display mode.

In the embodiment of the present invention, preferably, the panoramic photo display module 406 includes:

The bending degree determining sub-module 4061 is configured to determine each screen of the flexible screen based on the orientation information carried in each frame of the at least one frame of image if the screen type acquired by the screen type acquiring module is a flexible screen. The degree of curvature of the screen area;

A bending control submodule 4062, configured to control the flexible screen to bend according to the bending degree of each screen area determined by the bending degree determining submodule;

The first panoramic photo display submodule 4063 is configured to display the panoramic photo on the curved flexible screen controlled by the bending control submodule.

In the embodiment of the present invention, preferably, the panoramic photo display module 406 includes:

The three-dimensional model determination sub-module 4064 is configured to determine, based on the orientation information carried in each frame of the at least one frame of images, the screen used to display the Stereoscopic model of the panoramic image;

The second panorama photo display sub-module 4065 is used to display the panorama photo on the stereo model determined by the stereo model determination sub-module.

In the embodiment of the present invention, preferably, the mobile terminal further includes:

The depth-of-field information recording module 407 is configured to record the depth-of-field information of each frame of the at least one frame of image during the process of the image acquisition module controlling the camera to continuously acquire images.

In the embodiment of the present invention, preferably, the panoramic photo display module 406 includes:

The layer information confirmation sub-module 4066 is configured to determine the depth of field in the panoramic photo based on the depth of field information of each frame of image in the at least one frame of image if the screen type obtained by the screen type obtaining module is a flexible screen. Hierarchical information of the image area corresponding to the information;

The deformation control sub-module 4067 is configured to control the flexible screen to deform based on the layer information confirmed by the layer information confirmation sub-module during the process of displaying the panoramic photo.

In the embodiment of the present invention, preferably, the mobile terminal further includes:

The click operation receiving module 408 is used for receiving the panoramic photo displayed by the mobile terminal user during the process of displaying the panoramic photo after the panoramic photo display module displays the panoramic photo according to the display mode. click action on

The depth-of-field effect processing execution module 409 is configured to, if the image area where the click operation is received by the click operation receiving module contains the depth-of-field information, then based on the depth-of-field information, perform the same operation as the depth-of-field information on the image area where the click operation is located. Corresponding depth of field effect processing.

In the embodiment of the present invention, preferably, the orientation information includes direction information and displacement information.

According to an embodiment of the present invention, during the panoramic photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and perform image splicing on the at least one frame of images to generate a panoramic photo, Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Example 4

FIG. 5 is a block diagram of a mobile terminal according to another embodiment of the present invention. The mobile terminal 500 shown in FIG. 5 includes: at least one processor 501 , a memory 502 , at least one network interface 504 and a user interface 503 . Various components in the mobile terminal 500 are coupled together through the bus system 505 . It can be understood that the bus system 505 is used to realize connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in FIG. 5 , various buses are marked as a bus system 505 , and the mobile terminal 500 also includes a camera 506 .

Wherein, the user interface 503 may include a display, a keyboard, or a pointing device (for example, a mouse, a trackball (trackball), a touch panel, or a touch screen, and the like.

It can be understood that the memory 502 in the embodiment of the present invention may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the non-volatile memory can be read-only memory (Read-OnlyMemory, ROM), programmable read-only memory (ProgrammableROM, PROM), erasable programmable read-only memory (ErasablePROM, EPROM), electrically erasable Programming read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be random access memory (Random Access Memory, RAM), which acts as an external cache. By way of illustration and not limitation, many forms of RAM are available such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (Synchronous DRAM, SDRAM), Double data rate synchronous dynamic random access memory (DoubleDataRateSDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (SynchlinkDRAM, SLDRAM) and direct memory bus random access Memory (Direct Rambus RAM, DRRAM). The memory 502 of the systems and methods described in embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

In some implementations, the memory 502 stores the following elements, executable modules or data structures, or their subsets, or their extended sets: an operating system 5021 and an application program 5022 .

Among them, the operating system 5021 includes various system programs, such as framework layer, core library layer, driver layer, etc., for realizing various basic services and processing tasks based on hardware. The application program 5022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., and is used to implement various application services. The program for realizing the method of the embodiment of the present invention may be included in the application program 5022 .

In the embodiment of the present invention, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application program 5022, the processor 501 is configured to During this time, the camera is controlled to continuously collect at least one frame of images; during the process of the camera continuously collecting images, the orientation information of the mobile terminal when capturing each frame of the at least one frame of images is recorded; the at least one Image splicing is performed on the frame images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information.

The methods disclosed in the foregoing embodiments of the present invention may be applied to the processor 501 or implemented by the processor 501 . The processor 501 may be an integrated circuit chip and has signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 501 or instructions in the form of software. The above-mentioned processor 501 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, and the like. The steps of the methods disclosed in the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and completes the steps of the above method in combination with its hardware.

It can be understood that the embodiments described in the embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing device (DSPDevice, DSPD), programmable logic device (ProgrammableLogicDevice, PLD ), Field-Programmable Gate Array (Field-Programmable GateArray, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing the functions described in this application, or a combination thereof.

For software implementation, the techniques described in the embodiments of the present invention may be implemented through modules (such as procedures, functions, etc.) that execute the functions described in the embodiments of the present invention. Software codes can be stored in memory and executed by a processor. Memory can be implemented within the processor or external to the processor.

Optionally, the processor 501 is further configured to receive an instruction for viewing the panoramic photo input by the user of the mobile terminal after the step of stitching the at least one frame of images to generate a photo;

Obtain the screen type of the display screen of the mobile terminal;

Based on the screen type, a display manner of the panoramic photo is determined, and the panoramic photo is displayed according to the display manner.

Optionally, the processor 501 is further configured to: if the screen type is a flexible screen, determine the curvature of each screen area of the flexible screen based on the orientation information carried in each frame of the at least one image frame degree;

controlling the flexible screen to bend according to the bending degree of each screen area;

The panoramic photo is displayed on the curved flexible screen.

Optionally, the processor 501 is further configured to: if the screen type is a plane non-deformable screen, based on the orientation information carried in each frame of the at least one frame of image, determine an image for displaying the panoramic image. Diorama;

The panorama is displayed on the diorama.

Optionally, the processor 501 is further configured to: record the depth-of-field information of each frame of the at least one frame of images during the process of the camera continuously capturing images.

Optionally, the processor 501 is further configured to: if the screen type is a flexible screen, based on the depth of field information of each frame of image in the at least one frame of image, determine the image corresponding to the depth of field information in the panoramic photo Hierarchical information of the region;

In the process of displaying the panoramic photo, the flexible screen is controlled to deform based on the layer information.

Optionally, the processor 501 is further configured to: receive a click operation on the panoramic photo by the user of the mobile terminal during the process of displaying the panoramic photo;

If the image area where the click operation is located includes the depth of field information, then based on the depth of field information, perform depth of field effect processing corresponding to the depth of field information on the image area where the click operation is located.

Optionally, the processor 501 is further configured to: the orientation information includes direction information and displacement information.

The mobile terminal 500 can implement various processes implemented by the mobile terminal in the foregoing embodiments, and to avoid repetition, details are not repeated here.

According to an embodiment of the present invention, during the panoramic photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and perform image splicing on the at least one frame of images to generate a panoramic photo, Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Example 5

Fig. 6 is a schematic structural diagram of a mobile terminal according to another embodiment of the present invention. Specifically, the mobile terminal 600 in FIG. 6 may be a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or a vehicle-mounted computer.

Mobile terminal 600 in Fig. 6 comprises radio frequency (RadioFrequency, RF) circuit 610, memory 620, input unit 630, display unit 640, processor 660, audio frequency circuit 670, WiFi (WirelessFidelity) module 680 and power supply 690, described mobile terminal 600 also includes a camera 650 .

Wherein, the input unit 630 can be used to receive number or character information input by the user, and generate signal input related to the user setting and function control of the mobile terminal 600 . Specifically, in the embodiment of the present invention, the input unit 630 may include a touch panel 631 . The touch panel 631, also referred to as a touch screen, can collect user's touch operations on or near it (such as the user's operation on the touch panel 631 using any suitable object or accessory such as a finger, a stylus), and The specified program drives the corresponding connected device. Optionally, the touch panel 631 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the to the processor 660, and can receive and execute commands sent by the processor 660. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 631, the input unit 630 may also include other input devices 632, which may include but not limited to physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, etc. one or more of.

Wherein, the display unit 640 can be used to display information input by the user or information provided to the user and various menu interfaces of the mobile terminal 600 . The display unit 640 may include a display panel 641. Optionally, the display panel 641 may be configured in the form of an LCD or an organic light-emitting diode (Organic Light-Emitting Diode, OLED).

It should be noted that the touch panel 631 can cover the display panel 641 to form a touch display screen. When the touch display screen detects a touch operation on or near it, it is sent to the processor 660 to determine the type of the touch event, and then the processor 660 provides corresponding visual output on the touch display screen according to the type of the touch event.

The touch display screen includes an application program interface display area and a common control display area. The arrangement of the display area of the application program interface and the display area of the commonly used controls is not limited, and may be an arrangement in which the two display areas can be distinguished, such as vertical arrangement, left-right arrangement, and the like. The application program interface display area can be used to display the interface of the application program. Each interface may include at least one interface element such as an icon of an application program and/or a widget desktop control. The application program interface display area can also be an empty interface without any content. The commonly used control display area is used to display controls with a high usage rate, for example, application icons such as setting buttons, interface numbers, scroll bars, and phonebook icons.

Wherein the processor 660 is the control center of the mobile terminal 600, utilizes various interfaces and lines to connect the various parts of the whole mobile phone, by running or executing the software programs and/or modules stored in the first memory 621, and calling the software programs and/or modules stored in the second memory 621. The data in the memory 622 executes various functions of the mobile terminal 600 and processes data, so as to monitor the mobile terminal 600 as a whole. Optionally, the processor 660 may include one or more processing units.

In the embodiment of the present invention, by calling the software programs and/or modules stored in the first memory 621 and/or the data in the second memory 622, the processor 660 is configured to When commanding panoramic photography, control the camera to continuously capture at least one frame of images; record the orientation information of the mobile terminal when capturing each frame of images in the at least one frame of images during the continuous capture of images by the camera; Image splicing is performed on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information.

Optionally, the processor 660 is further configured to: after the step of stitching the at least one frame of images to generate a photo, receive an instruction for viewing the panoramic photo input by the user of the mobile terminal;

Obtain the screen type of the display screen of the mobile terminal;

Based on the screen type, a display manner of the panoramic photo is determined, and the panoramic photo is displayed according to the display manner.

Optionally, the processor 660 is further configured to: if the screen type is a flexible screen, determine the curvature of each screen area of the flexible screen based on the orientation information carried in each frame of the at least one image frame degree;

controlling the flexible screen to bend according to the bending degree of each screen area;

The panoramic photo is displayed on the curved flexible screen.

Optionally, the processor 660 is further configured to: if the screen type is a plane non-deformable screen, based on the orientation information carried by each frame image in the at least one frame image, determine the Diorama;

The panorama is displayed on the diorama.

Optionally, the processor 660 is further configured to: record the depth-of-field information of each frame of the at least one frame of images during the process of the camera continuously capturing images.

Optionally, the processor 660 is further configured to: if the screen type is a flexible screen, based on the depth of field information of each frame of image in the at least one frame of image, determine the image corresponding to the depth of field information in the panoramic photo Hierarchical information of the region;

In the process of displaying the panoramic photo, the flexible screen is controlled to deform based on the layer information.

Optionally, the processor 660 is further configured to: receive a click operation of the mobile terminal user on the panoramic photo during the process of displaying the panoramic photo;

If the image area where the click operation is located includes the depth of field information, then based on the depth of field information, perform depth of field effect processing corresponding to the depth of field information on the image area where the click operation is located.

Optionally, the processor 660 is further configured to: the orientation information includes direction information and displacement information.

It can be seen that, according to the embodiment of the present invention, during the panorama photographing process, the camera is used to continuously collect at least one frame of images, and at the same time record the orientation information of the mobile terminal when capturing each frame of images, and stitch the at least one frame of images to generate a panorama Each frame of image in the panoramic photo carries corresponding orientation information. In the panoramic photos obtained in the embodiment of the present invention, each frame of image carries corresponding orientation information, so when displaying the generated panoramic photos, the display mode of the panoramic photos can be determined according to the orientation information carried by each frame of images, and then the panoramic photos can be controlled. The curved display effect of the image makes the displayed panoramic image more in line with the actual scene, and overcomes the problem of image bending and distortion.

Since the embodiment of the mobile terminal basically corresponds to the method embodiment shown in the above-mentioned Figures 1-2, the details in the description of this embodiment can be referred to the relevant descriptions in the above-mentioned embodiments, and will not be repeated here. up.

Those of ordinary skill in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (16)

1. A panoramic photographing method, applied to a mobile terminal with a camera, characterized in that the method comprises: When receiving a panoramic photographing instruction input by the user of the mobile terminal, controlling the camera to continuously capture at least one frame of images; During the process of the camera continuously capturing images, recording the orientation information of the mobile terminal when capturing each frame of the at least one frame of images; performing image splicing on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information; Wherein, after the step of performing image splicing on the at least one frame of images to generate a photo, the method further includes: receiving an instruction to view the panoramic photo input by the user of the mobile terminal; Obtain the screen type of the display screen of the mobile terminal; Based on the screen type, determine the display mode of the panoramic photo, and display the panoramic photo according to the display mode; The step of determining the display mode of the panoramic photo based on the screen type, and displaying the panoramic photo according to the display mode includes: If the screen type is a flexible screen, then determine the degree of curvature of each screen area of the flexible screen based on the orientation information carried in each frame of the at least one image; controlling the flexible screen to bend according to the bending degree of each screen area; The panoramic photo is displayed on the curved flexible screen. 2. The method according to claim 1, wherein the step of determining the display mode of the panoramic photo based on the screen type, and displaying the panoramic photo according to the display mode includes: If the screen type is a plane non-deformable screen, then based on the orientation information carried by each frame of the at least one frame of image, determine a stereo model for displaying the panoramic image; The panorama is displayed on the diorama. 3. The method according to claim 1, characterized in that the method further comprises: In the process of continuously collecting images by the camera, the field depth information of each frame of the at least one frame of images is recorded. 4. The method according to claim 3, characterized in that, in the process of displaying the panoramic photo, the method further comprises: receiving the click operation of the mobile terminal user on the panoramic photo; If the image area where the click operation is located includes the depth of field information, then based on the depth of field information, perform depth of field effect processing corresponding to the depth of field information on the image area where the click operation is located. 5. The method according to claim 1, wherein the orientation information includes direction information and displacement information. 6. A panoramic photographing method, applied to a mobile terminal with a camera, characterized in that the method comprises: When receiving a panoramic photographing instruction input by the user of the mobile terminal, controlling the camera to continuously capture at least one frame of images; During the process of the camera continuously capturing images, recording the orientation information of the mobile terminal when capturing each frame of the at least one frame of images; performing image splicing on the at least one frame of images to generate a panoramic photo, and each frame of image in the panoramic photo carries corresponding orientation information; After the step of performing image splicing on the at least one frame of images to generate a photo, the method further includes: receiving an instruction to view the panoramic photo input by the user of the mobile terminal; Obtain the screen type of the display screen of the mobile terminal; Based on the screen type, determine the display mode of the panoramic photo, and display the panoramic photo according to the display mode; In the process of continuous image acquisition by the camera, record the depth of field information of each frame of image in the at least one frame of image; The step of determining the display mode of the panoramic photo based on the screen type, and displaying the panoramic photo according to the display mode includes: If the screen type is a flexible screen, then based on the depth of field information of each frame of image in the at least one frame of image, determine the level information of the image area corresponding to the depth of field information in the panoramic photo; In the process of displaying the panoramic photo, the flexible screen is controlled to deform based on the layer information. 7. The method according to claim 6, characterized in that, in the process of displaying the panoramic photo, the method further comprises: receiving the click operation of the mobile terminal user on the panoramic photo; If the image area where the click operation is located includes the depth of field information, then based on the depth of field information, perform depth of field effect processing corresponding to the depth of field information on the image area where the click operation is located. 8. The method according to claim 6, wherein the orientation information includes direction information and displacement information. 9. A kind of mobile terminal, described mobile terminal has camera, is characterized in that, comprises: An image acquisition module, configured to control the camera to continuously acquire at least one frame of images when a panoramic photographing instruction input by the user of the mobile terminal is received; An azimuth information recording module, configured to record the azimuth information of the mobile terminal when each frame of at least one frame of images collected by the image acquisition module is captured during the process of the image acquisition module controlling the camera to continuously acquire images; A panoramic photo generation module, configured to stitch the at least one frame of images collected by the image acquisition module to generate a panoramic photo, where each frame of the panoramic photo carries the corresponding information recorded by the orientation information recording module. orientation information; A viewing instruction receiving module, configured to receive a viewing instruction for the panoramic photo input by the mobile terminal user after the step of stitching the at least one frame of images by the panoramic photo stitching module to generate a photo; a screen type acquiring module, configured to acquire the screen type of the display screen of the mobile terminal; A panoramic photo display module, configured to determine the display mode of the panoramic photo based on the screen type acquired by the screen type acquisition module, and display the panoramic photo according to the display mode; The bending degree determination sub-module is configured to determine each screen of the flexible screen based on the orientation information carried by each frame of the image in the at least one frame of images if the screen type acquired by the screen type acquisition module is a flexible screen degree of curvature of the area; a bending control submodule, configured to control the flexible screen to bend according to the bending degree of each screen area determined by the bending degree determining submodule; The first panoramic photo display submodule is configured to display the panoramic photo on the curved flexible screen controlled by the bending control submodule. 10. The mobile terminal according to claim 9, wherein the panoramic photo display module comprises: A three-dimensional model determination submodule, configured to determine, based on the orientation information carried in each frame of the at least one frame of images, the screen used to display the Stereo models of panoramic images; The second panoramic photo display submodule is configured to display the panoramic photo on the stereo model determined by the stereo model determination submodule. 11. The mobile terminal according to claim 9, further comprising: The depth-of-field information recording module is configured to record the depth-of-field information of each frame of the at least one frame of image during the process of the image acquisition module controlling the camera to continuously acquire images. 12. The mobile terminal according to claim 11, further comprising: The click operation receiving module is used to receive the mobile terminal user's input on the panoramic photo during the process of displaying the panoramic photo after the panoramic photo display module displays the panoramic photo according to the display method. click operation; A depth-of-field effect processing execution module, configured to, if the image area where the click operation is received by the click operation receiving module includes the depth-of-field information, perform an operation corresponding to the depth-of-field information on the image area where the click operation is located based on the depth-of-field information. Depth of field effect processing. 13. The mobile terminal according to claim 9, wherein the orientation information includes direction information and displacement information. 14. A kind of mobile terminal, described mobile terminal has camera, is characterized in that, comprises: An image acquisition module, configured to control the camera to continuously acquire at least one frame of images when a panoramic photographing instruction input by the user of the mobile terminal is received; An azimuth information recording module, configured to record the azimuth information of the mobile terminal when each frame of at least one frame of images collected by the image acquisition module is captured during the process of the image acquisition module controlling the camera to continuously acquire images; A panoramic photo generation module, configured to stitch the at least one frame of images collected by the image acquisition module to generate a panoramic photo, and each frame of the panoramic photo carries the corresponding information recorded by the orientation information recording module. orientation information; A viewing instruction receiving module, configured to receive a viewing instruction for the panoramic photo input by the mobile terminal user after the step of stitching the at least one frame of images by the panoramic photo stitching module to generate a photo; a screen type acquiring module, configured to acquire the screen type of the display screen of the mobile terminal; A panoramic photo display module, configured to determine the display mode of the panoramic photo based on the screen type acquired by the screen type acquisition module, and display the panoramic photo according to the display mode; A depth of field information recording module, configured to record the depth of field information of each frame of image in the at least one frame of image during the process of the image acquisition module controlling the camera to continuously acquire images; The panoramic photo display module includes: The layer information confirmation sub-module is used to determine the depth of field information in the panoramic photo based on the depth of field information of each frame of the image in the at least one frame of image if the screen type obtained by the screen type acquisition module is a flexible screen Hierarchy information of the corresponding image area; The deformation control sub-module is configured to control the flexible screen to deform based on the layer information confirmed by the layer information confirmation sub-module during the process of displaying the panoramic photo. 15. The mobile terminal according to claim 14, further comprising: The click operation receiving module is used to receive the mobile terminal user's input on the panoramic photo during the process of displaying the panoramic photo after the panoramic photo display module displays the panoramic photo according to the display method. click operation; A depth-of-field effect processing execution module, configured to, if the image area where the click operation is received by the click operation receiving module contains the depth-of-field information, then based on the depth-of-field information, perform an operation corresponding to the depth-of-field information on the image area where the click operation is located. Depth of field effect processing. 16. The mobile terminal according to claim 14, wherein the orientation information includes direction information and displacement information.

Images