KR20200016702A - Method for generating a spin image and apparatus thereof - Google Patents

Abstract

The present invention relates to a spin image generation method and an apparatus thereof. The spin image generation method comprises the steps of: displaying a preview screen showing an image inputted through a camera in advance; receiving video frames relating to a specific subject from the camera when a photographing button is selected; tracking a subject existing in each image frame by using a predetermined tracking algorithm; cropping at least a part of each image frame according to a position of the subject; and stabilizing the cropped image frames.

Description

Spin image generation method and apparatus therefor {METHOD FOR GENERATING A SPIN IMAGE AND APPARATUS THEREOF}

The present invention relates to a method and apparatus for generating a video file, and more particularly, to a spin image generation method capable of generating a video file of a corresponding subject based on image frames photographed from a specific subject in a 360 degree direction. To the device.

Terminals may be divided into mobile / portable terminals and stationary terminals according to their mobility. The mobile terminal may be further classified into a handheld terminal and a mounted terminal according to whether a user can directly carry it.

The functions of mobile terminals are diversifying. For example, data and voice communication, taking a picture and video with a camera, recording a voice, playing a music file through a speaker system, and outputting an image or video to a display unit. Some terminals add an electronic game play function or perform a multimedia player function.

As such functions are diversified, for example, as a terminal, a multimedia player has a complex function such as taking a picture or a video, playing a music or video file, playing a game, or receiving a broadcast. Is implemented.

In the mobile terminal implemented in the form of such a multimedia device, new attempts have been variously applied in terms of hardware or software to implement complex functions. For example, with the development of image sensor technology and image processing technology, recent mobile terminals provide various additional functions along with a photographing function that approximates the performance of a DSLR camera.

In addition, due to the simplicity of portability and manipulation, more and more users are taking pictures or videos through mobile terminals. Accordingly, the mobile terminal provides various applications for easily taking a picture or a video.

Meanwhile, in recent years, as the on-line commodity trading market increases, the needs of users who want to photograph the appearance of a specific commodity in a 360 degree direction is increasing. In order to capture the merchandise to be displayed on the online sales site, existing operators would place the merchandise on a table that rotates 360 degrees, install a DSLR camera on a tripod, and then rotate the table and rotate the product in a 360 degree orientation with the DSLR camera The photographing method was mainly used. However, since this method requires the provision of separate equipment every time the product is photographed, it is difficult for the general seller to photograph the product using the method. Therefore, there is a need for a method for easily photographing goods in a 360 degree direction using a mobile terminal that is always carried by anyone.

The present invention aims to solve the above-mentioned problem and other problems. Another object of the present invention is to provide a spin image generating method and apparatus for automatically generating a moving image file of a corresponding subject based on image frames photographed while moving around a specific subject in a 360 degree direction.

Another object of the present invention is to provide a spin image generation method and apparatus for providing photographing guide information for guiding a photographing operation of a terminal user in real time when photographing a specific subject.

Another object of the present invention is to provide a spin image generation method capable of tracking a position of a subject present in each image frame when photographing a specific subject, and performing shake correction by cropping each image frame according to the position of the subject; In providing the device.

According to an aspect of the present invention to achieve the above or another object, displaying a preview screen for previewing the image input through the camera; Receiving image frames regarding a specific subject from the camera when a photographing button is selected; Tracking a subject present in each image frame by using a predetermined tracking algorithm; Cropping at least a portion of each image frame according to the position of the subject; And stabilizing the cropped image frames.

According to another aspect of the invention, the image frame receiving unit for receiving image frames for a particular subject from one or more cameras; A subject tracking unit which tracks a subject existing in each image frame by using a predetermined tracking algorithm; An image cropping unit that crops at least a portion of each image frame according to the position of the subject; And a shake correction unit for stabilizing the cropped image frames.

According to another aspect of the invention, the step of displaying a preview screen for previewing the image input through the camera; Receiving image frames regarding a specific subject from the camera when a photographing button is selected; Tracking a subject present in each image frame by using a predetermined tracking algorithm; Cropping at least a portion of each image frame according to the position of the subject; And a computer program stored in a computer-readable recording medium for executing the stabilization of the cropped image frames on a computer.

A spin image generating method and effects of the device according to embodiments of the present invention will be described below.

According to at least one of the embodiments of the present invention, by automatically generating a moving image file of the subject based on the image frames taken while moving around the particular subject in a 360-degree direction, to provide user convenience different from the prior art The advantage is that you can.

According to at least one of the embodiments of the present invention, when photographing a particular subject, by providing photographing guide information for guiding a photographing operation of the terminal user in real time, the terminal user can easily photograph the subject in the 360-degree direction There is an advantage in that the shaking of the camera image due to the movement of the terminal user can be minimized.

According to at least one of the embodiments of the present disclosure, when photographing a specific subject, the subject is moved by tracking the position of the subject present in each image frame and cropping the image by cropping each image frame according to the position of the subject. It has the advantage that it can be located in the center of the screen and increase the image processing speed for image stabilization.

However, the spin image generating method and the effect that the device can achieve according to the embodiments of the present invention is not limited to those mentioned above, and other effects not mentioned are described in the technical field to which the present invention belongs. It will be clearly understood by those skilled in the art.

1 is a diagram showing the configuration of a service providing system according to an embodiment of the present invention;
2 is a diagram illustrating a configuration of a mobile terminal according to one embodiment of the present invention;
3 is a diagram illustrating a configuration of a spin image generating unit according to an embodiment of the present invention;
4 is a diagram showing the type of optical flow method;
5 is a flowchart illustrating an operation of a mobile terminal according to an embodiment of the present invention;
6 is a flowchart illustrating an operation of a mobile terminal according to another embodiment of the present invention;
7 is a diagram referred to explain the operation of a mobile terminal displaying a home screen of a smart video application;
8 is a diagram referred to for explaining an operation of a mobile terminal for deleting a spin image;
9 is a diagram referred to explain the operation of a mobile terminal for inserting tag information into a spin image;
10 and 11 are diagrams for explaining the operation of a mobile terminal for reproducing a spin image;
FIG. 12 is a diagram for explaining an operation of a mobile terminal entering a camera mode according to an embodiment of the present disclosure; FIG.
FIG. 13 is a diagram for describing an operation of a mobile terminal entering a camera mode according to another embodiment of the present disclosure; FIG.
14 is a diagram referred to explain an operation of a mobile terminal displaying an indicator for designating a subject;
15 is a diagram referred to explain the operation of a mobile terminal for providing shooting guide information;
16 and 17 are diagrams for explaining the operation of the mobile terminal for reproducing the most recently generated spin image;
18 and 19 are referenced to explain the operation of a mobile terminal displaying a list of spin images;
20 is a diagram referred to for describing an operation of a mobile terminal sharing a spin image;
21 is a diagram referred to for explaining the operation of a mobile terminal for editing a spin image;
FIG. 22 is a diagram referred to describe an operation of a mobile terminal for providing object information for augmented reality; FIG.
FIG. 23 is a diagram referred to explain the operation of a mobile terminal capable of replacing a background of a spin image; FIG.
FIG. 24 is a diagram referred to for describing an operation of a mobile terminal capable of generating a panoramic image. FIG.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles. In other words, the term 'part' used in the present invention refers to a hardware component such as software, FPGA or ASIC, and 'part' plays a role. But wealth is not limited to software or hardware. The 'unit' may be configured to be in an addressable storage medium and may be configured to play one or more processors. Thus, as an example, a 'part' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are intended to facilitate understanding of the embodiments disclosed herein, but are not limited to the technical spirit disclosed herein by the accompanying drawings, all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

The present invention proposes a spin image generation method capable of automatically generating a moving image file of a corresponding subject based on image frames photographed while moving around a specific subject in a 360 degree direction. In addition, the present invention proposes a spin image generation method capable of providing in real time shooting guide information for guiding a photographing operation of a terminal user when photographing a specific subject. In addition, the present invention, when photographing a particular subject, track image position of the subject present in each image frame, crop image (crop) according to the position of the subject to perform a shake image generation method that can perform a shake correction Suggest. As described herein, a 'spin image' refers to a moving image or a still image or an image stream in which a terminal user photographs a subject while moving around a specific subject in a 360 degree direction. image stream).

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a service providing system according to an exemplary embodiment.

Referring to FIG. 1, the service providing system 10 according to an exemplary embodiment may include a communication network 100, a mobile terminal 200, a server 150, and the like.

The mobile terminal 200 and the server 150 may be connected to each other through the communication network 100. The communication network 100 may include a wired network and a wireless network, and specifically, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or the like. It may include various networks. The communication network 100 may also include a known World Wide Web (WWW). However, the communication network 100 according to the present invention is not limited to the networks listed above, and may include at least one of a known wireless data network, a known telephone network, and a known wired / wireless television network.

The mobile terminal 200 may provide various services based on the information provided from the server 150. For example, when the server 150 is a web server, the mobile terminal 200 may provide a web service based on the content provided from the server 150.

In the present embodiment, the mobile terminal 200 may generate a spin image related to a specific subject or may output (play) the spin image to the display unit (hereinafter referred to as a smart video application) for convenience of description. A program (hereinafter referred to as a "smart video program" for convenience of explanation) can be installed. In this case, the mobile terminal 200 may download a smart video application by accessing an app store or a play store, or may download the application through a separate storage medium. In addition, the mobile terminal 200 may download a smart video program provided by the server 150, or may download the program through a separate storage medium.

The mobile terminal 200 may execute a preinstalled smart video application or program in response to a predetermined user input. When the corresponding application or program is executed, the mobile terminal 200 may display a predefined user interface (UI) on the display unit. When a user input is received through the user interface, the mobile terminal 200 may perform a function of generating a spin image of a specific subject or outputting the spin image to the display unit according to the user input.

The mobile terminal 200 described herein includes a mobile phone, a smart phone, a laptop computer, a slate PC, a tablet PC, an ultrabook. , Digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), wearable devices (e.g., smartwatches, glass glasses, head mounted displays) ) May be included.

The server 150 may provide a service requested by the mobile terminal 200 as a service providing server (or service provider). For example, when the server 150 is a web server, the server 150 may provide a content requested by the terminal user in the form of a web page to the mobile terminal 200.

In the present embodiment, the server 150 may generate a smart video application or a program capable of generating and outputting a spin image of a specific subject and providing the same to the mobile terminal 200. The mobile terminal 200 may provide a function of generating and outputting a spin image of a specific subject by installing an application or a program received from the server 150.

On the other hand, in another embodiment, the server 150 may receive image frames of a specific subject from the mobile terminal 200 in real time to generate a spin image, and provide the generated spin image to the mobile terminal 200. have. The mobile terminal 200 may store the spin image provided from the server 150 and output the spin image to the display unit.

2 is a block diagram illustrating a configuration of a mobile terminal 200 according to an embodiment of the present invention.

Referring to FIG. 2, the mobile terminal 200 according to an embodiment of the present invention may include a communication unit 210, a sensing unit 220, an input unit 230, an output unit 240, a memory 250, and a controller 260. ) May be included. The components shown in FIG. 2 are not essential to implementing a mobile terminal, so the mobile terminal described herein may have more or fewer components than those listed above.

The communication unit 210 may include a wired communication module for supporting a wired network and a wireless communication module for supporting a wireless network. The wired communication module is provided on a wired communication network established according to technical standards or communication methods (for example, Ethernet, Power Line Communication, Home PNA, IEEE 1394, etc.) for wired communication. Send and receive wired signals with at least one of an external server and another terminal. The wireless communication module may include technical standards or communication schemes for wireless communication (eg, wireless LAN (WLAN), wireless-fidelity (Wi-Fi), digital living network alliance (DLNA), and global system for mobile communication). ), At least one of a base station, an access point, and a repeater in a wireless communication network established according to Code Division Multi Access (CDMA), Wideband CDMA (WCDMA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the like. Send and receive wireless signals with one.

In the present embodiment, the communication unit 210 may exchange data related to generation and reproduction of a spin image regarding a specific subject with the server 150. For example, the communication unit 210 may receive a smart video application or program from the server 150. In addition, the communication unit 210 may transmit image data photographed through the mobile terminal 200 to the server 150.

The sensing unit 220 may include one or more sensors for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 220 may include a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor G-. sensor, gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor), fingerprint scan sensor, ultrasonic sensor, optical sensor ( optical sensors, battery gauges, environmental sensors (eg, barometers, hygrometers, thermometers, radiation sensors, thermal sensors, gas sensors, etc.). Meanwhile, the mobile terminal disclosed herein may utilize a combination of information sensed by at least two or more of these sensors.

The sensing unit 220 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. The controller 260 may control driving or operation of the mobile terminal 200 or perform data processing, function or operation related to an application program installed in the mobile terminal 200 based on the sensing signal. In the present embodiment, the controller 260 may detect a moving speed, a moving direction, a posture, a shaking degree, etc. of the mobile terminal 200 based on sensing data measured by an acceleration sensor, a gyroscope sensor, a motion sensor, or the like.

The input unit 230 may include a camera for inputting an image signal, a microphone for inputting an audio signal, a user input unit for receiving information from a user (eg, a keyboard, a mouse, a touch key, a push key) mechanical keys), and the like. The voice data or the image data collected by the input unit 230 may be analyzed and processed as a control command of the user.

The camera processes an image frame such as a still image or a video obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit or stored in the memory 250. In the present embodiment, the processed image frame may be input to the spin image generator 265. Meanwhile, a plurality of cameras provided in the mobile terminal 200 may be arranged to form a matrix structure. A plurality of image information having various angles or focuss are input to the mobile terminal 200 through the camera having the matrix structure. Can be.

The microphone processes external sound signals into electrical voice data. The processed voice data may be variously used according to a function (or an application program being executed) performed by the mobile terminal 200.

The user input unit is for receiving information from a user. When information is input through the user input unit, the controller 260 may control an operation of the mobile terminal 200 to correspond to the input information. The user input unit may include a mechanical input means (for example, a button, a dome switch, a jog wheel, a jog switch, etc.) located at the front, rear, or side of the mobile terminal 200, and a touch input. Means may be included.

The output unit 240 is for generating an output related to visual, auditory or tactile, and may include at least one of a display unit, an audio output unit, a hap tip module, and an optical output unit.

The display unit displays (outputs) information processed by the mobile terminal 200. For example, the display unit may display execution screen information of an application program driven in the mobile terminal 200 or user interface (UI) and graphical user interface (GUI) information according to the execution screen information. In the present embodiment, the display unit may display the UI information according to the execution of the smart video application or program. In addition, the display unit may display shooting guide information for guiding a shooting operation of the terminal user.

The display unit may form a layer structure or an integrated structure with the touch sensor, thereby implementing a touch screen. Such a touch screen may function as a user input unit that provides an input interface between the terminal 200 and the user, and may provide an output interface between the terminal 200 and the user.

The sound output unit may output audio data received from the communication unit 210 or stored in the memory 250 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit may output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the mobile terminal 200. In the present embodiment, the sound output unit may output photographing guide information for guiding a correct photographing operation of the terminal user as a sound signal.

The haptic module generates various haptic effects that a user can feel. A representative example of the haptic effect generated by the haptic module may be vibration. The light output unit outputs a signal for notifying occurrence of an event by using light of a light source of the mobile terminal 200. Examples of events generated in the mobile terminal 200 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The memory 250 stores data supporting various functions of the mobile terminal 200. The memory 250 may store a plurality of application programs or applications that are driven by the mobile terminal 200, data for operating the mobile terminal 200, and instructions. In the present embodiment, the memory 250 may store a smart video application or a program for generating a spin image regarding a specific subject. In addition, the memory 250 may store photo or video files taken by the mobile terminal 200 and video files generated through a smart video application or program.

The memory 250 may be a flash memory type, a hard disk type, a solid state disk type, an SSD type, a silicon disk drive type, or a multimedia card micro type. ), Card type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read It may include at least one type of storage medium of -only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk and optical disk.

The controller 260 controls an operation related to an application program stored in the memory 250 and generally the overall operation of the mobile terminal 200. Further, in order to implement various embodiments described below on the mobile terminal 200 according to the present invention, the controller 260 may combine and control at least one of the above-described components.

In the present embodiment, the controller 260 may include a spin image generator 265 for generating and outputting a spin image of a specific subject. The spin image generator 265 may drive a smart video application or program stored in the memory 250 to generate a spin image of a specific subject or provide a function of outputting (playing) the spin image.

For example, the spin image generator 265 may automatically generate a video file of the subject based on the image frames photographed while moving around the specific subject in a 360 degree direction. In addition, the spin image generator 265 may provide shooting guide information for guiding a shooting operation of the terminal user in real time when capturing a specific subject. In addition, the spin image generator 265 tracks the positions of the subjects present in the respective image frames when capturing a specific subject, crops each image frame according to the position of the subject, and crops the cropped image frame. For each of these, shake compensation can be performed.

3 is a diagram illustrating a configuration of a spin image generator according to an exemplary embodiment.

Referring to FIG. 3, the spin image generator 300 according to an embodiment of the present invention may include an image frame receiver 310, a data analyzer 320, a guide information provider 330, and a subject tracker 340. The frame cropping unit 350 may include a shake correction unit 360, a file generator 370, and a file reproducing unit 380. The components shown in FIG. 3 are not essential to implementing the spin image generator, so the spin image generator described herein may have more or fewer components than those listed above.

The image frame receiving unit 310 may perform a function of receiving image frames captured by the terminal user while moving around the specific subject in a 360 degree direction from the camera. In another embodiment, the image frame receiver 310 may receive from the memory 250 in which image frames of a specific subject are stored.

The data analyzer 320 may detect distance information between the mobile terminal 200 and the subject by analyzing a size of a subject present in each image frame. Meanwhile, as another example, the data analyzer 320 may detect distance information between the mobile terminal 200 and the subject by using a distance sensor such as an ultrasonic sensor or an infrared sensor.

The data analyzer 320 may analyze location information of a subject existing in each image frame and detect direction information of a camera photographing the subject. In addition, the data analyzer 320 may detect sensing data measured by a gyroscope sensor, an acceleration sensor, a motion sensor, and the like to detect information about a moving speed, a moving direction, a posture, and a shaking degree of the mobile terminal 200. . The information detected through the data analyzer 320 may be used to generate photographing guide information for guiding a photographing operation of the terminal user.

The guide information provider 330 may generate photographing guide information for guiding a photographing operation of the terminal user, and may output the photographing guide information as a visual / acoustic / tactile signal.

When the terminal user shoots while moving around a specific subject in a 360 degree direction with a camera mounted on the mobile terminal 200, the shooting distance between the mobile terminal and the subject, the direction of the camera photographing the subject, the moving speed of the mobile terminal, The posture of the mobile terminal and the like continuously change. Therefore, it is necessary to maintain the shooting distance between the terminal user and the subject and the direction of the camera photographing the subject, and to provide shooting guide information for preventing camera shake due to the movement of the hand or arm.

The guide information provider 330 may analyze the photographing state or the photographing operation of the terminal user in real time based on the information received from the data analyzer 320, and provide photographing guide information accordingly. In this case, the photographing guide information may include notification information for guiding a photographing operation of the terminal user, and alarm information for correcting an incorrect photographing operation of the terminal user.

The subject tracking unit 340 may perform a function of tracking a subject existing in each image frame by using a predetermined object tracking algorithm. The method of tracking a subject may include a color based method, an area based method, an active contour based method, and a feature based method according to selection of feature values of a subject to be tracked. feature based method).

The subject tracker 340 includes a Mean-Shift algorithm, a Continuously Adaptive Mean-Shift (CAMshift) algorithm, a Difference Image algorithm, a Kalman filter algorithm, a particle filter algorithm, a Gaussian Mixture Model (GMM) algorithm, The position of a subject present in each image frame may be tracked using at least one of a Kernelized Correlation Filter (KCF) algorithm and a histogram of oriented gradients (HOG).

The subject tracking unit 340 may apply different tracking algorithms according to the type of the subject. Meanwhile, in the present embodiment, an example of tracking a position of a subject existing in each image frame in real time using a Kernelized Correlation Filter (KCF) algorithm and a Felzenszwalb-Histogram of Gradient (FHOG) algorithm is illustrated, but is not necessarily limited thereto. Do not.

When the initial position of the subject is designated by the terminal user, the subject tracking unit 340 may track the position of the subject existing in each image frame based on the designated initial position.

The frame cropping unit 350 may crop a portion of each image frame according to the position of the subject. That is, the frame cropping unit 350 may cut out the background of each image frame so that the subject is located in the center area of the video screen.

The frame cropping unit 350 may store the cropped image frames in an image data queue. Here, unlike the image data stack, the image data queue is a data structure (FIFO, First In First Out) in which data entered first comes first.

The frame cropping unit 350 may enlarge the cropped image frame to a predetermined size. This process is to provide the terminal user with the effect of emphasizing the subject. However, the process may be configured to be omitted according to an embodiment.

The shake correction unit 360 may perform a function of correcting a shake of the camera image due to hand shake of the terminal user by using a predetermined image stabilization method. As the image stabilization technique, a motion based stabilization technique may be used.

The motion-based stabilization technique is a method of measuring an amount of movement between adjacent frames and compensating an image by using information about the amount of movement. The motion-based stabilization technique may also be called an optical flow method.

As shown in FIG. 4, the optical flow method may be classified into a dense optical flow approach and a sparse optical flow approach. The dense optical flow method is useful for calculating an entire image frame and measuring an accurate optical flow, but there is a problem that it is difficult to apply to a mobile environment due to a large amount of calculation. On the other hand, since the rare optical flow method calculates only the feature points in the image frame, accurate optical flow cannot be measured. Therefore, in the present embodiment, a rare optical flow method may be used as the image stabilization technique, but is not necessarily limited thereto. The shake correction unit 360 may apply different image stabilization techniques according to the type of the subject.

The shake correction unit 360 may extract feature points of the subject from image frames stored in the image data queue. The shake correction unit 360 may detect a motion vector of the subject based on the feature point position information between the image frames, and calculate a trajectory of the subject based on the motion vector.

The shake correction unit 360 may smooth the trajectory of the subject by using a trajectory smoothing technique. For example, the shake correction unit 360 may soften the trajectory of the subject by using an average window.

The shake correction unit 360 may correct shake of image frames by using an affine transform. That is, the shake correction unit 360 may perform image warping by applying an affine transform to cropped image frames.

Through these processes, the shake correction unit 360 may generate stabilized image frames. Since the shake correction unit 360 performs the shake correction process only on the cropped image frame instead of the entire image frame, the shake correction unit 360 not only reduces the amount of calculation necessary for image processing, but also reduces the calculation time required for image processing. Can be.

The file generator 370 may perform a function of generating frames, which are image processed according to an image stabilization method, as a video file and storing them in the memory 250. In this case, the file generator 370 may store image frames in a YUV format. In addition, the file generator 370 may store the image frames in the form of a video file such as an AVI file, an MP4 file, an MKV file, a SKM file, or the like.

The file player 380 may perform a function of outputting (playing) a video file stored in the memory 250 to the display unit. The file player 380 may play a video file in the form of an image slide.

The file player 380 may play a video after a certain point in time or play a video before a certain point in time according to a user's gesture input. Accordingly, the terminal user can check the appearance of the subject in detail through the gesture input having the directivity.

As described above, the spin image generating unit according to the present invention may generate a moving image file of the subject based on the image frames photographed while moving around the specific subject in a 360 degree direction. The spin image generator 300 may be implemented in the server 150 as well as the mobile terminal 200.

5 is a flowchart illustrating an operation of a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 5, the mobile terminal 200 may execute a smart video application according to a user command (S510).

When the corresponding application is executed, the mobile terminal 200 may display a predefined user interface (UI) on the display unit (S520). In this case, the user interface may include a camera menu for generating a spin image, a home menu for displaying a list of spin images, a setting menu for setting an environment of a corresponding application, and the like.

When the camera menu is selected by the terminal user (S530), the mobile terminal 200 may display a preview screen for previewing an image input through the camera (S540). In this case, the preview screen may include a first capturing region positioned in the center region of the screen and a second capturing region positioned in the edge region of the screen. The second photographing area may be dimd to distinguish it from the first photographing area.

The first photographing area is an area corresponding to an image frame cropped for image processing (ie, image stabilization) and corresponds to an area recorded as a moving image file. Accordingly, the terminal user may adjust the direction of the camera so that the subject is located in the first photographing area. The size of the first photographing area may be configured to be automatically set by the mobile terminal 200 or manually by a terminal user.

The mobile terminal 200 may display an indicator for designating a subject on the display unit. The terminal user can accurately specify the subject by adjusting the position and size of the indicator. Meanwhile, in another embodiment, the mobile terminal 200 may automatically detect a subject through object recognition and display an indicator on the detected subject.

When the photographing button is selected by the terminal user (S550), the mobile terminal 200 may start capturing a moving image of the subject using one or more cameras (S560). The mobile terminal 200 may receive image frames regarding a subject from one or more cameras.

The mobile terminal 200 may analyze (monitor) the shooting state or the shooting operation of the terminal user in real time by analyzing the image data or the sensing data (S570).

For example, the mobile terminal 200 may detect distance information between the mobile terminal 200 and the subject by using size information of the subject or by using a distance sensor. In addition, the mobile terminal 200 may detect location information of a subject present in each image frame and detect direction information of a camera photographing the subject. In addition, the mobile terminal 200 may analyze the sensing data measured by the gyroscope sensor, the acceleration sensor, the motion sensor, and the like to detect information about the moving speed, the moving direction, the attitude, the shaking degree, and the like of the mobile terminal 200. The mobile terminal 200 may analyze the photographing state or the photographing operation of the terminal user in real time using the detected information.

The mobile terminal 200 may generate shooting guide information according to a shooting state or a shooting operation of the terminal user (S580). In this case, the photographing guide information may include notification information for guiding a photographing operation of the terminal user, and alarm information for correcting an erroneous photographing operation of the terminal user.

The mobile terminal 200 may output photographing guide information as at least one of a visual signal, an audio signal, and a tactile signal (S590). The terminal user can easily photograph the subject while moving around the subject in a 360 degree direction according to the photographing guide information output from the mobile terminal 200.

When the direction of the camera is rotated due to the carelessness of the terminal user, and the subject moves out of the first photographing area, the mobile terminal 200 may automatically stop capturing a video regarding the subject. When the subject comes back into the first photographing area, the mobile terminal 200 may automatically resume capturing a moving image of the subject.

The mobile terminal 200 may repeatedly perform the operations of steps 570 to 590 described above until the smart video application is terminated (S595). The above-described operations may be implemented by a spin image generating apparatus provided in the mobile terminal 200.

As described above, the mobile terminal 200 may provide photographing guide information for guiding a photographing operation of the terminal user in real time in the spin image photographing mode.

6 is a flowchart illustrating an operation of a mobile terminal according to another embodiment of the present invention.

Referring to FIG. 6, the mobile terminal 200 may execute a smart video application according to a user command or the like (S610).

When the application is executed, the mobile terminal 200 may display a predefined user interface (UI) on the display unit (S620). When the camera menu is selected by the terminal user, the mobile terminal 200 may display a preview image on the display unit, which previews an image input through the camera.

The mobile terminal 200 may display an indicator for designating a subject on the display unit. The terminal user can accurately specify the subject by adjusting the position and size of the indicator.

When the photographing button is selected by the terminal user (S630), the mobile terminal 200 may photograph a video regarding the subject using at least one camera (S640). The mobile terminal 200 may receive image frames regarding a subject from one or more cameras.

The mobile terminal 200 may track the position of the subject existing in each image frame using a predetermined object tracking algorithm (S650). In the present embodiment, the mobile terminal 200 may track the position of a subject existing in each image frame using a Kernelized Correlation Filter (KCF) algorithm and a Felzenszwalb-Histogram of Gradient (FHOG) algorithm. In addition, the mobile terminal 200 may track the subject based on the initial position of the subject designated through the indicator.

The mobile terminal 200 may crop a partial region of each image frame according to the position of the subject (S660). That is, the mobile terminal 200 may cut out the background of each image frame so that the subject is positioned at the center of the video playback screen.

The mobile terminal 200 may correct a camera image shake due to hand shake of the terminal user by using a predetermined image stabilization method (S670). In the present embodiment, the mobile terminal 200 may perform shake correction on each image frame by using a sparse optical flow approach.

The mobile terminal 200 may extract feature points of the subject from the cropped image frames. The mobile terminal 200 may detect a motion vector of the subject based on the feature point position information of the current image frame and the feature point position information of the next image frame, and calculate a trajectory of the subject based on this.

The mobile terminal 200 may smooth the trajectory of the subject by using a trajectory smoothing technique. In addition, the mobile terminal 200 may perform image warping by applying an affine transform to cropped image frames. That is, the mobile terminal 200 may compensate for the shaking of the image frames by correcting the position of the pixels for each cropped image frame.

The mobile terminal 200 may generate frames processed by the image according to an image stabilization technique as a moving image file and store them in the memory 250 (S680). In this case, the mobile terminal 200 may store the video file in YUV format.

The mobile terminal 200 may repeatedly perform the operations of steps 620 to 680 until the smart video application is terminated (S690). The above-described operations may be implemented by a spin image generating apparatus provided in the mobile terminal 200.

As described above, in the spin image capturing mode, the mobile terminal 200 tracks the position of a subject existing in each image frame and performs shake correction only on image frames cropped according to the position of the subject. can do.

7 is a diagram referred to describe an operation of a mobile terminal displaying a home screen of a smart video application.

Referring to FIG. 7, the mobile terminal 200 may display the home screen 710 on the display unit according to a user command. In this case, it is assumed that the home screen 710 includes an app icon 715 corresponding to the smart video application.

When the corresponding app icon 715 is selected by the terminal user, the mobile terminal 200 may execute a smart video application corresponding to the selected app icon 715.

The mobile terminal 200 may display the predefined user interface 720 on the display unit when the corresponding application is executed. The user interface 720 may include a home area 721 including thumbnail images corresponding to a plurality of spin images, and a menu area 723 disposed below the home area 721.

The plurality of thumbnail images displayed on the home area 721 may be sequentially arranged according to the generation order of the corresponding spin images. Also, the plurality of thumbnail images may be configured to be scrollable according to a gesture input of a user.

The menu area 723 may include a spin image menu for providing a list of spin images, a camera menu for generating a spin image, and a setting menu for setting an environment of a corresponding application.

8 is a diagram referred to describe an operation of a mobile terminal for deleting a spin image.

Referring to FIG. 8, when the smart video application is executed, the mobile terminal 200 may display a user interface 810 including thumbnail images corresponding to a plurality of spin images on the display unit.

When a user input of touching one of the thumbnail images 820 displayed on the user interface 810 and dragging in the first direction is received, the mobile terminal 200 determines at least one of the touched thumbnail images 820. The display unit on the first display area corresponding to the drag direction of the terminal user, and a delete menu 830 for deleting the spin image 820 corresponding to the touched thumbnail image corresponding to the opposite direction of the drag direction. It can be displayed on the second display area.

On the other hand, in another embodiment, when a user input for long touch (long touch) any one of the thumbnail images displayed on the user interface 810 is received, the mobile terminal 200 is the touched thumbnail At least a portion of the image 820 may be displayed on the first display area, and the delete menu 830 may be displayed on the second display area corresponding to the direction opposite to the drag direction.

When the delete menu 830 is selected by the terminal user, the mobile terminal 200 may delete the spin image 820 corresponding to the touched thumbnail image from the memory 250. In addition, the mobile terminal 200 may remove the thumbnail image 820 displayed on the user interface 810.

9 is a diagram referred to describe an operation of a mobile terminal for inserting tag information into a spin image.

Referring to FIG. 9, when the smart video application is executed, the mobile terminal 200 may display a user interface 910 including thumbnail images corresponding to a plurality of spin images on the display.

When a user input of touching one of the thumbnail images 920 displayed on the user interface 910 and dragging in the first direction is received, the mobile terminal 200 determines at least one of the touched thumbnail images 920. The tag menu 930 for displaying the portion on the first display area corresponding to the drag direction of the terminal user and inputting tag information into the spin image 920 corresponding to the touched thumbnail image in a direction opposite to the drag direction. It may be displayed on the second display area corresponding to.

When the tag menu 930 is selected by the terminal user, the mobile terminal 200 may display a keypad window 940 for inputting tag information and a display window 950 for displaying the input tag information on the display unit. have. The tag information may be a name of a subject (ie, a product), color information, size information, price information, and the like, but is not limited thereto.

When predetermined tag information (eg, product price: $ 100) is input through the keypad window 940, the mobile terminal 200 may insert the input tag information into the spin image. The mobile terminal 200 may generate a thumbnail image 960 in which tag information is inserted and display it on the display unit.

10 and 11 are diagrams for explaining an operation of a mobile terminal for reproducing a spin image.

10 and 11, when executing a smart video application, the mobile terminal 200 may display a first user interface 1010 including thumbnail images corresponding to a plurality of spin images on a display unit. .

When a user input of touching one of the thumbnail images 1020 displayed on the first user interface 1010 is received, the mobile terminal 200 plays a spin image corresponding to the touched thumbnail image 1020. The second user interface 1030 may be displayed on the display unit.

The second user interface 1030 may include a menu area 1040 including a home menu for moving to a home screen, a playback menu for playing a spin image, and the like, and an image display for providing a moving image or a still image related to the spin image. The region 1050 may include a progress bar region 1060 indicating a progress state of the video.

When the playback menu 1041 displayed in the menu area 1040 is selected, the mobile terminal 200 may reproduce the spin image corresponding to the touched thumbnail image 1020 and display it on the display unit. Accordingly, the terminal user may watch a video of photographing the specific subject 1040 in a 360 degree direction.

When the play menu 1041 is selected, the mobile terminal 200 may change the play menu 1041 displayed on the second user interface 1030 to the stop menu 1043. When the still menu 1043 is selected while the video is being played, the mobile terminal 200 may stop the currently playing video and display the still image at the corresponding time on the display unit.

Meanwhile, when the gesture input 1080 in the first direction is received while the video is being played, the mobile terminal 200 may play the video after a certain time point from the current time point. In contrast, when a gesture input (not shown) in a second direction opposite to the first direction is received while the video is being played, the mobile terminal 200 may play the video before a certain point in time from the current point in time.

In addition, when the gesture input 1080 in the first direction is received while the still image is being displayed, the mobile terminal 200 may display the still image after a certain time point from the current time point. On the contrary, when a gesture input (not shown) in the second direction opposite to the first direction is received while the still image is being displayed, the mobile terminal 200 may display the still image before a certain point in time from the current point in time. .

The mobile terminal 200 may detect the gesture input speed of the terminal user and adjust the screen switching speed of the moving image or the still image of the subject. Accordingly, the terminal user may check the appearance of the subject 1070 in more detail through the gesture input having the directivity.

12 is a diagram for explaining an operation of a mobile terminal entering a camera mode according to an embodiment of the present invention.

Referring to FIG. 12, when the smart video application is executed, the mobile terminal 200 may display a user interface 1210 including thumbnail images corresponding to a plurality of spin images on the display.

When the camera menu 1215 displayed at the bottom of the user interface 1210 is selected, the mobile terminal 200 may display a pop-up window 1220 on the display for inquiring whether the corresponding application is allowed to access the camera. . When the OK menu 1225 of the pop-up window 1220 is selected, the mobile terminal 200 may enter a camera mode for generating a spin image.

Meanwhile, in another embodiment, when the camera menu 1215 is selected, the mobile terminal 200 may directly enter a camera mode for generating a spin image without displaying a separate popup window.

When entering the camera mode, the mobile terminal 200 may display the predefined user interface 1230 on the display unit. The user interface 1230 may include a preview area 1240 for previewing an image input through a camera and a menu area 1250 for displaying menus related to a camera mode.

The preview area 1240 may include a first photographing area 1241 positioned in the center area of the screen and a second photographing area 1243 located in the edge area of the screen. The second photographing area 1243 may be dimd to distinguish the first photographing area 1241 from the first photographing area 1241.

The first photographing area 1241 is an area corresponding to an image frame cropped for image processing (ie, image stabilization) and corresponds to an area recorded as a moving image file. Accordingly, the terminal user may adjust the direction of the camera so that the subject 1260 is positioned in the first photographing area 1241.

The menu area 1250 may include a thumbnail menu 1251 for importing a most recently generated spin image, and a shooting button menu 1253 for starting capturing a moving image of the subject 1260.

Meanwhile, as another embodiment, as shown in FIG. 13, when an app icon 1315 corresponding to the smart video application displayed on the home screen 1310 is selected, the mobile terminal 200 selects the selected app icon 1315. Can execute a smart video application corresponding to.

The mobile terminal 200 may directly enter a camera mode for generating a spin image when executing the corresponding application. When entering the camera mode, the mobile terminal 200 may display a predefined user interface 1320 on the display unit. Since the configuration of the user interface 1320 is the same as that of the user interface 1230 illustrated in FIG. 12, a detailed description thereof will be omitted.

14 is a diagram referred to describe an operation of a mobile terminal displaying an indicator for designating a subject.

Referring to FIG. 14, when entering the camera mode, the mobile terminal 200 notifies the indicator 1430 and the indicator 1430 for specifying the subject 1420 to adjust the indicator 1430 according to the size of the subject. The user interface 1410 including the may be displayed on the display unit. In this case, the indicator 1430 may be formed in a square shape, but is not necessarily limited thereto.

The terminal user may adjust the size of the indicator 1430 according to the size of the subject 1420 by touching and dragging edges of the indicator 1430 according to the notification information 1440. In addition, the terminal user may move the position of the indicator 1430 by touching and dragging the inside of the indicator 1430.

Meanwhile, in another embodiment, when the terminal user touches the subject, the mobile terminal may detect the entire outline of the subject through object recognition and automatically adjust the indicator according to the shape and size of the detected subject.

FIG. 15 is a diagram referred to describe an operation of a mobile terminal for providing photographing guide information.

Referring to FIG. 15, when entering the camera mode, the mobile terminal 200 may display a user interface 1510 that previously shows a subject 1520 to be photographed through the camera on the display.

When the photographing button 1530 is selected by the terminal user, the mobile terminal 200 may start capturing a moving image of the subject 1520 by using one or more cameras.

In the spin image capturing mode, the mobile terminal 200 may change the capturing button 1530 for starting capturing a video into a stop button 1540 for stopping capturing video. When the stop button 1540 is selected, the mobile terminal 200 may stop capturing a video about a subject.

In the spin image capturing mode, the mobile terminal 200 may display the first indicator 1550 indicating the photographing angle of the subject in an area adjacent to the stop button 1540. When the terminal user photographs the subject while moving around the subject in a 360 degree direction, the mobile terminal 200 periodically detects the photographing angle of the subject based on the image data and the sensing data, and indicates the detected photographing angle. The first indicator 1550 may be displayed on the display unit.

In the spin image capturing mode, the mobile terminal 200 may display a second indicator 1560 and notification information 1570 for guiding a capturing operation of the terminal user in an adjacent region of the subject 1520. The second indicator 1560 may include one or more icons 1561 and 1563 indicating a shooting direction and a shooting degree of the terminal user. As an example, the first icon 1561 may be formed in a circle shape, and the second icon 1563 may be formed in an arrow shape.

When the terminal user photographs the subject while moving around the subject in a 360 degree direction, the mobile terminal 200 periodically detects the photographing direction and the photographing degree of the terminal user based on the image data and the sensing data, and detects the photographed subject. The second indicator 1560 indicating the direction and the degree of photographing may be displayed on the display unit.

In addition, the mobile terminal 200 may periodically analyze the photographing state or the photographing operation of the terminal user based on the information analyzed by the image data and the sensing data, and provide the notification information 1570 accordingly. Examples of the notification information may include 'move slowly (faster)', 'move closer (distant)', 'move (camera up)', 'do not shake your hand (arm)', 'subject' Go to "," Wrong direction "," Out of Screen ".

16 and 17 are diagrams for explaining an operation of a mobile terminal for reproducing the most recently generated spin image.

16 and 17, when entering the camera mode, the mobile terminal 200 may display a predefined user interface 1610 on the display unit. In this case, the user interface 1610 may include a preview area 1611 for previewing a subject to be photographed by the camera, a thumbnail menu 1613 for importing the most recently generated spin image, and a recording button for starting video recording. May include a menu 1615.

When the thumbnail menu 1613 is selected by the terminal user, the mobile terminal 200 may display a user interface 1620 for playing the most recently generated spin image on the display.

For example, the user interface 1620 may include a previous menu 1631 for returning to the camera mode, an information menu 1633 indicating date and time information, and a list of spin images as illustrated in FIG. 16B. A first menu area 1630 including a list menu 1635 for calling, an image display area 1640 providing a moving image or a still image related to the spin image, and a progress bar area indicating a moving state of the moving image; (1650), a playback menu (1661) for playing a video, an edit menu (1663) for editing a spin image, a sharing menu (1665) for sharing a spin image, and a delete menu (1667) for deleting a spin image. And a second menu area 1660 including the like.

When the playback menu 1661 is selected by the terminal user, as illustrated in FIG. 17A, the mobile terminal 200 may play back a video file corresponding to the spin image and display it on the display unit. Accordingly, the terminal user may watch a video of photographing the specific subject 1040 in a 360 degree direction.

When the still menu 1661 is selected by the terminal user, the mobile terminal 200 may stop the video currently being played and display the still image at the corresponding time on the display unit. In this state, when a gesture input having directionality is received through the display unit, the mobile terminal 200 may display still images photographing a subject in different directions on the display unit.

For example, as illustrated in (b) of FIG. 17, when a gesture input (not shown) having a direction is received while a still image is being displayed, the mobile terminal 200 may determine whether the mobile terminal 200 is before or after a certain point in time. A still image (that is, a still image of photographing a subject in a direction different from the current direction) may be displayed on the display unit. In this case, the mobile terminal 200 may display an indicator 1670 indicating the degree of rotation of the subject through the gesture input on the display.

On the other hand, as shown in (c) of FIG. 17, when a user input (not shown) for tapping the display unit for displaying a video or a still image is received, the mobile terminal 200 displays the video or still image. Can be displayed in full screen (1680).

18 and 19 are diagrams for explaining an operation of a mobile terminal displaying a list of spin images.

Referring to FIGS. 18 and 19, the mobile terminal 200 may display a user interface 1810 for playing a video file corresponding to a spin image on a display unit.

When the list menu 1815 displayed in one area of the user interface 1810 is selected, the mobile terminal 200 displays an image list screen 1820 including thumbnail images corresponding to the spin images stored in the memory 250. It can be displayed on the display unit.

When a user input for touching (or tapping) any one of the thumbnail images displayed on the image list screen 1820 is received, the mobile terminal 200 may select the touched thumbnail image 1821. In this case, the mobile terminal 200 may display the selected thumbnail image 1821 to be distinguished from other thumbnail images.

Meanwhile, as illustrated in FIG. 19, when a user input of double touching (or double tapping) any one of the thumbnail images displayed on the image list screen 1820 is received, the mobile terminal 200 may receive the above input. The user interface 1830 for reproducing the spin image corresponding to the double-touch thumbnail image 1823 may be displayed on the display unit. Since the configuration of the user interface 1830 is the same as that of the user interface 1620 illustrated in FIG. 16B, detailed description thereof will be omitted.

20 is a diagram referred to describe an operation of a mobile terminal sharing a spin image.

Referring to FIG. 20, the mobile terminal 200 may display a user interface 2010 for playing a video file corresponding to a spin image on a display unit.

When the sharing menu 2015 displayed in one area of the user interface 2010 is selected, the mobile terminal 200 converts the app icons and the spin image into different file formats corresponding to one or more SNS applications to share the spin image. A pop-up window 2020 including menu icons for converting may be displayed on the display unit. The terminal user may select at least one of the SNS applications displayed on the popup window 2020 to share the spin image. In addition, the terminal user may convert the spin image into various file formats such as MP4, Graphics Interchange Format (GIF), Live Photo, etc. using the menu icons displayed on the pop-up window 2020.

FIG. 21 is a diagram referred to for describing an operation of a mobile terminal for editing a spin image.

Referring to FIG. 21, the mobile terminal 200 may display a first user interface 2110 for playing a video file corresponding to a spin image on a display unit.

When the edit menu 2115 displayed in one area of the first user interface 2110 is selected, the mobile terminal 200 may display a second user interface 2120 for editing the spin image on the display. For example, the second user interface 2120 may display a first menu area 2130 including a cancel menu for canceling an image editing mode, a save menu for storing an edited image, and a spin image being edited. And an image editing area 2140 and a second menu area 2150 including a plurality of image editing menus.

The plurality of image editing menus include an automatic menu, a brightness menu, a contrast menu, a saturation menu, a sharpness menu, a color temperature menu, and the like. The terminal user can easily edit the spin image by selecting a plurality of image editing menus.

FIG. 22 is a diagram referred to describe an operation of a mobile terminal that provides object information for augmented reality.

Referring to FIG. 22, the mobile terminal 200 may display a user interface 2210 for reproducing a spin image of a specific subject on a display unit according to a user command.

The mobile terminal 200 may identify a specific subject by using a predetermined object recognition algorithm. The mobile terminal 200 may obtain object information about the identified subject by accessing the Internet or the like. The mobile terminal 200 may perform an operation of inserting object information 2220 about a subject into a spin image. The object information 2220 may include a name of a subject, color information, size information, price information, power information, and the like.

FIG. 23 is a diagram referred to describe an operation of a mobile terminal capable of replacing a background of a spin image.

Referring to FIG. 23, the mobile terminal 200 may display a user interface 2310 for reproducing a spin image regarding a specific subject on a display unit according to a user command.

When a specific background screen is selected through the manipulation of the user interface 2310, the mobile terminal 200 may change the background screen 2320 of the subject to the selected background screen 2330. Accordingly, the terminal user can immediately check which background the particular subject is well suited to.

Meanwhile, in addition to the above, the mobile terminal 200 may provide various functions related to generation and reproduction of the spin image. For example, the mobile terminal 200 may provide a function of improving image quality of the spin image. In addition, the mobile terminal 200 may provide a function of generating a panoramic image as shown in FIG. 24 as well as a spin image of photographing a specific subject in a 360 degree direction.

In the spin image capturing mode, the mobile terminal 200 may perform a function of automatically detecting a specific subject based on an object recognition algorithm using deep learning. In addition, the mobile terminal 200 may provide a function of extracting feature points of a specific subject and visualizing the extracted feature points in the spin image capturing mode.

The present invention described above can be embodied as computer readable codes on a medium on which a program is recorded. The computer-readable medium, continue to store executable computer program, or may be temporarily stored for a run or download. In addition, the medium may be one which can be a variety of recording device or storage means of a single or several hardware combined form, is not limited to the medium to be directly connected to any computer system, there distributed to the network. Examples of media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, the media can be recorded to storage media managed as an example for other media, website, etc. supplied to the app store or other retail distributor of various software applications, and servers. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

10: service providing system 100: communication network
150: server 200: mobile terminal
210: communication unit 220: sensing unit
230: input unit 240: output unit
250: memory 260: control unit
265/300: spin image generating unit 310: image frame receiving unit
320: data analysis unit 330: guide information provider
340: subject tracking unit 350: frame crop unit
360: image stabilizer 370: file generator
380: file playback unit

Claims (20)

Displaying a preview screen for previewing an image input through a camera;
Receiving image frames regarding a specific subject from the camera when a photographing button is selected;
Tracking a subject present in each image frame by using a predetermined tracking algorithm;
Cropping at least a portion of each image frame according to the position of the subject; And
Stabilizing the cropped image frames. The method of claim 1,
The spin image may include moving images around a specific subject and moving images or still images taken in a 360 degree direction. The method of claim 1,
The preview screen includes a first image capturing region recorded as a moving image file corresponding to a spin image, and a second image capturing region except the first image capturing region. The method of claim 1,
And displaying an indicator for designating the subject on the preview screen. The method of claim 1,
Detecting a photographing state of a terminal user by analyzing at least one of image data and sensing data when the photographing button is selected; And
And providing photographing guide information for guiding a photographing operation of the terminal user. The method of claim 5,
The photographing guide information includes an indicator for indicating the photographing direction and the photographing degree of the terminal user. The method of claim 1, wherein the tracking step,
A method of generating a spin image comprising tracking a position of the subject using a Kernelized Correlation Filter (KCF) algorithm and a Felzenszwalb-Histogram of Gradient (FHOG) algorithm. The method of claim 1, wherein the cropping step,
And a background of each image frame is cut out such that a subject is positioned in a center region of a playback screen for the spin image. The method of claim 1, wherein the stabilizing step,
And performing image stabilization on the cropped image frames by using an optical flow approach. The method of claim 9, wherein the stabilizing step,
Extracting feature points of a subject from the cropped image frames; And
And detecting a trajectory of the subject based on feature point position information between image frames adjacent to each other. The method of claim 10, wherein the stabilizing step,
Smoothing the trajectory of the subject using a trajectory smoothing technique; And
And performing image warping by applying an affine transform to the cropped image frames. The method of claim 1,
And generating a moving image file of the subject based on the stabilized image frames. The method of claim 12,
And storing the video file in YUV format or MP4 format. The method of claim 1,
And displaying a moving image or a still image corresponding to the spin image. The method of claim 1,
Identifying the subject using a predetermined object recognition algorithm, obtaining object information regarding the identified subject, and inserting the object information into the spin image. The method of claim 1,
And replacing the wallpaper of the subject with another wallpaper. A computer program stored in the computer readable recording medium for executing the method according to any one of claims 1 to 16 on a computer. An image frame receiving unit configured to receive image frames related to a specific subject from one or more cameras;
A subject tracking unit which tracks a subject existing in each image frame by using a predetermined tracking algorithm;
An image cropping unit that crops at least a portion of each image frame according to the position of the subject; And
And a shake correction unit configured to stabilize the cropped image frames. The method of claim 1,
A data analyzing unit configured to detect a photographing state of a terminal user by analyzing image data and sensing data in a photographing mode; And
And a guide information providing unit configured to provide shooting guide information according to the shooting state of the terminal user. The method of claim 1,
A file generator configured to generate a moving image file of the subject based on the stabilized image frames; And
And a file reproducing unit configured to provide a moving image or a still image corresponding to the spin image.

Images