CN106851114B - Photo display device, photo generation device, photo display method, photo generation method and terminal - Google Patents
Photo display device, photo generation device, photo display method, photo generation method and terminal Download PDFInfo
- Publication number
- CN106851114B CN106851114B CN201710206286.2A CN201710206286A CN106851114B CN 106851114 B CN106851114 B CN 106851114B CN 201710206286 A CN201710206286 A CN 201710206286A CN 106851114 B CN106851114 B CN 106851114B
- Authority
- CN
- China
- Prior art keywords
- area
- photo
- user
- sight
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Telephone Function (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a photo display device, a photo generation method and a terminal, wherein the photo display device comprises: the first determination module is used for determining an area where the sight of the user stays on the photo; the second determining module is used for determining a main object of the area where the sight of the user stays on the photo; and the display module is used for dynamically presenting the main object in the area where the determined sight line stays, wherein the picture is formed by identifying the main object and the background when being shot and integrating dynamic information of the main object and static information of the background. According to the method and the device, the shot picture is synthesized based on the dynamic data of the main body area and the static data of the background area, the main body object aimed by the sight of the user can be identified when the user views the picture, and the main body object aimed by the sight of the user is subjected to a dynamic form, so that the diversity of picture presentation is improved.
Description
Technical Field
The invention relates to the field of image processing, in particular to a photo display and photo generation device and method and a terminal.
Background
With the continuous development of internet technology and mobile terminals such as mobile phones and the like. More and more intelligent terminals integrate the camera shooting function, so that people can realize the functions of shooting and the like through mobile phone terminals.
When the current picture is shot, the whole picture area is displayed in the same way. For example, video is displayed dynamically in the entire area, and pictures are displayed statically in the entire area. This results in a loss of much flexibility in the implementation of the picture.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The invention mainly aims to provide a photo display device, a photo generation method and a terminal, aiming at improving the diversity of picture presentation.
To achieve the above object, the present invention provides a photograph display apparatus, comprising:
the first determination module is used for determining an area where the sight of the user stays on the photo;
the second determining module is used for determining a main object of the area where the sight of the user stays on the photo;
and the display module is used for dynamically presenting the main object in the area where the determined sight line stays, wherein the picture is formed by identifying the main object and the background when being shot and integrating dynamic information of the main object and static information of the background.
Further, the second determining module comprises:
a first determination unit configured to determine a coordinate position of a user's sight line in the photograph when a plurality of subject objects exist on the photograph;
a second determination unit for determining one or more subject objects including the coordinate position within the photograph;
and the generating unit is used for taking one or more main objects including the coordinate positions in the determined photo as the main objects of the determined area where the sight of the user stays on the photo.
Further, the first determining module comprises: the sensing unit is used for sensing the visual center of the eyes of the user through a front camera on the terminal;
and the generating unit is used for taking the sensed area within the preset range of the visual center of the eyes of the user as the area where the sight of the user stays on the photo.
In addition, to achieve the above object, the present invention also provides a photograph creation apparatus, including:
the identification module is used for identifying a main body area and a background area of the shot area;
the first acquisition module is used for acquiring dynamic data of the main body area and static data of the background area;
and the synthesis module is used for synthesizing the dynamic data of the main body area and the static data of the background area into a shot picture.
Further, the above apparatus further comprises: and the second acquisition module is used for acquiring the image data of the shot area through the first camera and the second camera in the terminal before the main body area and the background area of the shot area are identified.
Further, the first obtaining module includes:
a first acquisition unit, configured to acquire dynamic data of the subject area through the first camera;
and the second acquisition unit is used for acquiring the static data of the background area through the second camera.
In addition, to achieve the above object, the present invention further provides a terminal, including: the photograph display apparatus of any one of the above, and/or the photograph creation apparatus of any one of the above.
In addition, to achieve the above object, the present invention further provides a photo display method, including:
determining an area where the user's gaze stays on the photograph;
determining a main object of a region where the sight of a user stays on the photo;
and dynamically presenting the main object in the area where the determined sight line stays, wherein the picture is formed by identifying the main object and the background when being shot and integrating the dynamic information of the main object and the static information of the background.
Further, in a case where a plurality of subject objects exist on the photograph, the determining a subject object in which the line of sight of the user stays in the region on the photograph includes:
determining the coordinate position of the sight of the user in the photo;
determining one or more subject objects within the photograph that include the coordinate locations;
and one or more objects including the coordinate positions in the determined photo are taken as the objects of the regions where the determined sight line of the user stays on the photo.
Further, determining an area where the user's gaze stays on the photograph includes:
sensing a visual center of eyes of a user through a front camera on the terminal;
and taking the sensed area within the preset range of the visual center of the eyes of the user as the area where the sight line of the user stays on the photo.
In addition, in order to achieve the above object, the present invention further provides a photo generation method, including:
identifying a subject area and a background area of a shot area;
acquiring dynamic data of the subject area and static data of the background area;
and synthesizing the dynamic data of the main body area and the static data of the background area into a shot picture.
Further, before identifying the subject area and the background area of the photographed area, the method further includes:
and acquiring the image data of the shot area through a first camera and a second camera in the terminal.
Further, acquiring dynamic data of the subject area and static data of the background area includes:
acquiring dynamic data of the subject area through the first camera;
and acquiring static data of the background area through the second camera.
Further, the identifying the subject area and the background area of the photographed area includes:
and taking a region with motion change in a preset time in the shot region as a subject region.
The photo display and generation device and method provided by the invention can be used for synthesizing the shot photo based on the dynamic data of the main body area and the static data of the background area, identifying the main body object aimed by the sight of the user when the user views the photo, and carrying out dynamic form on the main body object aimed by the sight of the user, thereby improving the diversity of picture presentation.
Drawings
Fig. 1 is a schematic diagram of a hardware structure of an optional mobile terminal for implementing various embodiments of the present invention;
FIG. 2 is a diagram of a wireless communication system for the mobile terminal shown in FIG. 1;
FIG. 3 is a block diagram of a photo display device according to an embodiment of the present invention;
FIG. 4 is a block diagram of a photograph creation apparatus according to an embodiment of the present invention;
FIG. 5 is a method flow diagram of a photo display method according to an embodiment of the invention;
FIG. 6 is a method flow diagram of a method of photo generation according to an embodiment of the invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.
The mobile terminal may be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, it will be understood by those skilled in the art that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for moving purposes.
Fig. 1 is a schematic hardware configuration of a mobile terminal implementing various embodiments of the present invention. Of course, the hardware structure of the mobile terminal will be fully described in the embodiments of the present invention, and in the following embodiments, if the mobile terminal is also referred to, the hardware structure description in the embodiments may also be used, and the following description shall be applicable to each embodiment of the present invention, which will not be repeatedly explained in the following embodiments.
The mobile terminal 100 may include a wireless communication unit 110, an a/V (audio/video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190, etc. Fig. 1 illustrates a mobile terminal having various components, but it is to be understood that not all illustrated components are required to be implemented. More or fewer components may alternatively be implemented. Elements of the mobile terminal will be described in detail below.
The wireless communication unit 110 typically includes one or more components that allow radio communication between the mobile terminal 100 and a wireless communication system or network. For example, the wireless communication unit may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short-range communication module 114, and a location information module 115.
The broadcast receiving module 111 receives a broadcast signal and/or broadcast associated information from an external broadcast management server via a broadcast channel. The broadcast channel may include a satellite channel and/or a terrestrial channel. The broadcast management server may be a server that generates and transmits a broadcast signal and/or broadcast associated information or a server that receives a previously generated broadcast signal and/or broadcast associated information and transmits it to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and the like. Also, the broadcast signal may further include a broadcast signal combined with a TV or radio broadcast signal. The broadcast associated information may also be provided via a mobile communication network, and in this case, the broadcast associated information may be received by the mobile communication module 112. The broadcast signal may exist in various forms, for example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB), an Electronic Service Guide (ESG) of digital video broadcasting-handheld (DVB-H), and the like. The broadcast receiving module 111 may receive a signal broadcast by using various types of broadcasting systems. In particular, the broadcast receiving module 111 may receive a broadcast signal by using a signal such as multimedia broadcasting-terrestrial (DMB-T)) Digital multimedia broadcasting-satellite (DMB-S), digital video broadcasting-handheld (DVB-H), forward link media (MediaFLO)@) A digital broadcasting system of a terrestrial digital broadcasting integrated service (ISDB-T), etc. receives digital broadcasting. The broadcast receiving module 111 may be constructed to be suitable for various broadcasting systems that provide broadcast signals as well as the above-mentioned digital broadcasting systems. The broadcast signal and/or broadcast associated information received via the broadcast receiving module 111 may be stored in the memory 160 (or other type of storage medium).
The mobile communication module 112 transmits and/or receives radio signals to and/or from at least one of a base station (e.g., access point, node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received according to text and/or multimedia messages.
The wireless internet module 113 supports wireless internet access of the mobile terminal. The module may be internally or externally coupled to the terminal. The wireless internet access technology to which the module relates may include WLAN (wireless LAN) (Wi-Fi), Wibro (wireless broadband), Wimax (worldwide interoperability for microwave access), HSDPA (high speed downlink packet access), and the like.
The short-range communication module 114 is a module for supporting short-range communication. Some examples of short-range communication technologies include bluetoothTMRadio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), zigbeeTMAnd so on.
The location information module 115 is a module for checking or acquiring location information of the mobile terminal. A typical example of the location information module is a GPS (global positioning system). According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites and applies triangulation to the calculated information, thereby accurately calculating three-dimensional current location information according to longitude, latitude, and altitude. Currently, a method for calculating position and time information uses three satellites and corrects an error of the calculated position and time information by using another satellite. In addition, the GPS module 115 can calculate speed information by continuously calculating current position information in real time.
The a/V input unit 120 is used to receive an audio or video signal. The a/V input unit 120 may include a camera 121 and a microphone 122, and the camera 121 processes image data of still pictures or video obtained by an image capturing apparatus in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 151. The image frames processed by the cameras 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and two or more cameras 121 may be provided according to the construction of the mobile terminal. The microphone 122 may receive sounds (audio data) via the microphone in a phone call mode, a recording mode, a voice recognition mode, or the like, and can process such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the mobile communication module 112 in case of a phone call mode. The microphone 122 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.
The user input unit 130 may generate key input data according to a command input by a user to control various operations of the mobile terminal. The user input unit 130 allows a user to input various types of information, and may include a keyboard, dome sheet, touch pad (e.g., a touch-sensitive member that detects changes in resistance, pressure, capacitance, and the like due to being touched), scroll wheel, joystick, and the like. In particular, when the touch pad is superimposed on the display unit 151 in the form of a layer, a touch screen may be formed.
The sensing unit 140 detects a current state of the mobile terminal 100 (e.g., an open or closed state of the mobile terminal 100), a position of the mobile terminal 100, presence or absence of contact (i.e., touch input) by a user with the mobile terminal 100, an orientation of the mobile terminal 100, acceleration or deceleration movement and direction of the mobile terminal 100, and the like, and generates a command or signal for controlling an operation of the mobile terminal 100. For example, when the mobile terminal 100 is implemented as a slide-type mobile phone, the sensing unit 140 may sense whether the slide-type phone is opened or closed. In addition, the sensing unit 140 can detect whether the power supply unit 190 supplies power or whether the interface unit 170 is coupled with an external device. The sensing unit 140 may include a proximity sensor 1410 as will be described below in connection with a touch screen.
The interface unit 170 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The identification module may store various information for authenticating a user using the mobile terminal 100 and may include a User Identity Module (UIM), a Subscriber Identity Module (SIM), a Universal Subscriber Identity Module (USIM), and the like. In addition, a device having an identification module (hereinafter, referred to as an "identification device") may take the form of a smart card, and thus, the identification device may be connected with the mobile terminal 100 via a port or other connection means. The interface unit 170 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal and the external device.
In addition, when the mobile terminal 100 is connected with an external cradle, the interface unit 170 may serve as a path through which power is supplied from the cradle to the mobile terminal 100 or may serve as a path through which various command signals input from the cradle are transmitted to the mobile terminal. Various command signals or power input from the cradle may be used as signals for recognizing whether the mobile terminal is accurately mounted on the cradle. The output unit 150 is configured to provide output signals (e.g., audio signals, video signals, alarm signals, vibration signals, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and the like.
The display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphical User Interface (GUI) related to a call or other communication (e.g., text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or an image and related functions, and the like.
Meanwhile, when the display unit 151 and the touch pad are overlapped with each other in the form of a layer to form a touch screen, the display unit 151 may serve as an input device and an output device. The display unit 151 may include at least one of a Liquid Crystal Display (LCD), a thin film transistor LCD (TFT-LCD), an Organic Light Emitting Diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as transparent displays, and a typical transparent display may be, for example, a TOLED (transparent organic light emitting diode) display or the like. Depending on the particular desired implementation, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown). The touch screen may be used to detect a touch input pressure as well as a touch input position and a touch input area.
The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 into an audio signal and output as sound when the mobile terminal is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output module 152 may provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output module 152 may include a speaker, a buzzer, and the like.
The alarm unit 153 may provide an output to notify the mobile terminal 100 of the occurrence of an event. Typical events may include call reception, message reception, key signal input, touch input, and the like. In addition to audio or video output, the alarm unit 153 may provide output in different ways to notify the occurrence of an event. For example, the alarm unit 153 may provide an output in the form of vibration, and when a call, a message, or some other incoming communication (incomingmunication) is received, the alarm unit 153 may provide a tactile output (i.e., vibration) to inform the user thereof. By providing such a tactile output, the user can recognize the occurrence of various events even when the user's mobile phone is in the user's pocket. The alarm unit 153 may also provide an output notifying the occurrence of an event via the display unit 151 or the audio output module 152.
The memory 160 may store software programs and the like for processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a phonebook, messages, still images, videos, and the like) that has been or will be output. Also, the memory 160 may store data regarding various ways of vibration and audio signals output when a touch is applied to the touch screen.
The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. Also, the mobile terminal 100 may cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.
The controller 180 generally controls the overall operation of the mobile terminal. For example, the controller 180 performs control and processing related to voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 181 for reproducing (or playing back) multimedia data, and the multimedia module 181 may be constructed within the controller 180 or may be constructed separately from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.
The power supply unit 190 receives external power or internal power and provides appropriate power required to operate various elements and components under the control of the controller 180.
The various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, the embodiments described herein may be implemented using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, such embodiments may be implemented in the controller 180. For a software implementation, the implementation such as a process or a function may be implemented with a separate software module that allows performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in the memory 160 and executed by the controller 180.
Up to this point, mobile terminals have been described in terms of their functionality. Hereinafter, a slide-type mobile terminal among various types of mobile terminals, such as a folder-type, bar-type, swing-type, slide-type mobile terminal, and the like, will be described as an example for the sake of brevity. Accordingly, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.
The mobile terminal 100 as shown in fig. 1 may be configured to operate with communication systems such as wired and wireless communication systems and satellite-based communication systems that transmit data via frames or packets.
A communication system in which a mobile terminal according to the present invention is operable will now be described with reference to fig. 2.
Such communication systems may use different air interfaces and/or physical layers. For example, the air interface used by the communication system includes, for example, Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and Universal Mobile Telecommunications System (UMTS) (in particular, Long Term Evolution (LTE)), global system for mobile communications (GSM), and the like. By way of non-limiting example, the following description relates to a CDMA communication system, but such teachings are equally applicable to other types of systems.
Referring to fig. 2, the CDMA wireless communication system may include a plurality of mobile terminals 100, a plurality of Base Stations (BSs) 270, Base Station Controllers (BSCs) 275, and a Mobile Switching Center (MSC) 280. The MSC280 is configured to interface with a Public Switched Telephone Network (PSTN) 290. The MSC280 is also configured to interface with a BSC275, which may be coupled to the base station 270 via a backhaul. The backhaul may be constructed according to any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, frame Relay, HDSL, ADSL, or xDSL. It will be understood that a system as shown in fig. 2 may include multiple BSCs 275.
Each BS270 may serve one or more sectors (or regions), each sector covered by a multi-directional antenna or an antenna pointing in a particular direction being radially distant from the BS 270. Alternatively, each partition may be covered by two or more antennas for diversity reception. Each BS270 may be configured to support multiple frequency allocations, with each frequency allocation having a particular frequency spectrum (e.g., 1.25MHz,5MHz, etc.).
The intersection of partitions with frequency allocations may be referred to as a CDMA channel. The BS270 may also be referred to as a Base Transceiver Subsystem (BTS) or other equivalent terminology. In such a case, the term "base station" may be used to generically refer to a single BSC275 and at least one BS 270. The base stations may also be referred to as "cells". Alternatively, each sector of a particular BS270 may be referred to as a plurality of cell sites.
As shown in fig. 2, a Broadcast Transmitter (BT)295 transmits a broadcast signal to the mobile terminal 100 operating within the system. A broadcast receiving module 111 as shown in fig. 1 is provided at the mobile terminal 100 to receive a broadcast signal transmitted by the BT 295. In fig. 2, several Global Positioning System (GPS) satellites 300 are shown. The satellite 300 assists in locating at least one of the plurality of mobile terminals 100.
In fig. 2, a plurality of satellites 300 are depicted, but it is understood that useful positioning information may be obtained with any number of satellites. The GPS module 115 as shown in fig. 1 is generally configured to cooperate with satellites 300 to obtain desired positioning information. Other techniques that can track the location of the mobile terminal may be used instead of or in addition to GPS tracking techniques. In addition, at least one GPS satellite 300 may selectively or additionally process satellite DMB transmission.
As a typical operation of the wireless communication system, the BS270 receives reverse link signals from various mobile terminals 100. The mobile terminal 100 is generally engaged in conversations, messaging, and other types of communications. Each reverse link signal received by a particular base station 270 is processed within the particular BS 270. The obtained data is forwarded to the associated BSC 275. The BSC provides call resource allocation and mobility management functions including coordination of soft handoff procedures between BSs 270. The BSCs 275 also route the received data to the MSC280, which provides additional routing services for interfacing with the PSTN 290. Similarly, the PSTN290 interfaces with the MSC280, the MSC interfaces with the BSCs 275, and the BSCs 275 accordingly control the BS270 to transmit forward link signals to the mobile terminal 100.
Based on the above hardware structure of the mobile terminal and the communication system, the photo display apparatus in the embodiment of the present invention is provided, as shown in fig. 3, the photo display apparatus in the embodiment of the present invention may include: a first determining module 301, a second determining module 302 and a display module 303. The following description of the various modules follows:
1) a first determination module 301, configured to determine an area where a user's gaze stays on a photo;
when the staying area of the sight line of the user on the screen is identified, the visual center of the eyes of the user is sensed through a front camera on the terminal, and the sensed area within the preset range of the visual center of the eyes of the user is used as the area where the sight line of the user stays on the picture.
That is, it is considered that when a general person looks at a picture, the region where the visual center faces is the region that the person is looking at. Therefore, the current interest area of the viewer, namely the area faced by the sight line of the user, can be recognized as the interest area of the viewer through the front camera.
2) A second determining module 302, configured to determine a subject of an area where the sight of the user stays on the photo;
after the area of interest of the viewer is determined, the main objects in the area can be determined, and one or more main objects can be determined.
For example, if there are two subjects currently in the user's line of sight, one moving person and one dog walking, then both subjects can be identified as recognized subjects.
Or, if there is only one dynamic subject in the current picture, when the user looks at the picture, the dynamic subject is directly determined as the identified subject.
It should be noted, however, that the above-listed manners for determining the subject matter are only schematic descriptions, and other manners for determining the subject matter may be adopted in practical implementation. This is not a limitation of the present application.
In one embodiment, the second determining module 302 may include: a first determination unit configured to determine a coordinate position of a user's sight line in the photograph when a plurality of subject objects exist on the photograph; a second determination unit for determining one or more subject objects including the coordinate position within the photograph; and the generating unit is used for taking one or more main objects including the coordinate positions in the determined photo as the main objects of the determined area where the sight of the user stays on the photo.
That is, the subject is specified by the coordinate position on the screen.
3) And the display module 303 is configured to dynamically present the subject in the determined area where the line of sight stays, wherein the picture is formed by identifying the subject and the background when shooting, and integrating dynamic information of the subject and static information of the background.
The picture is a picture with dynamic and static combination, that is, when shooting, the main object is a dynamic picture shot, for example, the background is a static picture used in all areas, and then the two pictures are combined to obtain the picture with dynamic and static combination.
For example, a picture is taken of two persons (small a and small B) who eat while sitting on a table, and although a still displayed picture is taken, picture information of small a having a meal in motion and picture information of small B having a meal in motion are stored therein.
When the fact that the user is looking at the small A currently is detected, the dynamic picture data of the small A is displayed, and if the user is looking at the small B currently, the dynamic picture data of the small B is displayed, so that the flexibility and the interestingness of picture display can be improved in the mode.
Further, when identifying which subject dynamic data are currently displayed, not only the above-described manner of identifying the line of sight of the viewer may be adopted, but also other manners may be adopted, for example, by a user performing a click operation on a certain target object, and in response to the operation, the dynamic data of the subject corresponding to the click position are displayed. Specifically, what kind of mode is adopted to trigger and carry out the demonstration of main part thing dynamic data, can select according to actual need, and this application does not limit this.
This example also provides a photograph creation apparatus, as shown in fig. 4, which may include: the identification module 401, the first obtaining module 402 and the synthesis module 403, which are described in detail below:
1) an identification module 401, configured to identify a subject area and a background area of a captured area;
in consideration of the fact that the method and the device mainly aim to realize the photographing and displaying of the picture in a dynamic and static combination mode, some areas of the photographed picture are dynamic, but the formed picture is static, and the dynamic areas can be displayed only when the dynamic data need to be displayed.
If these dynamic regions are stationary target objects, it is obviously meaningless, and therefore the subject to be recognized needs to be a dynamic object to make this picture taking and displaying process meaningful. Therefore, when the subject region is identified, it is possible to identify a region in which a dynamic change exists in the region, and to set the region in which a dynamic change exists for a predetermined time as the subject region. For example, if several persons eat the area to be photographed, the person's upper body moves in the area, and thus the individual upper body can be used as the main body area. In the case of recognition, a region that dynamically changes within a predetermined time is recognized as a subject region, and for example, a region in which changes occur in three seconds is counted as a subject region.
In one embodiment, the identification module 401 may be specifically configured to use a region in which motion changes exist within a predetermined time in the captured region as a subject region.
2) A first obtaining module 402, configured to obtain dynamic data of the subject area and static data of the background area;
after the subject area is identified, other areas can be used as background areas, and when picture information is acquired, dynamic data of the subject area can be acquired, such as: the dynamic data changed within 3s only needs to acquire static information once for the background area. When there are a plurality of body regions, dynamic data of each body region is acquired.
If there are two cameras on the shooting terminal: the first camera and the second camera can acquire the image data of the shot area through the first camera and the second camera in the terminal before the main body area and the background area of the shot area are identified. Specifically, in order to realize effective division of functions, dynamic data of the subject area may be acquired by the first camera, and static data of the background area may be acquired by the second camera.
That is, the first camera may acquire one picture as a picture of the final base image, or may acquire a still picture that is finally displayed, and then acquire the dynamic data of the recognized subject by the second camera. The data of the two are combined to obtain the final dynamic and static combined picture.
3) A synthesizing module 403, configured to synthesize the dynamic data of the subject area and the static data of the background area into a captured picture.
That is, the final shot picture is obtained by combining the dynamic data and the static data. Because the photo contains both dynamic data and static data, the user can see the dynamic change when the user wants to display the dynamic change. Furthermore, if the visual center of the eyes of the viewer is sensed, the automatic display area is triggered, so that the main object in the picture can be dynamically played, and the technical effect of seeing where the main object is supposed to move can be achieved.
Based on the photo display and photo generation device, the invention provides various embodiments of the method.
As shown in fig. 5, an embodiment of the present invention provides a photo display method, which may include:
step 501: determining an area where the user's gaze stays on the photograph;
when the staying area of the sight line of the user on the screen is identified, the visual center of the eyes of the user is sensed through a front camera on the terminal, and the sensed area within the preset range of the visual center of the eyes of the user is used as the area where the sight line of the user stays on the picture.
That is, it is considered that when a general person looks at a picture, the region where the visual center faces is the region that the person is looking at. Therefore, the current interest area of the viewer, namely the area faced by the sight line of the user, can be recognized as the interest area of the viewer through the front camera.
Step 502: determining a main object of a region where the sight of a user stays on the photo;
after the area of interest of the viewer is determined, the main objects in the area can be determined, and one or more main objects can be determined.
For example, if there are two subjects currently in the user's line of sight, one moving person and one dog walking, then both subjects can be identified as recognized subjects.
Or, if there is only one dynamic subject in the current picture, when the user looks at the picture, the dynamic subject is directly determined as the identified subject.
It should be noted, however, that the above-listed manners for determining the subject matter are only schematic descriptions, and other manners for determining the subject matter may be adopted in practical implementation. This is not a limitation of the present application.
In one embodiment, in a case where a plurality of subject objects exist on the photograph, determining the subject object in which the line of sight of the user stays on the photograph may include: determining the coordinate position of the sight of the user in the photo; determining one or more subject objects within the photograph that include the coordinate locations; and one or more objects including the coordinate positions in the determined photo are taken as the objects of the regions where the determined sight line of the user stays on the photo.
Step 503: and dynamically presenting the main object in the area where the determined sight line stays, wherein the picture is formed by identifying the main object and the background when being shot and integrating the dynamic information of the main object and the static information of the background.
The picture is a picture with dynamic and static combination, that is, when shooting, the main object is a dynamic picture shot, for example, the background is a static picture used in all areas, and then the two pictures are combined to obtain the picture with dynamic and static combination.
For example, a picture is taken of two persons (small a and small B) who eat while sitting on a table, and although a still displayed picture is taken, picture information of small a having a meal in motion and picture information of small B having a meal in motion are stored therein.
When the fact that the user is looking at the small A currently is detected, the dynamic picture data of the small A is displayed, and if the user is looking at the small B currently, the dynamic picture data of the small B is displayed, so that the flexibility and the interestingness of picture display can be improved in the mode.
Further, when identifying which subject dynamic data are currently displayed, not only the above-described manner of identifying the line of sight of the viewer may be adopted, but also other manners may be adopted, for example, by a user performing a click operation on a certain target object, and in response to the operation, the dynamic data of the subject corresponding to the click position are displayed. Specifically, what kind of mode is adopted to trigger and carry out the demonstration of main part thing dynamic data, can select according to actual need, and this application does not limit this.
As shown in fig. 6, an embodiment of the present invention provides a photo generation method, which may include the following steps:
step 601: identifying a subject area and a background area of a shot area;
in consideration of the fact that the method and the device mainly aim to realize the photographing and displaying of the picture in a dynamic and static combination mode, some areas of the photographed picture are dynamic, but the formed picture is static, and the dynamic areas can be displayed only when the dynamic data need to be displayed.
If these dynamic regions are stationary target objects, it is obviously meaningless, and therefore the subject to be recognized needs to be a dynamic object to make this picture taking and displaying process meaningful. Therefore, when the subject region is identified, it is possible to identify a region in which a dynamic change exists in the region, and to set the region in which a dynamic change exists for a predetermined time as the subject region. For example, if several persons eat the area to be photographed, the person's upper body moves in the area, and thus the individual upper body can be used as the main body area. In the case of recognition, a region that dynamically changes within a predetermined time is recognized as a subject region, and for example, a region in which changes occur in three seconds is counted as a subject region.
Step 602: acquiring dynamic data of the subject area and static data of the background area;
after the subject area is identified, other areas can be used as background areas, and when picture information is acquired, dynamic data of the subject area can be acquired, such as: the dynamic data changed within 3s only needs to acquire static information once for the background area. When there are a plurality of body regions, dynamic data of each body region is acquired.
If there are two cameras on the shooting terminal: the first camera and the second camera can acquire the image data of the shot area through the first camera and the second camera in the terminal before the main body area and the background area of the shot area are identified. Specifically, in order to realize effective division of functions, dynamic data of the subject area may be acquired by the first camera, and static data of the background area may be acquired by the second camera.
That is, the first camera may acquire one picture as a picture of the final base image, or may acquire a still picture that is finally displayed, and then acquire the dynamic data of the recognized subject by the second camera. The data of the two are combined to obtain the final dynamic and static combined picture.
Step 603: and synthesizing the dynamic data of the main body area and the static data of the background area into a shot picture.
That is, the final shot picture is obtained by combining the dynamic data and the static data. Because the photo contains both dynamic data and static data, the user can see the dynamic change when the user wants to display the dynamic change. Furthermore, if the visual center of the eyes of the viewer is sensed, the automatic display area is triggered, so that the main object in the picture can be dynamically played, and the technical effect of seeing where the main object is supposed to move can be achieved.
The invention also provides a photo display and generation device and a method, which are used for synthesizing the shot photo based on the dynamic data of the main body area and the static data of the background area, identifying the main body object aimed by the sight of the user when the user views the photo, and carrying out dynamic form on the main body object aimed by the sight of the user, thereby improving the diversity of the picture presentation.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A photo display device, characterized in that the device comprises:
the first determination module is used for determining an area where the sight of the user stays on the photo;
the second determining module is used for determining a main object of the area where the sight of the user stays on the photo;
the display module is used for dynamically presenting the main object in the area where the determined sight line stays, wherein the photo is formed by identifying the main object and the background when being shot and integrating dynamic information of the main object and static information of the background;
the second determining module includes:
and a first determination unit configured to determine a coordinate position of a line of sight of a user in the photograph when a plurality of subject objects exist on the photograph.
2. The apparatus of claim 1, wherein the second determining module further comprises:
a second determination unit for determining one or more subject objects including the coordinate position within the photograph;
and the generating unit is used for taking one or more main objects including the coordinate positions in the determined photo as the main objects of the determined area where the sight of the user stays on the photo.
3. The apparatus of claim 1, wherein the first determining module comprises:
the sensing unit is used for sensing the visual center of the eyes of the user through a front camera on the terminal;
and the generating unit is used for taking the sensed area within the preset range of the visual center of the eyes of the user as the area where the sight of the user stays on the photo.
4. A photograph generation apparatus, characterized in that the apparatus comprises:
the identification module is used for identifying a main body area and a background area of the shot area;
the first acquisition module is used for acquiring dynamic data of the main body area and static data of the background area;
the synthesis module is used for synthesizing the dynamic data of the main body area and the static data of the background area into a shot picture;
and the identification module is also used for identifying a dynamically changed area in the shooting area, and taking the dynamically changed area in a preset time as a main body area.
5. The apparatus of claim 4, further comprising:
and the second acquisition module is used for acquiring the image data of the shot area through the first camera and the second camera in the terminal before the main body area and the background area of the shot area are identified.
6. The apparatus of claim 5, wherein the first obtaining module comprises:
a first acquisition unit, configured to acquire dynamic data of the subject area through the first camera;
and the second acquisition unit is used for acquiring the static data of the background area through the second camera.
7. A terminal, comprising: the photo display device of any one of claims 1 to 3, and/or the photo generation device of any one of claims 4 to 6.
8. A method of displaying a photograph, the method comprising:
determining an area where the user's gaze stays on the photograph;
determining a main object of a region where the sight of a user stays on the photo;
dynamically presenting a main object in an area where the determined sight line stays, wherein the picture is formed by identifying the main object and the background when being shot and integrating dynamic information of the main object and static information of the background;
when a plurality of subjects exist on the photograph, determining a subject in which the line of sight of the user stays in the photograph includes:
determining the coordinate position of the sight of the user in the photo;
determining one or more subject objects within the photograph that include the coordinate locations.
9. The method of claim 8, wherein determining the subject in the area where the line of sight of the user stays on the photograph when a plurality of subjects exist on the photograph further comprises:
and one or more objects including the coordinate positions in the determined photo are taken as the objects of the regions where the determined sight line of the user stays on the photo.
10. A method of generating a photograph, the method comprising:
identifying a subject area and a background area of a shot area;
acquiring dynamic data of the subject area and static data of the background area;
synthesizing the dynamic data of the subject area and the static data of the background area into a shot picture;
and identifying a dynamically changing area in the shooting area, and taking the area with the dynamic change in a preset time as a subject area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710206286.2A CN106851114B (en) | 2017-03-31 | 2017-03-31 | Photo display device, photo generation device, photo display method, photo generation method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710206286.2A CN106851114B (en) | 2017-03-31 | 2017-03-31 | Photo display device, photo generation device, photo display method, photo generation method and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106851114A CN106851114A (en) | 2017-06-13 |
| CN106851114B true CN106851114B (en) | 2020-02-18 |
Family
ID=59142817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710206286.2A Active CN106851114B (en) | 2017-03-31 | 2017-03-31 | Photo display device, photo generation device, photo display method, photo generation method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106851114B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767430A (en) * | 2017-09-21 | 2018-03-06 | 努比亚技术有限公司 | One kind shooting processing method, terminal and computer-readable recording medium |
| CN111083553B (en) * | 2019-12-31 | 2021-08-17 | 联想(北京)有限公司 | Image processing method and image output equipment |
| CN113849142B (en) * | 2021-09-26 | 2024-05-28 | 深圳市火乐科技发展有限公司 | Image display method, device, electronic equipment and computer readable storage medium |
| CN117037279A (en) * | 2023-08-25 | 2023-11-10 | 北京智芯微电子科技有限公司 | Action detection and counting method and device, heterogeneous chip, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102110369A (en) * | 2010-12-21 | 2011-06-29 | 汉王科技股份有限公司 | Jaywalking snapshot method and device |
| CN104954689A (en) * | 2015-06-30 | 2015-09-30 | 努比亚技术有限公司 | Method and shooting device for acquiring photo through double cameras |
| CN106162355A (en) * | 2015-04-10 | 2016-11-23 | 北京云创视界科技有限公司 | video interactive method and terminal |
| CN106453864A (en) * | 2016-09-26 | 2017-02-22 | 广东欧珀移动通信有限公司 | Image processing method and device and terminal |
-
2017
- 2017-03-31 CN CN201710206286.2A patent/CN106851114B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102110369A (en) * | 2010-12-21 | 2011-06-29 | 汉王科技股份有限公司 | Jaywalking snapshot method and device |
| CN106162355A (en) * | 2015-04-10 | 2016-11-23 | 北京云创视界科技有限公司 | video interactive method and terminal |
| CN104954689A (en) * | 2015-06-30 | 2015-09-30 | 努比亚技术有限公司 | Method and shooting device for acquiring photo through double cameras |
| CN106453864A (en) * | 2016-09-26 | 2017-02-22 | 广东欧珀移动通信有限公司 | Image processing method and device and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106851114A (en) | 2017-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106454121B (en) | Double-camera shooting method and device | |
| CN106909274B (en) | Image display method and device | |
| CN105468158B (en) | Color adjustment method and mobile terminal | |
| CN106713716B (en) | Shooting control method and device for double cameras | |
| CN105303398B (en) | Information display method and system | |
| CN106982273B (en) | Mobile terminal and control method thereof | |
| CN106200991B (en) | Angle adjusting method and device and mobile terminal | |
| CN106657782B (en) | Picture processing method and terminal | |
| CN110928708B (en) | Icon display method and device, electronic equipment and computer readable storage medium | |
| CN106851114B (en) | Photo display device, photo generation device, photo display method, photo generation method and terminal | |
| CN106911881B (en) | Dynamic photo shooting device and method based on double cameras and terminal | |
| CN106598538B (en) | Instruction set updating method and system | |
| CN106131327B (en) | Terminal and image acquisition method | |
| CN106993134B (en) | Image generation device and method and terminal | |
| CN106250130B (en) | Mobile terminal and method for responding key operation | |
| CN107018326B (en) | Shooting method and device | |
| CN106937056B (en) | Focusing processing method and device for double cameras and mobile terminal | |
| CN106791119B (en) | Photo processing method and device and terminal | |
| CN106161790B (en) | Mobile terminal and control method thereof | |
| CN105955474B (en) | Application evaluation prompting method and mobile terminal | |
| CN106990896B (en) | Stereo photo display method and device based on double cameras and mobile terminal | |
| CN104639428B (en) | Self-adaptive method for session scene in instant messaging and mobile terminal | |
| CN107071275B (en) | Image synthesis method and terminal | |
| CN106951350B (en) | Method and device for checking mobile terminal disk | |
| CN107197084B (en) | Method for projection between mobile terminals and first mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |