WO2020077494A1

WO2020077494A1 - Intelligent photographing method and system, and related device

Info

Publication number: WO2020077494A1
Application number: PCT/CN2018/110247
Authority: WO
Inventors: 康凤霞; 陈绍君; 陈浩; 周胜丰
Original assignee: 华为技术有限公司
Priority date: 2018-10-15
Filing date: 2018-10-15
Publication date: 2020-04-23
Also published as: CN112840635A; EP3846439A1; US20210385370A1; US11470246B2; EP3846439A4

Abstract

Provided is an intelligent photographing method. The method comprises: first, a terminal extracts one or more first tags in general data of a user, the general data being used for representing identity features of the user. The terminal extracts one or more second tags in photography related data, the photography related data being used for representing photographing preferences of the user. Then, the terminal determines a third tag according to the one or more first tags and the one or more second tags. At last, the terminal adjusts, according to a set of image quality effect parameters corresponding to the third tag, the image quality of an image captured by the terminal. In this way, the terminal can provide with a user a photography effect that matches the personal preferences of the user, thereby improving the user experience.

Description

Intelligent photographing method, system and related device

Technical field

This application relates to the field of electronic technology, in particular to an intelligent photographing method, system and related device.

Background technique

With the development of artificial intelligence technology, smart terminal devices have also been continuously developed. Using smart terminal devices to take pictures and record bits and pieces of life has become a way of life for people. Therefore, people take photos of smart terminal devices The picture effect of the photos is also getting more and more attention.

At present, after the shooting is completed, the smart terminal device analyzes the image object and the environment before the image format is formed, and then performs the corresponding beautification processing according to the analysis result, and then further forms the beautified image data into the picture format . However, the processing of the picture data by the intelligent terminal device is aimed at all users, and cannot meet the personalized preferences and demands of different users for the photographing effect.

Summary of the invention

The present application provides an intelligent photographing method, system and related device, which provides a user with a photographing effect that matches the user's personal preferences and improves the user's experience.

In a first aspect, the present application provides an intelligent photographing method, which includes: first, a terminal extracts one or more first tags in a user's general data; the general data is used to characterize the user's identity characteristics. Then, the terminal extracts one or more second tags in the user's shooting related data. Next, the shooting-related data is used to characterize the user's shooting preferences. Next, the terminal determines a third label based on the one or more first labels and the one or more second labels. Finally, the terminal adjusts the image quality of the image captured by the terminal according to the image quality effect parameter set corresponding to the third label.

Through the embodiment of the present application, the terminal can collect user data, extract a first tag representing the user's identity characteristics, extract a second tag representing the user's shooting preferences, and merge the user's third according to the first tag and the second tag The label, that is, the third label combines the user's identity characteristics and the user's shooting preferences. Next, the terminal uses the image quality effect parameter set corresponding to the third label to assist the user to take a photographing effect that matches the user's characteristics, to provide the user with a photographing effect that matches the user's personality, and to improve the user experience.

In a possible situation, the terminal extracting one or more first tags in the general data may specifically include: the terminal extracts one or more first tags corresponding to the general data according to the first mapping relationship Label; wherein, the first mapping relationship includes mapping of multiple sets of common data with multiple first labels. In this way, the terminal can match the general data with the first mapping relationship to obtain the user's general feature label, that is, the first label, so that the terminal can quickly extract the user's general feature label.

In a possible case, the terminal extracts one or more first tags in the shooting related data, which may specifically include: First, the terminal extracts one or more first shooting related parameters from the shooting related data set. Then, the terminal inputs the one or more first shooting-related parameter sets into the first neural network model to obtain the one or more first score vector sets; wherein the first score vector set includes multiple fourth The first score of each tag. The first score is used to characterize the degree of matching between the first shooting-related parameter set and the fourth tag. Then, the terminal determines the one or more second tags from the plurality of fourth tags according to the first score vector set corresponding to each of the one or more first shooting related parameter sets. That is to say, the terminal can use the neural network model to extract the feature labels in the shooting related data, so that the terminal can use the self-learning ability of the neural network model to improve the accuracy of the terminal to extract the feature labels in the shooting related data .

In a possible case, the one or more second tags include a first set of score vectors corresponding to each of the one or more sets of first shooting-related parameters, and the first score is greater than a first threshold One or more fourth labels. That is to say, the terminal may determine one or more fourth tags with a fourth score greater than the first threshold as one or more second tags, because the size of the first score is used to indicate that the user and the second tag The matching degree of the four tags, the larger the first score, the higher the matching degree of the user's shooting related data and the fourth tag, so that the terminal can extract one or more second tags that match the characteristics of the user's shooting related data.

In a possible case, the one or more second tags include a first set of score vectors corresponding to each of the first shooting-related parameter sets, and one or more fourth tags with the highest first score. That is to say, the terminal extracts one or more fourth labels with the highest first score in each first score vector set of the user, and determines one or the second label above, so that the terminal can improve the extraction of a user Or the accuracy of multiple second labels.

In a possible situation, before the terminal inputs the one or more first shooting-related parameter sets into the first neural network model, the terminal may obtain sample data; the sample data includes multiple sets of first training sets, Wherein, each set of first training set includes a set of second shooting related parameter set and a set of second score vector set. The terminal trains the first neural network model through a deep learning algorithm based on the sample data. That is to say, the terminal can continuously train the neural network model according to the sample data, so that the terminal can improve the accuracy of extracting one or more second labels of the user.

In a possible case, the terminal displays a first interface, and the first interface includes multiple sample pictures. Each sample picture corresponds to a set of the second shooting related parameter set and a set of the second score vector set; the second shooting related parameter set is used to characterize the image quality of the sample picture, the second grouping The vector set includes the first score of each of the plurality of fourth tags corresponding to the sample picture. The terminal receives the user's first input operation of selecting one or more training pictures from the plurality of sample pictures. In response to the first input operation, the terminal may determine the second shooting-related parameter set and the second score vector set corresponding to the one or more training pictures as the sample data. That is to say, the terminal can train the first neural network model through the sample data corresponding to the sample image preselected by the user, so that the terminal can extract one or more second feature tags that meet the user's personalized shooting preferences.

In a possible case, the terminal can determine whether the number of sample pictures is less than the number of trainings. If so, the terminal takes one or more sets of the second shooting-related parameter set from the pre-stored training set database and The second set of score vectors is used as the sample data. That is to say, the terminal can use the pre-stored training set to train the first neural network when the number of sample images selected by the user is insufficient, which reduces the user's input operation and improves the user experience.

In a possible situation, each first label and each second label jointly have an associated score. The size of the association score is used to characterize the degree of association between the first label and the second label. The method specifically includes: first, the terminal may determine the total association score of each second label according to the one or more first labels and the one or more second labels

Where T _i is the total relevance score of the i-th second label among the one or more second labels, L _{1 is} the weight of the one or more first labels, and L _{2 is} the one The weight of the second label or multiple labels, where W _k is the association score corresponding to the k-th first label and the i-th second label of the one or more first labels, and R is the one Or the number of multiple first labels. Then, the terminal determines the third label according to the total association score of each second label, where the third label is the one with the highest total association score among the one or more second labels. That is to say, the terminal may set the weight value for the first label and the second label of the user, and set the corresponding degree of association value for each first label and each second label. In this way, the terminal can make the image quality adjustment parameters recommended by the terminal to the user more in line with the user's personalized preferences and improve the user experience.

In a second aspect, the present application provides a terminal, including one or more processors and one or more memories. The one or more memories are coupled to one or more processors. The one or more memories are used to store computer program code. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the communication device is executed. An intelligent photographing method in any possible implementation manner of any one of the above aspects.

In a third aspect, an embodiment of the present application provides a computer storage medium, including computer instructions, which, when the computer instructions run on an electronic device, cause a communication device to execute the smart photographing method in any possible implementation manner of any of the above aspects .

According to a fourth aspect, an embodiment of the present application provides a computer program product, which, when the computer program product runs on a computer, causes the computer to execute the smart photographing method in any possible implementation manner of any one of the above aspects.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art.

FIG. 1 is a schematic structural diagram of a terminal provided by an embodiment of this application;

2 is a schematic diagram of a software architecture provided by an embodiment of the present application;

3 is a schematic diagram of a set of interfaces provided by an embodiment of the present application;

4 is a schematic diagram of another group of interfaces provided by an embodiment of the present application;

5 is a schematic diagram of another group of interfaces provided by an embodiment of the present application;

6 is a schematic diagram of another group of interfaces provided by an embodiment of the present application;

7 is a schematic structural diagram of an intelligent camera system provided by an embodiment of the present application;

8 is a schematic flowchart of a user data preprocessing process provided by an embodiment of this application;

9 is a schematic flowchart of extracting a user's general data features provided by an embodiment of the present application;

FIG. 10 is a schematic flowchart of extracting features of a user ’s shooting related data provided by an embodiment of the present application;

11 is a schematic flowchart of a feature label fusion provided by an embodiment of this application;

12 is a schematic flowchart of setting a parameter when a terminal takes a picture provided by an embodiment of the present application;

13 is a schematic structural diagram of a user database provided by an embodiment of this application;

FIG. 14 is a schematic flowchart of an intelligent photographing method provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the drawings. In the description of the embodiments of the present application, unless otherwise stated, “/” means or, for example, A / B may mean A or B; “and / or” in the text is merely a description of the related object The association relationship indicates that there can be three kinds of relationships, for example, A and / or B, which can indicate that there are three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, “plurality” refers to two or more than two.

In the following, the terms "first" and "second" are used for description purposes only and cannot be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise stated, the meaning of "plurality" is two or more.

FIG. 1 shows a schematic structural diagram of the terminal 100.

As shown in FIG. 1, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, and a battery 142 , Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193 , A display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or less components than shown, or combine some components, or split some components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), and an image signal processor. (image) signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural-network processing unit (NPU) Wait. Among them, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the terminal 100. The controller can generate the operation control signal according to the instruction operation code and the timing signal to complete the control of fetching instructions and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. The repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. Interfaces can include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit, sound, I2S) interface, pulse code modulation (pulse code modulation (PCM) interface, universal asynchronous transceiver (universal asynchronous) receiver / transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input / output (GPIO) interface, subscriber identity module (SIM) interface, and / Or universal serial bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the terminal 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, to realize the function of answering the phone call through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to realize the function of answering the call through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 may transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI) and so on. In some embodiments, the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the terminal 100. The processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the terminal 100.

The GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the terminal 100, or may be used to transfer data between the terminal 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only a schematic description, and does not constitute a limitation on the structure of the terminal 100. In some other embodiments of the present application, the terminal 100 may also use different interface connection methods in the foregoing embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the terminal 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the terminal 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a wireless communication solution including 2G / 3G / 4G / 5G and the like applied to the terminal 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), and the like. The mobile communication module 150 can receive the electromagnetic wave from the antenna 1, filter and amplify the received electromagnetic wave, and transmit it to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it to electromagnetic wave radiation through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110, and may be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (bluetooth, BT), and global navigation satellite systems that are applied to the terminal 100. (global navigation system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives the electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 may also receive the signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and convert it to electromagnetic waves through the antenna 2 to radiate it out.

In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global mobile communication system (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation system (BDS), and a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and / or satellite-based augmentation system (SBAS).

The terminal 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations, and is used for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel may use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (active-matrix organic light) emitting diode, AMOLED), flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal 100 can realize a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP processes the data fed back by the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also optimize the algorithm of image noise, brightness and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be set in the camera 193.

The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The digital signal processor is used to process digital signals. In addition to digital image signals, it can also process other digital signals. For example, when the terminal 100 is selected at a frequency point, the digital signal processor is used to perform Fourier transform on the energy at the frequency point.

Video codec is used to compress or decompress digital video. The terminal 100 may support one or more video codecs. In this way, the terminal 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (moving picture experts, MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, the input information can be processed quickly, and it can also continuously learn by itself. The NPU can realize applications such as intelligent recognition of the terminal 100, such as image recognition, face recognition, voice recognition, and text understanding.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to achieve expansion of the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the terminal 100. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area may store an operating system, at least one function required application programs (such as sound playback function, image playback function, etc.) and so on. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the terminal 100 and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and so on.

The terminal 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, and an application processor. For example, music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The terminal 100 may listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also known as "handset", is used to convert audio electrical signals into sound signals. When the terminal 100 answers a call or voice message, it can answer the voice by holding the receiver 170B close to the ear.

Microphone 170C, also known as "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a person's mouth, and input a sound signal to the microphone 170C. The terminal 100 may be provided with at least one microphone 170C. In other embodiments, the terminal 100 may be provided with two microphones 170C. In addition to collecting sound signals, it may also implement a noise reduction function. In other embodiments, the terminal 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headset interface 170D is used to connect wired headsets. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile electronic device (open mobile terminal) platform (OMTP) standard interface, the American Telecommunications Industry Association (cellular telecommunications industry association of the United States, CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The capacitive pressure sensor may be a parallel plate including at least two conductive materials. When force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the terminal 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal 100 may calculate the touched position based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the movement posture of the terminal 100. In some embodiments, the angular velocity of the terminal 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for shooting anti-shake. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the terminal 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shaking of the terminal 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates the altitude by using the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The terminal 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the terminal 100 is a clamshell machine, the terminal 100 may detect the opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the holster or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three axes). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the posture of electronic devices, and be used in applications such as horizontal and vertical screen switching and pedometers.

The distance sensor 180F is used to measure the distance. The terminal 100 can measure the distance by infrared or laser. In some embodiments, when shooting scenes, the terminal 100 may use the distance sensor 180F to measure distance to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal 100 emits infrared light outward through the light emitting diode. The terminal 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the terminal 100. When insufficient reflected light is detected, the terminal 100 may determine that there is no object near the terminal 100. The terminal 100 can use the proximity light sensor 180G to detect that the user is holding the terminal 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the brightness of ambient light. The terminal 100 may adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 100 is in a pocket to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to unlock the fingerprint, access the application lock, take a picture of the fingerprint, and answer the call with the fingerprint.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the terminal 100 performs to reduce the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In some other embodiments, when the temperature is below another threshold, the terminal 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

Touch sensor 180K, also known as "touch panel". The touch sensor 180K may be provided on the display screen 194, and the touch sensor 180K and the display screen 194 constitute a touch screen, also called a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal 100, which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human body part. The bone conduction sensor 180M can also contact the pulse of the human body and receive a blood pressure beating signal. In some embodiments, the bone conduction sensor 180M may also be provided in the earphone and combined into a bone conduction earphone. The audio module 170 may parse out the voice signal based on the vibration signal of the vibrating bone block of the voice part acquired by the bone conduction sensor 180M to realize the voice function. The application processor may analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M to implement the heart rate detection function.

The key 190 includes a power-on key, a volume key, and the like. The key 190 may be a mechanical key. It can also be a touch button. The terminal 100 may receive key input and generate key signal input related to user settings and function control of the terminal 100.

The motor 191 may generate a vibration prompt. The motor 191 can be used for vibration notification of incoming calls and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. For the touch operation in different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effect can also support customization.

The indicator 192 may be an indicator light, which may be used to indicate a charging state, a power change, and may also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the terminal 100. The terminal 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal 100 interacts with the network through the SIM card to realize functions such as call and data communication. In some embodiments, the terminal 100 uses eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the terminal 100 and cannot be separated from the terminal 100.

The software system of the terminal 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes an Android system with a layered architecture as an example to exemplarily explain the software structure of the terminal 100.

2 is a block diagram of the software structure of the terminal 100 according to an embodiment of the present invention.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.

The application layer may include a series of application packages.

As shown in FIG. 2, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The application framework layer provides an application programming interface (application programming interface) and programming framework for applications at the application layer. The application framework layer includes some predefined functions.

As shown in FIG. 2, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and so on.

The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, intercept the screen, etc.

Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text and controls for displaying pictures. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including an SMS notification icon may include a view that displays text and a view that displays pictures.

The phone manager is used to provide the communication function of the terminal 100. For example, the management of the call state (including connection, hang up, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminders, etc. The notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, the text message is displayed in the status bar, a prompt sound is emitted, the electronic device vibrates, and the indicator light flashes.

Android Runtime includes core library and virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

The core library contains two parts: one part is the function function that Java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

The system library may include multiple functional modules. For example: surface manager (surface manager), media library (Media library), 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

In the following, the workflow of software and hardware of the terminal 100 will be exemplarily described in conjunction with capturing a photographing scene.

When the touch sensor 180K receives the touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into original input events (including touch coordinates, time stamps and other information of touch operations). The original input events are stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, for example, the control corresponding to the click operation is a camera application icon. The camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer. The camera 193 captures a still image or video.

The smart photographing method provided by the embodiments of the present application will be described in detail below with reference to the drawings and application scenarios. In the following embodiments of the present application, the terminal may be the terminal 100 shown in FIG. 1 or FIG. 2 described above.

The terminal may receive user input and enable or disable the collection of user data required by the smart camera function in the embodiments of the present application. Please refer to FIG. 3, 3a in FIG. 3 shows a setting interface 310 displayed on the touch screen of the terminal. The setting interface 310 may include a system setting bar 311 and other setting bars (such as a sound setting bar, a notification center setting bar, an application management setting bar, a battery setting bar, a storage setting bar, a security and privacy setting bar, user and account settings) Column, etc.). The terminal may receive a user input operation 312 (for example, click) to the system setting field 311, and in response to the input operation 312 (for example, click), the terminal may display the system setting interface 320 as shown in 3b in FIG. 3.

As shown in 3b of FIG. 3, the system setting interface 320 may include a smart ability enhancement setting bar 321 and other setting bars (such as a phone setting bar, a system update setting bar, a system navigation setting bar, a language and input method setting bar , Date and time setting bar, easy mode setting bar, data migration setting bar, backup and recovery setting bar, reset setting bar, user experience improvement plan setting bar and certification logo setting bar, etc.). The terminal may receive a user's input operation 322 (for example, click) to the smart ability enhancement setting field 321, and in response to the input operation 322 (for example, click), the terminal may display the smart ability enhancement setting as shown in 3c in FIG. 3 Interface 330.

As shown in 3c in FIG. 3, the smart ability enhancement setting interface 330 may include a smart suggestion setting column 331 and other setting columns (for example, a function introduction and about of the smart ability enhancement). Among them, the smart suggestion setting field 331 is associated with a smart suggestion setting control 332. In 3c of FIG. 3, the smart suggestion setting control 332 is turned off, and the terminal closes the collection of user data on the terminal. The terminal may receive a user's input operation 333 (for example, click) on the smart suggestion setting control 332, and in response to the input operation 333, the terminal may switch the smart suggestion setting control 332 from the closed state to the open state, and start the smart advice When smart advice is turned on, the terminal may collect user data required by the smart camera function in the embodiment of the present application. Among them, the user data includes general data and shooting related data. Among them, the general data includes personal basic information, behavior habits, interests and hobbies that can include the user. The shooting-related data may include the user's shooting preferences, browsing picture habits, and so on.

After the terminal collects the user data, the terminal can preprocess the user data. The preprocessed user data can be stored in the user's database, which can be local to the terminal or a remote server.

The terminal may pop up a user preference survey interface in the camera application. The user preference survey interface includes one or more pictures, and the terminal may receive a user's selection operation on the picture in the user preference survey interface (for example, click on the picture), and respond to the user For the selection operation of the picture in the user preference survey interface (for example, clicking on the picture), the terminal may use the camera-related parameters corresponding to the picture selected by the user and the shooting feature label score vector set corresponding to the picture selected by the user.

For example description, please refer to FIG. 4, where 4a shows the main interface 410 displayed on the touch screen of the terminal. The main interface 410 may include icons 411 of the camera application and icons of other applications (such as Alipay, notepad, music, WeChat, settings, dialing, information, contacts, etc.). The terminal may receive a user's input operation 412 (for example, click) on the icon 411 of the camera application, and in response to the input operation 412, the terminal may turn on a camera (for example, a front camera or a rear camera) and display it on the touch screen The camera shooting interface 420 shown in 4b in FIG. 4 is displayed.

As shown in 4b in FIG. 4, the camera shooting interface 420 may include a camera capture display area 423, a camera setting button 421, a photograph button 425, and the like. The camera capture display area 423 is used to display the screen captured by the camera (front camera or rear camera). The terminal may receive a user's input operation 422 (eg, click) to the camera setting button 421, and in response to the input operation 422, the terminal may display the camera setting interface 430 as shown in 4c in FIG. 4.

As shown in 4c in FIG. 4, the camera shooting interface 430 may include an intelligent auxiliary photograph setting bar 431 and other setting bars (such as a resolution setting bar, a geographic location setting bar, an automatic watermark setting bar, a voice control setting bar, a reference line Setting bar, glove mode setting bar, photo mute setting bar, timer photo setting bar, voice control camera setting bar, etc.). Wherein, the state of the intelligent auxiliary photographing setting field 431 is closed, that is, the terminal disables the intelligent photographing function provided by the embodiment of the present application when photographing. The terminal may receive a user's input operation to the smart auxiliary photographing setting field 431, and in response to the user's input operation to the user assisting photographing setting field, the terminal may turn on the smart photographing function provided by the embodiment of the present application. In a possible situation, when the terminal receives the user's input operation 432 (for example, click) on the smart assisted photography setting field 431, in response to the user's first input operation 432 (for example, click), the terminal may display The user preference survey interface 440 shown in 4d in FIG. 4.

As shown in 4d in FIG. 4, the user preference survey interface 440 may include multiple groups (for example, 10 groups) of pictures, and each group of pictures may include multiple (for example, 4) pictures. The content of the pictures in each group of pictures is the same, but the shooting related parameter sets P corresponding to different pictures in each group of pictures are different, and each picture corresponds to a shooting feature label (such as strong beauty, weak beauty, small freshness, Japanese Equal) score vector set S. Among them, the shooting related parameter set P includes the shooting parameter set of the picture and the picture quality (PQ) effect parameter set of the picture. The shooting parameter set can be {a1, a2, a3, ...}. The PQ effect parameters can be {b1, b2, b3, ...}.

Exemplarily, the shooting parameter set may be {white balance (a1), ISO (a2), exposure compensation (a3), shutter speed (a4), focus mode (a5), metering mode (a6), brightness (a7) , Saturation (a8), contrast (a9), sharpness (a10), ...}. The PQ effect parameter set can be used for the terminal to adjust the PQ effect of the picture, for example, contrast adjustment, brightness adjustment, color saturation adjustment, hue adjustment, sharpness adjustment (such as digital noise reduction (DNR) adjustment), Image quality adjustments such as color edge enhancement (chroma TI, CTI) adjustments. This example is only for explaining this application, and should not constitute a limitation.

For example, the shooting feature label score vector set S may be {strong beauty face score, weak beauty face score, small fresh score, Japanese score}. As shown in 4d in FIG. 4, the user preference interface 440 displays the first group of pictures, and the first group of pictures includes picture a, picture b, picture c, and picture d. Among them, the shooting-related parameter sets corresponding to the four pictures, and the corresponding shooting feature label score vector set S can be shown in Table 1 below:

Table 1 The shooting related parameter set and shooting feature label score vector set S corresponding to the effect picture

It can be seen from Table 1 above that the photograph-related parameters corresponding to the picture a are P_a, and the shooting feature label score vector set S_a corresponding to the picture a is {0.6, 0, 0.2, 0.2}, which indicates the score of the strong beauty in the picture a The value is 0.6, the weak beauty score is 0, the small fresh score is 0.2, and the Japanese score is 0.2. The relevant parameter set of the photograph corresponding to the picture b is P_b, and the score vector set S_b of the shooting feature label corresponding to the picture b is {0, 0.2, 0.4, 0.4}, which means that the score of the strong beauty in the picture b is 0, the weak beauty The score of Yan is 0.2, the score of Xiaoqing is 0.4, and the score of Japanese is 0.4. The picture-related parameter set corresponding to picture c is P_c, and the shooting feature label score vector set S_c corresponding to picture c is {0, 0.2, 0.8, 0}, which means that the score of strong beauty in picture c is 0, weak beauty The score of Yan is 0.2, the score of Xiaoqing is 0.8, and the score of Japanese is 0. The picture-related parameter set corresponding to picture d is P_d, and the shooting feature label score vector set S_d corresponding to picture d is {0, 0.2, 0.2, 0.6}, which means that the score of strong beauty in picture d is 0, weak beauty The score of Yan is 0.2, the score of Xiaoqing is 0.2, and the score of Japanese is 0.6. The example shown in Table 1 above is only used to explain the present application, and should not constitute a limitation.

Among them, the sum of the scores of the shooting feature tags of the effect picture can be 1, and the higher the score of a shooting feature tag in the effect picture, it means that the shooting related parameter set corresponding to the effect picture and the shooting related parameter corresponding to the shooting feature tag The higher the matching degree of the set, that is, the higher the score of the shooting feature label, the more the feature characterized by the user's shooting related parameters.

After the terminal receives the effect picture selected by the user, the terminal can record the shooting related parameter set P (eg shooting related parameter set P_b) corresponding to the user selected effect picture (eg picture b), and the corresponding effect picture selected by the user Shooting feature label score vector set S (for example, shooting feature label score vector set S_b), and shooting related parameter set P (for example, shooting related parameter set P_b) and shooting feature label score vector set S (for example, shooting feature label The score vector set S_b) is the training set Q {P → S} of the neural network model (for example {P_b → S_b}), which is trained using the deep learning algorithm to obtain the shooting related parameter set P and the shooting feature label score vector set S Mapping function f (x). Among them, in the mapping function f (x) of the neural network model, the shooting related parameter set P is taken as an input, and the shooting feature label score vector set S is taken as an output. Among them, the terminal can input the shooting related parameter set P corresponding to the multiple effect pictures selected by the user and the training parameter set Q composed of the shooting feature label score vector set S into the neural network model for training to obtain more in line with the user Favorite mapping function f (x). For a specific implementation process of the mapping function f (x) for training the neural network model, reference may be made to the neural network training process in the embodiment shown in FIG. 10 below, and details are not described herein again.

Exemplarily, as shown in 4d in FIG. 4, the terminal may receive a user input operation 442 for an effect picture (for example, picture b), and in response to the input operation 442, the terminal may set the shooting-related parameter set P_b corresponding to the picture b, And the shooting feature label score set S_b corresponding to the picture b as a training set Q_1 {P_b → S_b}, and input the training set Q_1 {P_b → S_b} into the neural network model, and use the deep learning algorithm to train, Obtain the mapping function f (x).

In a possible situation, the terminal may receive an operation from the user to the user to open the camera application when the user first receives the user's operation to open the camera application or every user preference survey period (for example, the terminal investigates the user's photography preferences every 10 days) At this time, a user preference survey interface pops up to collect the camera-related parameters P corresponding to the picture selected by the user and the shooting feature label score vector set S corresponding to the picture selected by the user.

For example, please refer to FIG. 5, 5 a in FIG. 5 shows the main interface 510 displayed on the touch screen of the terminal. The main interface 510 may include icons 511 of the camera application and icons of other applications (such as Alipay, notepad, music, WeChat, settings, dialing, information, contacts, etc.). The terminal may receive a user's input operation 512 (for example, click) on the icon 511 of the camera application, and in response to the input operation 512, the terminal may turn on a camera (for example, a front camera or a rear camera) and display it on the touch screen The camera shooting interface 520 shown in 5b in FIG. 5, and a user preference survey interface 530 pops up in the camera shooting interface 520.

As shown in 5b in FIG. 5, a user preference survey interface 530 may pop up in the camera shooting interface 520. The user preference survey interface 530 includes multiple groups (for example, 10 groups) of pictures, and each group of pictures may include multiple (for example, 4) pictures. The content of the pictures in each group of pictures is the same, but the shooting related parameter sets P corresponding to different pictures in each group of pictures are different, and each picture corresponds to a shooting feature label (such as strong beauty, weak beauty, small freshness, Japanese Equal) score vector set S. The terminal may receive the user's input operation 532 for the effect picture 531 (for example, picture b), and in response to the input operation 532, the terminal may set the shooting-related parameter set P_b corresponding to the effect picture 531 (for example, picture b) and the effect picture 531 (for example Picture b) The corresponding shooting feature label score set S_b, as a training set Q_1 {P_b → S_b}, and input the training set Q_1 {P_b → S_b} into the neural network, using deep learning algorithm for training to obtain The mapping function f (x).

After the terminal collects user data (including user general data and user shooting related parameters), and uses the training set Q {P → S} to train the mapping function f (x) in the above neural network more times than the preset training At the threshold of the number of times (for example, 10 times), when the user opens the camera application to take a photo, the terminal can use the PQ effect parameter set corresponding to the smart photo tag (such as Xiaoqing) matched by the user for the user to take an image of the photo taken by the user Process and display on the touch screen of the terminal. For the process of matching the smart camera tag (for example, Xiaoqing) for the user by the terminal, reference may be made to the embodiments shown in FIG. 11 and FIG. 12 below, and details are not described herein again.

Exemplarily, please refer to FIG. 6, 6 a in FIG. 6 shows the main interface 510 displayed on the touch screen of the terminal. The main interface 510 may include icons 611 of the camera application and icons of other applications (such as Alipay, notepad, music, WeChat, settings, dialing, information, contacts, etc.). The terminal may receive a user's input operation 612 (for example, click) on the icon 611 of the camera application, and in response to the input operation 612, the terminal may turn on a camera (for example, a front camera or a rear camera) and display it on the touch screen The camera shooting interface 620 shown in 6b in FIG. 6.

As shown in 6b in FIG. 6, the camera shooting interface 620 may display an image 621 captured by a camera (such as a front camera or a rear camera), and the terminal may be a label of a smart camera tag (such as Xiaoqing) that is matched by the user Recommendation button 623, the terminal may receive a user's input operation 624 (for example, click) on the label recommendation button 623, and in response to the input operation 624 (for example, click), the label recommendation button 623 may be switched from a closed state to an open state, The terminal may enable a function of performing image processing on the screen 621 photographed by the terminal using the PQ effect parameter set corresponding to the smart camera tag (for example, Xiaoqing). In a possible situation, the terminal may receive the user's re-input operation (for example, clicking) of the tab recommendation button 623 in the open state, the label recommendation button 623 may be switched from the open state to the closed state, and the terminal may close The PQ effect parameter set corresponding to the smart photo tag (for example, Xiaoqing) performs the image processing on the screen 621 captured by the terminal.

As shown in 6c in FIG. 6, the camera shooting interface 630 may display an image 631 captured by a camera (for example, a front camera or a rear camera), and a label of a smart camera tag (for example, Xiaoqing) that the terminal matches for the user The recommendation button 633, the shooting button 635, etc., in 6c of FIG. 6, the label recommendation button 633 is turned on, that is, the terminal can turn on the PQ effect parameter set corresponding to the smart camera label (for example, Xiaoqing) to capture The screen 631 performs image processing functions. The terminal may receive the user's input operation 636 (for example, click) on the shooting button 635, and in response to the input operation 636 (for example, click), the terminal may use the PQ effect parameter set corresponding to the smart camera tag (for example, Xiaoqing). The image 631 captured by the terminal performs image processing, and saves the image after the image processing in the gallery, where the image after the image processing may be the image 647 shown in 6d of FIG. 6. As shown in 6d in FIG. 6, the terminal may mark an intelligent photo identification 649 (for example, a smart shot) on the image 647 after image processing, and store the image 647 after image processing in a gallery.

The smart camera system provided by the embodiment of the present application is introduced below.

Please refer to FIG. 7, which is a schematic structural diagram of an intelligent camera system provided by an embodiment of the present application. As shown in FIG. 7, the smart camera system 700 may include a system setting module 710, a data collection module 720, a data preprocessing module 730, a data storage module 740, a feature extraction module 750, and a parameter setting module 760. among them,

The system setting module 710 can be used to turn on and off the smart camera function provided by the embodiment of the present application.

The data collection module 720 may be used to periodically collect data information of users on the terminal after the smart camera function is turned on (for example, the collection period may be 10 days, 15 days, 1 month, or longer, etc.). Among them, the user's data information includes the user's personal basic information (gender, year of birth, place of usual residence, etc.), behavioral habits (most commonly used APP, most used time APP, use the app after plugging in headphones, frequent places, bed time, wake up time ), Hobbies (reading preferences, Internet browsing habits), shooting preferences (shooting parameters, shooting content, shooting content), picture browsing habits (shared pictures, deleted pictures, favorite pictures, edited pictures).

The data preprocessing module 730 can be used to preprocess the user data collected by the data collection module 720 to extract valid data. For the pre-processing flow, reference may be made to the data pre-processing flow shown in FIG. 8 below, which will not be repeated here.

The data storage module 740 may be used to construct a database to store user data (general data, photograph-related data) collected by the data collection module 720, and valid data (preprocessed data of user data) processed by the data preprocessing module 730 3. The feature extraction module 750 performs feature value data (feature values corresponding to feature tags) after feature extraction.

The feature extraction module 750 may be used to extract the user's feature tag according to the user data information collected by the data collection module 720. Among them, the user's feature tags include general feature tags and shooting feature tags, and fusion feature tags after the general feature tags and shooting feature tags undergo the feature tag fusion process. For the feature label fusion process, reference may be made to the feature label fusion process shown in FIG. 11 and FIG. 12 below, which will not be repeated here.

The parameter setting module 760 can be used to perform image processing on the screen captured by the camera of the terminal according to the fusion feature label with the highest user score (that is, the smart camera label), and set the The PQ effect parameter corresponding to the fusion feature label with the highest value.

The following specifically describes the process of collecting user data by the terminal in the embodiment of the present application.

The terminal can collect the user's data by means of data embedding, that is, the terminal can listen to events in the running process of the software application, judge and capture when the event that needs attention occurs, and then the terminal can obtain the relevant information of the event and convert the The related information of the event is sorted and stored in the terminal's local database or a remote server. Among them, the events monitored by the terminal can be provided by platforms such as the operating system, browser, application (APP) framework, etc., and can also be customized trigger events based on the basic events (such as clicking on a specific button).

Exemplarily, the terminal can monitor events such as the user clicking the favorite button, delete button, and share button in the gallery APP. When the terminal receives the user ’s wallpaper picture_1 in the wallpaper APP and clicks the favorite button, the terminal can record the wallpaper picture. And store the wallpaper picture_1 as the data of favorite pictures in the terminal local or remote server. If the terminal receives that the user clicks the delete button on the wallpaper picture_2 in the wallpaper APP, the terminal can record the wallpaper picture and store the wallpaper picture_2 as data for deleting the picture in the terminal local or remote server. If the terminal receives the user's wallpaper image _3 in the wallpaper APP and clicks the share button, the terminal can record the wallpaper image and store the wallpaper image _3 as data for deleting the image on the terminal local or remote server. The examples are only for explaining this application and should not be construed as limitations.

Exemplarily, if the terminal wants to count the number of times the user opens the APP and the time the user stays in the APP, the terminal can count the number of times the user opens the APP by monitoring the event that the operating system opens the APP. The terminal successfully opens the APP once and records the number of times the APP is opened; the terminal enters the APP after the receiving user clicks the home button to switch to the background, and the terminal does not count the number of times the APP is opened. The terminal monitors the user's input operation to enter the APP and exits the APP to calculate the duration of the user's access to the APP. The examples are only for explaining this application and should not be construed as limitations.

Using the above-mentioned data burying method, the user data collected by the terminal may include general data and shooting related data.

The general data may include personal basic information, behavior habits, hobbies, etc. of the user. The specific data subtypes can be shown in Table 2 below:

Table 2 General data

As can be seen from Table 2 above,

1. Basic personal information can include gender, year of birth, place of usual residence, etc.

Exemplarily, the terminal may obtain the previously entered gender from the personal information of the terminal system account (for example, Huawei account center of Huawei terminal, Apple account center of Apple terminal (Apple ID), etc.). In a possible situation, the terminal may also perform picture analysis on multiple photos taken by the user's front camera to calculate the user's gender. In a possible situation, the terminal may also call a third-party APP (for example, QQ, WeChat, Taobao, Weibo, etc.) to provide a data access interface for access rights, and obtain the user's gender from the server of the third-party APP. The above method for obtaining the user's gender is only used to explain this application, and should not constitute a limitation. In specific implementation, the user's gender can also be obtained through other methods.

Exemplarily, the terminal may obtain the user's previous personal information in the system account by obtaining personal information from the terminal's system account center (such as Huawei terminal's Huawei account center, Apple terminal's Apple account center (Apple ID), etc.) Fill in the year of birth entered. In a possible situation, the terminal may also call a third-party APP (for example, QQ, WeChat, Taobao, Weibo, etc.) to provide a data access interface for access rights, and obtain the user's birth year from the server of the third-party APP. The above-mentioned method of obtaining the birth year of the user is only used to explain this application, and should not constitute a limitation. In a specific implementation, the birth year of the user can also be obtained through other methods.

Exemplarily, the terminal may fill in the personal information in the system account previously obtained by the user from the personal information of the terminal system account center (such as Huawei terminal's Huawei account center, Apple terminal's Apple account center (Apple ID), etc.) Enter the resident location (for example, the area in the personal information of the Huawei account center, the delivery address in the personal information of the Apple ID center). In a possible situation, the terminal may also call a third-party APP (for example, QQ, WeChat, Taobao, Weibo, Baidu Maps, etc.) to provide a data access interface for access rights, and obtain the user ’s Permanent residence. The above-mentioned method of obtaining the resident location of the user is only used to explain this application, and should not constitute a limitation. In a specific implementation, the user's birth year can also be obtained through other methods.

2. Behavioral habits may include the user's most commonly used APP, the most used APP, the APP used after plugging in headphones, frequent places, bedtime, and wake-up time.

Exemplarily, the terminal may record the usage record of each APP, where the usage record of the APP includes the number of times the APP is opened within a period (such as one day, one week, one month, etc.), and one period (such as one day, one week, one month Etc.) The APP that the terminal uses after plugging in the headset within a period of time (for example, one day, one week, one month, etc.) within the terminal running APP. The terminal can determine the APP that has been opened most frequently in a cycle (such as a day, week, month, etc.) as the most commonly used APP, and the terminal can determine the APP that has the longest running time in a cycle (such as a day, week, month, etc.) For the APP that uses the most time, the terminal may determine the APP that the terminal uses the most after inserting the headset in a cycle (for example, one day, one week, one month, etc.) as the APP that is used after inserting the headset. The above-mentioned most commonly used APPs, the most used APPs, and the APPs obtained after plugging in headphones are used to obtain information only for the purpose of explaining this application, and should not constitute a limitation. In specific implementation, the user ’s most commonly used APPs, Apps that use the most time, apps that are used after plugging in headphones.

Exemplarily, the terminal may simultaneously obtain location information when taking a picture, and record the location and date of the user's picture. Therefore, the terminal can determine the user's frequent places based on the place and time the user took the picture. The frequently visited place of the user may be the number of times the terminal takes pictures on the same place on different dates. For example, the terminal records that the user took pictures at the beach on January 10, 2018; the user took pictures at Nanshan on February 1, 2018; the user took pictures at the mall on March 1, 2018; and the user took pictures on April 2, 2018 at Take photos at the beach; users took photos at the beach on May 1, 2018. The terminal can determine that the user's frequent place is "Beach". In a possible situation, the terminal may also call a third-party APP (for example, Baidu Map, Gaode Map, etc.) to provide a data access interface for access rights, and obtain the user's frequented places from the server of the third-party APP. The above-mentioned way of obtaining the frequent visits of the user is only for explaining the application, and should not be construed as a limitation. In specific implementation, the frequent visits of the user may also be obtained by other means.

Exemplarily, when the terminal is in a bed where the user sleeps, the terminal can detect vibration information (including vibration frequency, vibration amplitude, etc.) and surrounding sound information (sound of the sound) of the bed surface through various sensors (such as a motion sensor, a microphone, etc.) Amplitude, frequency of sound, etc.), because after the user sleeps, the user ’s regular sound and the user ’s breathing or other actions will cause regular movement of the bed surface, therefore, when the terminal determines the vibration information of the bed surface When the vibration law of the bed after the user sleeps is satisfied, and the sound information around the terminal meets the sound law after the user sleeps, the terminal can determine the sleep time of the user. In a possible situation, the terminal may also monitor the user's heart rate, breathing, body temperature, blood pressure, exercise, and other information through auxiliary devices (eg, smart watches, smart bracelets, etc.) to obtain the user's sleeping time. The above method for acquiring the sleeping time of the user is only used to explain the present application, and should not constitute a limitation. In a specific implementation, the sleeping time of the user may also be acquired through other methods.

Exemplarily, the terminal may obtain the alarm time set by the user by accessing the record in the alarm clock application, thereby obtaining the user's wake-up time. In a possible situation, the terminal may detect the time when the user picks up the terminal the earliest each day through the motion sensor, and determine the time when the user picks up the terminal as the user's wake-up time. In a possible situation, the terminal may monitor the time at which the user unlocks the terminal at the earliest every day, and determine the time to unlock the terminal as the user's wake-up time. The above method for obtaining the user's wake-up time is only used to explain the present application, and should not be construed as a limitation. In specific implementation, the user's wake-up time may also be obtained through other methods.

3. Hobbies can include reading preferences, Internet browsing habits, etc.

Exemplarily, the terminal may obtain the user's reading preferences through a reading application on the terminal (for example, Huawei reading on the Huawei terminal, etc.). In a possible situation, the terminal may also call a third-party APP (for example, WeChat reading, QQ reading, etc.) to provide a data access interface for access rights, and obtain the user's frequent places from the server of the third-party APP. The above-mentioned way of obtaining the user's reading preference is only used to explain this application, and should not be construed as a limitation. In a specific implementation, the user's frequented places can also be obtained through other methods.

Exemplarily, the terminal may receive a user's input operation to open the browser (for example, click an icon of the browser application on the main interface of the terminal, enter "open browser" through a voice assistant), and respond to the user's input operation to open the browser , The terminal can display the search page of the browser, the terminal can record the search content entered by the user on the search page of the browser, and extract the keywords of the search content within a period of time (such as one day, one week, one month, etc.) ( For example "Attractions"). The terminal can also record the user's visit URL and extract the user's visit URL type (for example, video website, travel website, shopping website, etc.). Among them, the user's Internet browsing habits may include information such as keywords searched by the user, types of visited URLs, etc. In a specific implementation, the user's Internet browsing habits may also include other information. In a possible situation, the terminal may also call a third-party APP (for example, Weibo, Baidu search, etc.) to provide a data access interface for access rights, and obtain the user's Internet browsing habits from the server of the third-party APP. The above method for obtaining the user's Internet browsing habits is only used to explain this application, and should not be construed as a limitation. In specific implementation, the user's Internet browsing habits can also be obtained through other methods.

The shooting-related data may include the user's shooting preferences, browsing picture habits, and so on. The specific data types can be shown in Table 3 below:

Table 3 Shooting related data

As can be seen from Table 3 above, shooting preferences include shooting parameters, shooting modes, shooting content, etc. Picture browsing habits include shared pictures, deleted pictures, favorite pictures, edited pictures, etc. among them,

1. Shooting parameters can include white balance, sensitivity (international standards), exposure compensation, shutter speed, focus mode, metering mode, brightness, saturation, contrast, sharpness, etc.

Exemplarily, the terminal may receive a user's input operation to open the camera application (for example, click a camera application icon on the terminal's main interface, and input "open camera" through a voice assistant), and in response to the input operation to open the camera, the terminal may enable the camera And the screen captured by the camera is displayed on the touch screen. When the terminal receives the user's input operation for setting shooting parameters, in response to the user's input operation for setting shooting parameters, the terminal can record and collect the user's shooting parameters. Exemplarily, the terminal may extract the shooting parameters in the above pictures by performing picture analysis on the pictures shared by the user, deleted pictures, favorite pictures, and edited pictures. The above-mentioned user's shooting parameter acquisition method is only used to explain this application, and should not constitute a limitation. In a specific implementation, the user's shooting parameter can also be acquired through other methods.

Exemplarily, the user's shooting parameters acquired by the terminal may be as shown in Table 4 below:

Table 4 User's shooting parameters

拍摄参数Shooting parameters	数据据值Data value
白平衡White balance	2400K2400K
ISOISO	100100
曝光补偿Exposure compensation	+0.5EV+ 0.5EV

快门速度 Shutter speed	1/125s1 / 125s
对焦模式Focus mode	AFAF
测光模式Metering mode	中央重点测光Center-weighted metering
亮度 brightness	10EV10EV

饱和度saturation	120120
对比度Contrast	100100
锐度Sharpness	MTF50MTF50

According to the shooting parameters of the user shown in Table 4 above, the shooting parameters of the user: the value of white balance is 2400K, the value of sensitivity (international standards / organization, ISO) is 100, and the value of exposure compensation is + 0.5EV, Shutter speed is 1 / 125s, focus mode is auto focus (AF), metering mode is center-weighted metering, brightness value is 10 exposure value (exposure value (EV), saturation value is 120, contrast The value is 100 and the sharpness value is MTF50. The above Table 4 is only used to explain this application and should not constitute a limitation.

2. The shooting mode can include ordinary photography, large aperture, portrait mode, gourmet mode, black and white camera, professional photography, 3D dynamic panorama, high dynamic range imaging (high dynamic range imaging, HDR) photography, etc.

Each shooting mode can correspond to a set of shooting-related parameter sets, where the shooting-related parameter sets can include shooting parameter sets {a1, a2, a3, ...} and picture quality (PQ) effect parameters { b1, b2, b3, ...}. The PQ effect parameter set can be used by the terminal to adjust the PQ effect of the picture, for example, contrast adjustment, brightness adjustment, color saturation adjustment, hue adjustment, and sharpness adjustment (such as digital noise reduction (DNR) adjustment). ), Color edge enhancement (chroma TI, CTI) adjustment and other image quality adjustments.

Exemplarily, the correspondence between the shooting mode and the shooting-related parameter set may be as shown in Table 5 below:

Table 5 Correspondence between shooting modes and shooting related parameter sets

拍摄模式Shooting mode	拍摄相关参数集Shooting related parameter set
普通ordinary	P_1P_1
大光圈Large aperture	P_2P_2
人像Portrait	P_3P_3
美食Food	P_4P_4
黑白相机Black and white camera	P_5P_5
专业profession	P_6P_6
3D动态全景3D dynamic panorama	P_7P_7
HDRHDR	P_8P_8
……...	……...

It can be seen from the correspondence between the shooting modes and shooting related parameter sets shown in Table 5 above, the shooting related parameter set corresponding to the ordinary shooting mode in the shooting mode is P_1, and the shooting related parameter set corresponding to the large aperture shooting mode is P_2, The shooting-related parameter set corresponding to the portrait camera mode is P_3, the shooting-related parameter set corresponding to the gourmet camera mode is P_4, the shooting-related parameter set corresponding to the black-and-white camera camera is P_5, and the shooting-related parameter set corresponding to the professional camera mode is P_6, 3D dynamic The shooting related parameter set corresponding to the panorama is P_7, and the shooting related parameter set corresponding to the HDR camera mode is P_8. The above Table 5 is only for explaining this application and should not be construed as limiting.

The terminal can record the shooting mode used each time the user takes a picture. Therefore, according to the correspondence between the preset shooting mode and the shooting-related parameter set P, the terminal can increase the number of times the user uses more than a preset threshold (for example, the preset number The threshold may be a shooting mode of once, twice, 3 times, 4 times, 5 times, 10 times, etc.), which is determined to be a common shooting mode of the user.

Exemplarily, the user's common shooting modes and the shooting related parameter sets corresponding to the common shooting modes acquired by the terminal may be as shown in Table 6 below:

Table 6 Correspondence between users' common shooting modes and shooting related parameter sets

用户的常用拍摄模式User's common shooting mode	拍摄相关参数集Shooting related parameter set
普通ordinary	P_1P_1
大光圈Large aperture	P_2P_2
人像Portrait	P_3P_3
美食Food	P_4P_4
HDRHDR	P_8P_8

As can be seen from the user's common shooting modes and corresponding shooting related parameter sets shown in Table 6 above, the user's common shooting modes and corresponding shooting related parameter sets acquired by the terminal are: the ordinary camera mode and the ordinary camera mode corresponding to Shooting related parameter set P_1, large aperture mode and shooting related parameter set P_2 corresponding to large aperture mode, shooting mode related parameter setting P_3 corresponding to portrait mode and portrait mode, shooting related parameter set P_4 corresponding to gourmet mode and gourmet mode, HDR camera mode And the relevant shooting parameter set P_8 corresponding to the HDR camera mode. The examples shown in Table 6 above are only used to explain the present application, and should not constitute a limitation.

3. The shooting content can include: portraits, green plants, flowers, food, sunrise, sunset, etc.

Each shooting mode can correspond to a set of shooting-related parameter sets. Among them, the shooting related parameter set includes a shooting parameter set and a PQ effect parameter set. The PQ effect parameter set can be used for the terminal to adjust the PQ effect of the picture, for example, contrast adjustment, brightness adjustment, color saturation adjustment, hue adjustment, sharpness adjustment (such as digital noise reduction (DNR) adjustment), Image quality adjustments such as color edge enhancement (chroma TI, CTI) adjustments.

Exemplarily, the correspondence between the shooting content and the shooting-related parameter set may be as shown in Table 7 below:

Table 7 Correspondence between shooting content and shooting related parameter sets

拍摄内容Shooting content	拍摄相关参数集Shooting related parameter set
人像Portrait	P_9P_9
绿植Greenery	P_10P_10
花朵Flowers	P_11P_11
食物food	P_12P_12
日出sunrise	P_13P_13
夕阳Sunset	P_14P_14
……...	……...

It can be seen from the correspondence between the shooting modes and the shooting related parameter sets shown in Table 7 above, the shooting related parameter set corresponding to the portrait shooting content is P_9, the shooting related parameter set corresponding to the green plant shooting content is P_10, and the flower shooting content corresponds to The shooting related parameter set is P_11, the shooting related parameter set corresponding to food shooting content is P_12, the shooting related parameter set corresponding to sunrise shooting content is P_13, and the shooting related parameter set corresponding to sunset shooting content is P_14. The above Table 7 is only used to explain this application, and should not constitute a limitation.

The terminal can record the shooting content recognized by the terminal through the camera each time the user takes a picture. Therefore, according to the correspondence between the preset shooting content and the shooting-related parameter set P, the terminal can record the number of times the terminal is recognized through the camera when the user takes a photo The shooting content of the second threshold (for example, the second threshold may be 1, 2, 3, 4, 5, 10, etc.) is determined to be the commonly used shooting content of the user.

Exemplarily, the terminal's acquired common shooting content of the user and shooting related parameter sets corresponding to the shooting content may be as shown in Table 8 below:

Table 8 Common shooting content and corresponding shooting related parameter sets of users

用户的常用拍摄内容User's common shooting content	拍摄相关参数集Shooting related parameter set
人像Portrait	P_9P_9
食物food	P_12P_12
日出sunrise	P_13P_13
夕阳Sunset	P_14P_14

It can be seen from the user's common shooting content and corresponding shooting related parameter sets shown in Table 8 above, that the user's common shooting content and corresponding shooting related parameter sets acquired by the terminal are: portrait shooting content and portrait shooting content corresponding to Shooting related parameter set P_9, food shooting content and shooting related parameter set corresponding to food shooting content P_12, sunrise shooting content and sunrise shooting content corresponding shooting related parameter set P_13, sunset shooting content and sunset shooting related shooting related parameters Set P_14. The example shown in Table 8 above is only for explaining the present application, and should not constitute a limitation.

4. Picture browsing habits can include shared pictures, deleted pictures, favorite pictures, edited pictures, etc.

Exemplarily, the terminal may receive a user's input operation to open an album (for example, click an album application icon on the terminal's main interface, and enter "open album" through a voice assistant), and in response to the input operation to open an album, the terminal may open an album application , And the album application interface is displayed on the touch screen, and the album application interface may include one or more photos. The terminal can receive the user's sharing operation, deletion operation, collection operation or editing operation of the photo in the album application. When the terminal receives the user's sharing operation of the user's selected photo, the terminal can obtain the user's shared photo through image analysis Corresponding shooting related parameter set; when the terminal receives the user's deletion of the user-selected photo, the terminal can obtain the shooting related parameter set corresponding to the user's deleted photo through image analysis; when the terminal receives the user's photo selected by the user During the collection operation, the terminal can obtain the shooting-related parameter set corresponding to the photos collected by the user through image analysis; when the terminal receives the user's editing operation on the photo selected by the user, the terminal can obtain the photo edited by the user through image analysis Corresponding shooting related parameter set.

Among them, the user-shared pictures, deleted pictures, favorite pictures, edited pictures obtained by the terminal, and the shooting-related parameter sets corresponding to these pictures can be shown in Table 9 below:

Table 9 User browsing picture habits and shooting related parameter sets corresponding to pictures

According to the user's picture browsing habits and the corresponding PQ effect parameters shown in Table 9 above, the pictures shared by the user include picture_A and picture_B, where picture_A corresponds to the shooting related parameter set P_15 and picture_B corresponds to The shooting related parameter set is P_16. The pictures deleted by the user include picture_C and picture_D, where the shooting-related parameter set corresponding to picture_C is P_17, and the shooting-related parameter set corresponding to picture_D is P_18. The pictures collected by the user include picture_E and picture_F, where the shooting-related parameter set corresponding to picture_E is P_19, and the shooting-related parameter set corresponding to picture_F is P_20. The pictures edited by the user include picture_G and picture_H, where the shooting-related parameter set corresponding to picture_G is P_21, and the shooting-related parameter set corresponding to picture_H is P_22. The above Table 9 is only for explaining the present application, and should not constitute a limitation.

The following specifically describes the process for the terminal to pre-process the collected user data in the embodiment of the present application.

After the terminal collects user data, the terminal can preprocess the collected user data to extract valid source data and store it in the database.

Please refer to FIG. 8, which shows a schematic diagram of user data pre-processing. As shown in FIG. 8, the user data preprocessing process in FIG. 8 includes the following steps:

1. The terminal determines whether the collected user data is general data. If so, the terminal performs redundant data removal and abnormal data filtering on the collected general data, and stores the general data removed redundant data and abnormal data to the user Database. Among them, the user's database may be on a local terminal or a remote server, which is not limited here.

Redundant data removal means that the terminal removes the repetitive data in the collected user data to reduce the size of the effective general data stored in the database. For example, when the terminal collects user data, each data collection of the terminal can collect multiple copies of data through different paths. In a certain data collected by the terminal, the terminal obtains from the life service application that the user sets the default courier receiving address to "Shenzhen", then the terminal can determine the default courier receiving address as the normal in the user's general data Station, that is, the terminal obtains a piece of data of the user's permanent residence as "the permanent residence is 'Shenzhen'". In addition, if the terminal obtains the location address of the terminal in the evening of more than 20 days (such as 23:00 to 6:00 the next day) in a month through the location service function of the mobile phone, the terminal can change the location address " "Shenzhen" is determined as the resident location of the user, that is, the terminal obtains a piece of data of the resident location of the user as "the resident location is 'Shenzhen'". After the terminal obtains the two pieces of data of the same permanent residence, it can retain a copy of the data of the permanent residence and determine that the data of the permanent residence is valid and stored in the user's database. For another example, when the terminal has stored the user's resident location as "Shenzhen" in the user's database, the terminal has a piece of data in the collected user data that the user's resident location is "Shenzhen", and the user's database If the stored resident location of the user is the same, the terminal can remove the collected piece of data that “the resident location of the user is 'Shenzhen'”. The above examples are only used to explain this application and should not be construed as limitations.

Abnormal data filtering refers to the terminal filtering and removing unreasonable data in the collected user data. The reason for the abnormal data may be that the unreasonable data may mean that the data exceeds the value range of the data attribute. For example, the terminal can regulate the value of the user's age data attribute: the value range of age is 0 to 150 years old, if the user's age is 200 years old in a piece of data collected by the terminal, it is not within the value range of age, Then, the terminal determines that this piece of data is abnormal data, and filters to remove the abnormal data. The examples are only for explaining this application and should not be construed as limitations.

2. In the case that the user data collected by the terminal is not general data, the terminal determines whether the collected user data is a shooting parameter, and if so, the terminal stores the shooting parameters in the user's database.

Among them, the shooting parameters may include white balance, ISO, exposure compensation, shutter speed, focus mode, metering mode, brightness, saturation, contrast, sharpness, etc.

3. In the case that the user data collected by the terminal is neither general data nor shooting parameters, if the terminal determines that the collected user data is a shooting mode or shooting content, the terminal extracts the shooting mode or shooting content. A preset shooting related parameter set, and the shooting related parameter set extracted from the shooting mode or shooting content is stored in the user's database. If not, the terminal judges whether the picture is clear. If the picture is clear, the terminal analyzes the picture to extract the shooting-related parameter set corresponding to the picture and stores it in the user's database. Exemplarily, the terminal may determine whether the sharpness value of the picture is greater than a preset sharpness threshold, and if so, the picture is clear.

In the following, the process of feature extraction of end user data in the embodiment of the present application is specifically described.

1. The terminal performs feature extraction on the user's general data

Please refer to FIG. 9, which is a flowchart of terminal extraction of a user ’s general data features in an embodiment of the present application. As shown in FIG. 5, after the smart camera function provided by the present application is turned on, the terminal can collect user data information, where the user data information collected by the terminal includes the user's personal basic information, behavior habits, hobbies, and shooting hobbies , Picture browsing habits, etc. Next, the terminal performs the preprocessing process shown in FIG. 8 on the collected user data to extract valid source data. Finally, the terminal matches the pre-processed general data (including personal basic information, behavior habits, and hobbies) with the data in the pre-trained general feature tag library, and stores the feature values corresponding to the matched feature tags to the user In the database.

Exemplarily, the general data preprocessed by the terminal may be as shown in Table 10 below:

Table 10 General valid data

According to the general valid data shown in Table 10 above, the user's personal basic information: gender is female, year of birth is 1994, and habitual residence is Shenzhen; behavioral habits of the user: the most commonly used application is the daily P chart, The most used APP is WeChat, the APP used after plugging in the headset is NetEase Cloud Music, the frequent place is the beach, the sleeping time is 23:00, and the wake-up time is 8:00; the user ’s hobbies: reading preferences are romantic novels , Fashion magazines, Internet browsing habits are often searched for the keyword "attractions". The above Table 10 is only used to explain the present application, and should not constitute a limitation. In a specific implementation, the general valid data of the user may also include more information.

Exemplarily, the pre-trained general feature label library of the terminal may be as shown in Table 11 below:

Table 11 Pre-trained general feature label library

From the pre-trained feature tag library shown in Table 11 above, it can be known that the feature tag for males has a feature ID value of 0000, and the feature tag for women is a feature ID value of 0001. The year of birth is from 1978 to 2018. The corresponding feature label is "Young People", and the feature ID value is 0002. The year of birth is from 1959 to 1977. The corresponding feature label is "middle-aged person", and the feature ID value is 0003. The feature label corresponding to the year of birth before 1958 is "elderly", and the feature ID value is 0004. The feature tag corresponding to the most commonly used or most used time or the type of APP used after plugging in the headset is "Music" is "Music" (feature ID value is 0005). The feature tag corresponding to the most commonly used or most used time or the type of APP used after plugging in the headset is "shopping" (the feature ID value is 0006). The feature label corresponding to the most commonly used or most used time or the type of APP used after plugging in the headset is "tourism" is "tourism" (feature ID value is 0007). The feature label corresponding to the most commonly used or actually used time or the type of APP used after plugging in the headset is "Game" is "Game" (feature ID value is 0008). The feature label corresponding to the most commonly used or actually used time or the type of APP used after plugging in the headset is "social" is "social" (feature ID value is 0009). The feature label corresponding to the most commonly used or actually used time or the type of APP used after plugging in the headset is "entertainment" is "entertainment" (feature ID value is 0010). The feature tag corresponding to the most commonly used or actually used time or the type of APP used after plugging in a headset is "movie" is a movie (feature ID value is 0011). The characteristic label corresponding to the reading preference type "sports information" is "sports" (the characteristic ID value is 0012). The feature tag corresponding to the average e-book reading time longer than 30 minutes is "reading" (feature ID value is 0013). The feature tag corresponding to the average daily Internet browsing time of more than 30 minutes is "news information" (feature ID value is 0014). On the Internet, it is often searched for keywords such as "finance" or "investment" and the corresponding feature label is "finance" (feature ID value is 0015). On the Internet, it is often searched for keywords such as "workplace" or "work" and the corresponding feature tag is "business" (feature ID value is 0016). The above Table 11 is only for explaining the present application, and should not constitute a limitation.

Combining the above Table 10 and the above Table 11, the user's general data feature label may be as shown in Table 12 below:

Table 12 The user's universal feature tag, that is, the feature ID value corresponding to the universal feature tag

用户的通用特征标签User's common feature label	通用特征标签对应的特征ID值Feature ID value corresponding to the universal feature tag
女性female	00010001
年轻人young people	00020002
音乐music	00050005
购物shopping	00060006
旅游tourism	00070007
社交Socializing	00090009

From the user's general feature tags shown in Table 12 above, the user's general feature tags are female (feature ID value 0001), young people (feature ID value 0002), music (general feature ID value 0004), Shopping (feature ID value 0006), travel (feature ID value 0007), and social networking (feature ID value 0009). The terminal may store the above-mentioned universal feature tags in the form of feature ID values in the user's database. The above Table 12 is only for explaining this application, and should not constitute a limitation.

2. The terminal performs feature extraction on the user's shooting related data.

The process of training the mapping function f (x) of the shooting related parameter set P and the shooting feature label vector set S through the neural network (such as convolutional neural network (CNN)) is first introduced by the terminal.

Please refer to FIG. 10, 10a in FIG. 10 is a flowchart of a terminal training a neural network in an embodiment of the present application. As shown in 10a in FIG. 10, first, the terminal may pre-store several identical pictures with different tag information in different categories, where the tag information may refer to the shooting feature tags corresponding to the picture (eg, strong beauty, weak beauty, small freshness) , Japanese, etc.) score vector set (such as {strong beauty score, weak beauty score, small fresh score, Japanese score}), and the shooting related parameter set corresponding to the picture. Among them, in a group of pictures, different pictures correspond to different shooting related parameter sets (including shooting parameter sets {a1, a2, a3, ...} and PQ effect parameters can be {b1, b2, b3, ...} ), And corresponding to different shooting feature label score vector set S. Then, the terminal may select several groups (for example, 10 groups) for the user to select. Then, the terminal can use the label information corresponding to the picture selected by the user as the cold start training parameter set Q {P → S} of the neural network, where P includes the shooting parameter set and the PQ effect parameter set, and S is the shooting feature label (such as strong Beauty, weak beauty, small freshness, Japanese, etc.). Finally, the terminal can input the training parameter set Q to a neural network (such as a convolutional neural network), and use a deep learning algorithm to obtain a mapping function f (x).

Exemplarily, the 10 sets of effect pictures shown in 4d of FIG. 4 above. The first group of effect pictures may include picture a, picture b, picture c, and picture d. After the terminal receives the input operation of selecting the picture b by the user (for example, clicking on the picture b), the terminal may use the shooting related parameter set P_b and the shooting feature label score vector set S_b corresponding to the picture b as a set of cold start training parameter sets Q_b { P_b → S_b} is input into the neural network, and a deep learning algorithm is used to obtain the mapping function f (x). Among them, the terminal can receive multiple effect pictures selected by the user. The cold start training parameter set Q includes not only the shooting related parameter set P_b and the shooting feature label score vector set S_b corresponding to the picture b, but also other pictures selected by the user ( For example, picture e) corresponding shooting related parameter set P_e and shooting feature label score vector set S_e. For the part not detailed in the specific content, reference may be made to the embodiment shown in FIG. 4 above. In this way, the terminal trains the neural network by acquiring the training set corresponding to the effect picture selected by the user, so that the shooting feature label score vector set output by the mapping function f (x) obtained by the training meets the user's preferences and improves the user's experience .

In a possible situation, the terminal may periodically train the neural network, for example, the training period T may be 10 days, 15 days, 1 month or longer.

As shown in 10b of FIG. 10, before training the neural network model, the terminal can determine whether the number N of training sets required for the training is less than a threshold M (for example, 11), and if so, the terminal can check the picture with the user The training set (such as Q_b {P_b → S_b}) composed of the corresponding shooting related parameter set (such as P_b corresponding to picture b) and the shooting feature label score vector set (such as S_b corresponding to picture b) trains the neural network model, If not, the terminal can use the pre-stored training set Q_n {P_n → S_n} to input to the neural network model, where P_n is a pre-stored shooting related parameter set for training the neural network model, and S_n is pre-stored for training Neural network model shooting feature label score vector set to get the mapping function f (x). Wherein, the number of pre-stored training sets is greater than the threshold M (for example, 11). In this way, when the mapping function f (x) is trained using the neural network model, the more training sets, the terminal will use the training mapping function f (x) to output the shooting feature label score vector set will be more and more in line with user preferences Therefore, when there are fewer training sets corresponding to the effect pictures manually selected by the user, the terminal can train the neural network model through the pre-stored sample training set, so that the shooting feature label score vector set output by the mapping function f (x) , More in line with user preferences and improve the user experience.

The following describes the process of extracting the shooting feature labels in the shooting related data of the user after the terminal has trained the neural network model.

As shown in 10c in FIG. 10, after the terminal trains the neural network model to obtain the mapping function f (x), the terminal can use the shooting-related parameter set P in the user's shooting-related data collected by the terminal as the mapping function f ( The input vector of x) is input into the mapping function f (x) of the neural network model to output the shooting feature label score vector set S corresponding to the user's shooting related data. The terminal may extract the shooting feature tag with the highest score in the output shooting feature tag score vector set S, and store it as a shooting feature tag for the user in the user's database. Among them, since there are multiple sets of shooting-related parameter sets P in the shooting-related data collected by the terminal (for example, P_1, P_2, P_3, P_4, P_8, etc.), the terminal sequentially uses multiple sets of shooting-related parameter sets P as input vector In the mapping function f (x), multiple sets of score vectors S of shooting feature labels can be obtained. In a possible case, the terminal may extract multiple shooting feature tags from the multiple sets of shooting feature tag score vector sets S.

Exemplarily, with reference to the above Table 6, Table 8, and Table 9, the shooting related parameter set P in the shooting related data of the user collected by the terminal may be as shown in Table 13 below:

Table 13 The shooting related parameter set in the shooting related data of the user

As can be seen from Table 13 above, the shooting related parameter sets P in the shooting related data collected by the terminal are: P_1, P_2, P_3, P_4, P_8, P_9, P_12, P_13, P_14, P_15, P_16, P_17, P_18, P_19 , P_20, P_21, P_22. Table 13 is only for explaining the application and should not constitute a limitation.

The terminal may input multiple sets of shooting-related parameter sets P in the above Table 13 into the mapping function f (x) in turn to obtain the shooting feature label score vector set S corresponding to each shooting-related parameter set P, and from each shooting The feature tag score vector set S extracts the shooting feature tag with the highest score.

Exemplarily, the shooting feature label score vector set S can be expressed as {c1, c2, c3, c4, c5, c6}, where c1 is the shooting feature label "strong beauty" score, and c2 is the shooting feature label The score of "weak beauty", c3 is the score of the shooting feature label "Small Fresh", c4 is the score of the shooting feature label "Japanese", c5 is the score of the shooting feature label "European style", and c6 is the shooting The score of the feature label "Small Fresh + Weak Beauty".

Among them, the shooting feature label score vector set S corresponding to each shooting related parameter set P in Table 13 above, and the shooting feature label with the highest score in each shooting feature label score vector set S may be as shown in Table 14 below:

Table 14 The shooting feature label score vector set corresponding to each shooting related parameter set, and the shooting feature label with the highest score

As can be seen from Table 14 above, the terminal performs feature extraction on the user's shooting related data, and the extracted shooting feature tags of the user include weak beauty, small freshness, and Japanese. And the feature ID value corresponding to the shooting feature tag can be shown in Table 15 below:

Table 15 User's shooting feature tags and their corresponding feature ID values

用户的拍摄特征标签User's shooting feature tag	特征ID值Feature ID value
弱美颜Weak beauty	002002
小清新Small fresh	003003
日系Japanese	004004

It can be seen from the user's shooting feature tags and their corresponding feature ID values shown in Table 15 above, that the user's shooting feature tag "weak beauty" corresponds to a feature ID value of 002, and the user's shooting feature tag "small The feature ID value corresponding to "Fresh" is 003, and the feature ID value corresponding to the user's shooting feature label "Japanese" is 004. The terminal may store the above-mentioned universal feature tags in the form of feature ID values in the user's database. The above Table 15 is only for explaining the present application, and should not constitute a limitation.

In a possible case, the shooting feature tag of the user may be a shooting feature tag whose score value in each shooting feature tag score vector set S is greater than the first threshold. For example, the first threshold may be 0.7, and in combination with Table 14 above, it can be seen that the user's shooting feature tag includes "small freshness". The examples are only for explaining this application and should not be construed as limitations.

The following specifically describes the process in which the terminal performs feature tag fusion on the user's general feature tag and the user's shooting feature tag in the embodiment of the present application.

Please refer to FIG. 11, which is a feature label fusion process provided by an embodiment of the present application. As shown in FIG. 11, the user's general feature tags may include: women, young people, music, shopping, travel, and socializing. The shooting feature tags of the user may include: weak beauty, small freshness, and Japanese. Among them, each universal feature tag corresponds to each shooting feature tag with a score. For example, the score corresponding to women with weak beauty is x1, the score corresponding to women with Xiaoqing is y1, and the score corresponding to women with Japanese is z1. The score for young people and weak beauty is x2, the score for young people and Xiaoqing is y2, and the score for young people and Japanese is z2. The score corresponding to music and weak beauty is x3, the score corresponding to music and Xiaoqing is y3, and the score corresponding to music and Japanese is z3. The score for shopping and weak beauty is x4, the score for shopping and Xiaoqing is y4, and the score for shopping and Japanese is z4. The score corresponding to tourism and weak beauty is x5, the score corresponding to tourism and small freshness is y5, and the score corresponding to tourism and Japanese is z5. The score corresponding to social and weak beauty is x6, the score corresponding to social and small fresh is y6, and the score corresponding to social and Japanese is z6.

Among them, the fusion weight value of the user's general feature label is L1, and the fusion weight value of the user's shooting feature label is L2. The fusion feature tag obtained after the terminal performs feature tag fusion is the same as the user's shooting feature tag, that is, the fusion feature tag may include weak beauty, small freshness, and Japanese. The fusion feature label score T1 corresponding to the fusion feature label "weak beauty" can be calculated by the following formula (1):

T1 = L1 * (x1 + x2 + x3 + x4 + x5 + x6) + L2 * 1 formula (1)

In the above formula (1), L1 is the fusion weight of the user's general feature label, L2 is the user's shooting feature label fusion weight, and x1 is the general feature label "female" and the shooting feature label "weak beauty" Corresponding score, x2 is the score corresponding to the general feature label "young man" and the shooting feature label "weak beauty", x3 is the score corresponding to the general feature label "music" and the shooting feature label "weak beauty", x4 is the score corresponding to the general feature label "shopping" and the shooting feature label "weak beauty", x5 is the score corresponding to the general feature label "tourism" and the shooting feature label "weak beauty", and x6 is the general feature label " The score of "Social" and the shooting feature label "weak beauty".

The fusion feature label score T2 corresponding to the fusion feature label "weak beauty" can be calculated by the following formula (2):

T2 = L1 * (y1 + y2 + y3 + y4 + y5 + y6) + L2 * 1 formula (2)

Among them, in the above formula (2), L1 is the fusion weight of the user's general feature label, L2 is the user's shooting feature label fusion weight, and y1 is the general feature label "female" corresponds to the shooting feature label "small fresh" , Y2 is the score corresponding to the general feature label "Young People" and the shooting feature label "Small Fresh", y3 is the score corresponding to the general feature Label "Music" and the shooting feature label "Small Fresh", y4 is the general purpose The score corresponding to the feature tag "Shopping" and the shooting feature tag "Small Fresh", y5 is the score corresponding to the general feature tag "Tourism" and the shooting feature tag "Small Fresh", and y6 is the generic feature tag "Social" and shooting features The score corresponding to the label "Small Fresh".

The fusion feature label score T3 corresponding to the fusion feature label "weak beauty" can be calculated by the following formula (3):

T3 = L1 * (z1 + z2 + z3 + z4 + z5 + z6) + L2 * 1 formula (3)

Among them, in the above formula (3), L1 is the fusion weight of the user's general feature label, L2 is the user's shooting feature label fusion weight, and z1 is the corresponding of the general feature label "female" and the shooting feature label "Japanese" Score, z2 is the score corresponding to the general feature label "Young People" and the shooting feature label "Japanese", z3 is the score corresponding to the general feature label "Music" and the shooting feature label "Japanese", z4 is the generic feature label " "Shopping" is the score corresponding to the shooting feature tag "Japanese", z5 is the score corresponding to the general feature tag "Travel" and the shooting feature tag "Small Fresh", and z6 is the general feature tag "Social" and the shooting feature tag "Japanese" The corresponding score.

Exemplarily, the fusion weight L1 of the user's general feature label may be 0.6, and the fusion weight L2 of the user's general feature label may be 0.4. The scores corresponding to the general feature tag and the shooting feature tag can be shown in Table 16 below:

Table 16 Scores corresponding to general feature tags and shooting feature tags

From the above Table 16, and the above formula (1), formula (2), and formula (3), the terminal can calculate that the fusion tag score T1 corresponding to the fusion feature tag “weak beauty” is 1.54, and the fusion feature tag “small fresh "The corresponding fusion label score T2 is 1.96, and the fusion feature label" Japanese "corresponds to the fusion label score T3 of 1.3. Among them, the fusion feature label with the highest fusion feature label score is Xiaoxin.

The terminal may determine the fusion feature tag with the highest fusion feature tag score as the user's smart camera tag and store the smart camera tag in the user's database, where the user's smart camera tag may be used by the terminal to take photos of the user To set the PQ effect parameters of the shooting screen.

The following describes the PQ effect parameters set when the user uses the smart camera function to take pictures in the embodiment of the present application.

As shown in FIG. 12, the terminal may pre-store the PQ effect parameter set corresponding to each fusion feature label. For example, the PQ effect parameter set corresponding to the fusion feature label "weak beauty" is parameter set 1, the PQ effect parameter set corresponding to the fusion feature label "Xiao Qingxin" is parameter set 2, and the PQ effect parameter corresponding to the fusion feature label "Japanese" Set is parameter set 3.

After calculating the fusion feature tag score corresponding to each fusion feature tag, the terminal may use the PQ effect parameter set corresponding to the fusion feature tag with the highest fusion feature tag score (that is, the smart camera tag) to capture the Image processing, to get smart photos. Exemplarily, the fusion feature tag with the highest fusion feature tag score (that is, the smart camera tag) can be "small fresh", and the PQ effect parameter set corresponding to the smart camera tag "small fresh" is parameter set 3, that is, the terminal can use This parameter set 3 performs image processing on the screen captured when the terminal takes a picture, so as to obtain an intelligently taken picture.

Through the intelligent photographing method provided by the embodiment of the present application, the terminal can collect user data, extract the user's characteristic tags, and assist the user to take a photographing effect that matches the user's characteristics according to the feature tag, thereby realizing a photo effect that matches the user's personality To improve the user experience.

Please refer to FIG. 13, which is a user database 1300 constructed by the data storage module in the smart camera system in the embodiment of the present application. As shown in FIG. 13, the user database 1300 may include user data collected by the terminal, valid data after preprocessing the collected user data, and feature value data corresponding to the user's feature tags.

The user data may include general data and photograph-related data.

The valid data may include: shooting parameters (such as shooting parameter set 1, shooting parameter set 2, shooting parameter set 3, etc.), PQ effect parameters (such as PQ effect parameter set 1, PQ effect parameter set 2, PQ effect parameter set 3, etc. ), General valid data (such as data 1, data 2, data 3, etc.).

The feature value data may include: general feature values (such as general feature ID1, general feature ID2, general feature ID3, etc.), shooting related feature values (such as shooting related feature ID1, shooting related feature ID2, shooting related feature ID3, etc.), fusion Feature values (for example, fusion feature ID1, fusion feature ID2, fusion feature ID3, etc.). Among them, the common feature value is used to indicate the common feature label of the user, and each common feature value corresponds to a common feature label. The shooting related feature value is used to indicate the shooting feature label of the user, and each common feature value corresponds to one common feature label. The fusion feature value is used to indicate the user's fusion feature label, where each fusion feature value corresponds to a fusion feature label.

In the embodiment of the present application, the user database 1300 is only used to explain the present application, and should not constitute a limitation. In a specific implementation, the user database 1300 may include more information, for example, a PQ effect parameter set corresponding to the fusion feature label.

Please refer to FIG. 14, which is a schematic flowchart of an intelligent photographing method provided by the present application. Among them, as shown in FIG. 14, the intelligent photographing method includes:

S1401: The terminal extracts one or more first tags in the user's general data; the general data is used to characterize the identity characteristics of the user.

Among them, the user's general data may include the user's personal basic information, behavior habits, hobbies, etc. Among them, personal basic information may include gender, year of birth, place of usual residence, etc. Behavioral habits can include the user's most commonly used APP, the most used APP, the APP used after plugging in headphones, frequent places, bedtime, wake-up time, etc. Hobbies can include reading preferences, Internet browsing habits, etc. For the process of collecting user data by the terminal, reference may be made to the foregoing embodiment, and details are not described herein again.

Exemplarily, the one or more first tags extracted by the terminal may be the general feature tag of the user in the foregoing embodiment, for example, the general feature tag of the user in Table 12 above, the general feature tag of the user is female , Young people, music, shopping, travel, socializing. For the process of extracting one or more first tags in the user's general data by the terminal, reference may be made to the general data feature extraction process in the embodiment shown in FIG. 9 above, and details are not described herein again.

S1402. The terminal extracts one or more second tags in the user's shooting related data; the shooting related data is used to characterize the user's shooting preferences.

Wherein, the user's shooting related data may include the user's shooting preferences, browsing picture habits, and so on. Among them, the user's shooting preferences include shooting parameters, shooting modes, shooting content and so on. Picture browsing habits include shared pictures, deleted pictures, favorite pictures, edited pictures, etc.

Exemplarily, the shooting parameters may include white balance, ISO, exposure compensation, shutter speed, focus mode, metering mode, brightness, saturation, contrast, sharpness, and the like. Shooting modes can include ordinary photography, large aperture, portrait mode, gourmet mode, black and white camera, professional photography, 3D dynamic panorama, HDR photography, etc. The examples are only for explaining this application and should not be construed as limitations.

Exemplarily, the one or more second tags extracted by the terminal may be shooting feature tags of the user in the foregoing embodiment. For example, the one or more second tags may be the shooting feature tags of the user shown in Table 15 above, and the shooting feature tags of the user are weak beauty, small freshness, and Japanese. The examples are only for explaining this application and should not be construed as limitations.

Wherein the terminal extracts one or more second tags in the shooting related data of the user, reference may be made to the shooting feature extraction process of the shooting related data in the embodiment shown in FIG. 10, which will not be repeated here.

S1403. The terminal determines a third label according to the one or more first labels and the one or more second labels.

The third tag may be the smart camera tag in the foregoing embodiment, for example, the smart camera tag in FIG. 12: small and fresh. The process for the terminal to determine the third label according to the one or more first labels and the one or more second labels may refer to the aforementioned feature label fusion process shown in FIG. 11 and FIG. 12, which will not be repeated here.

S1404. The terminal adjusts the image quality of the image captured by the terminal according to the image quality effect parameter set corresponding to the third label.

The picture quality (PQ) effect parameter set can be used by the terminal to adjust the image quality effect of the captured image, for example, contrast adjustment, brightness adjustment, color saturation adjustment, hue adjustment, sharpness adjustment (such as digital Image quality adjustments such as digital noise reduction (DNR adjustment), color edge enhancement (chroma TI, CTI) adjustment, etc.

Exemplarily, the third label may be the smart camera label in the embodiment shown in FIG. 12: Small and fresh, and the image quality effect parameter set corresponding to the third label may be parameter set 3 in the embodiment described in FIG. 12 . For specific content, reference may be made to the content in the foregoing embodiment of FIG. 12, and details are not described herein again.

In a possible situation, the terminal extracting one or more first tags in the general data may specifically include: the terminal extracts one or more first tags corresponding to the general data according to the first mapping relationship Label; wherein, the first mapping relationship includes mapping of multiple sets of common data with multiple first labels.

Exemplarily, the first mapping relationship may be the pre-trained general feature label library shown in Table 11 in the foregoing embodiment, and one or more of the general data can be extracted through the pre-trained general feature label library First label (eg, female, young, music, shopping, travel, social). The above examples are only used to explain this application and should not be construed as limitations. In this way, the terminal can match the general data with the first mapping relationship to obtain the user's general feature label, that is, the first label, so that the terminal can quickly extract the user's general feature label.

In a possible case, the terminal extracts one or more first tags in the shooting related data, which may specifically include: First, the terminal extracts one or more first shooting related parameters from the shooting related data set. Then, the terminal inputs the one or more first shooting-related parameter sets into the first neural network model to obtain the one or more first score vector sets; wherein the first score vector set includes multiple fourth The first score of each tag. The first score is used to characterize the degree of matching between the first shooting-related parameter set and the fourth tag. Next, the terminal determines the one or more second tags from the plurality of fourth tags according to the first score vector set corresponding to each of the one or more first shooting related parameter sets.

Exemplarily, the first neural network model may be a convolutional neural network (CNN). The first shooting related parameter set includes a shooting parameter set {a1, a2, a3, ...} of the shooting related data and a PQ effect parameter set {b1, b2, b3, ...}. Wherein, the plurality of fourth tags may include shooting feature tags in the example described in Table 14 (for example, strong beauty, weak beauty, small freshness, Japanese, European and American style, small freshness + weak beauty), the For the first score vector set, reference may be made to the shooting feature tag score vector set S in the example described in Table 14 above. For specific content, reference may be made to the example described in FIG. 10, and details are not described herein again. That is to say, the terminal can use the neural network model to extract the feature labels in the shooting related data, so that the terminal can use the self-learning ability of the neural network model to improve the accuracy of the terminal to extract the feature labels in the shooting related data .

In a possible case, the one or more second tags include a first set of score vectors corresponding to each of the one or more sets of first shooting-related parameters, and the first score is greater than a first threshold One or more fourth labels.

Exemplarily, the first threshold may be 0.7. With reference to the above Table 14, it can be concluded that the user's shooting feature label may include "small freshness". For specific contents, reference may be made to the foregoing embodiments shown in Table 14 and Table 15, and details are not described herein again.

In a possible case, the one or more second tags include a first set of score vectors corresponding to each of the first shooting-related parameter sets, and one or more fourth tags with the highest first score.

Exemplarily, for the one or more second tags, reference may be made to the user's shooting feature tags shown in Table 15 (for example, weak beauty, small freshness, Japanese). For specific contents, reference may be made to the foregoing embodiments shown in Table 14 and Table 15, and details are not described herein again. That is to say, the terminal may determine one or more fourth tags with a fourth score greater than the first threshold as one or more second tags, because the size of the first score is used to indicate that the user and the second tag The matching degree of the four tags, the larger the first score, the higher the matching degree of the user's shooting related data and the fourth tag, so that the terminal can extract one or more second tags that match the characteristics of the user's shooting related data.

In a possible situation, before the terminal inputs the one or more first shooting-related parameter sets into the first neural network model, the terminal may obtain sample data; the sample data includes multiple sets of first training sets, Wherein, each set of first training set includes a set of second shooting related parameter set and a set of second score vector set. The terminal trains the first neural network model through a deep learning algorithm based on the sample data.

Exemplarily, the first training set may be the training parameter set Q {P → S} in the foregoing embodiment shown in FIG. 10, where P includes a shooting parameter set and a PQ effect parameter set, and S is a shooting feature label (such as strong Beauty, weak beauty, small freshness, Japanese, etc.). The second shooting related parameter set may include the shooting parameters and the PQ effect parameters. The second set of score vectors may include scores of shooting feature labels (such as strong beauty, weak beauty, small freshness, Japanese, etc.). For specific content, reference may be made to the foregoing embodiment shown in FIG. 10, and details are not described herein again. That is to say, the terminal extracts one or more fourth labels with the highest first score in each first score vector set of the user, and determines one or the second label above, so that the terminal can improve the extraction of a user Or the accuracy of multiple second labels.

In a possible case, the terminal displays a first interface, and the first interface includes multiple sample pictures. Each sample picture corresponds to a set of the second shooting related parameter set and a set of the second score vector set; the second shooting related parameter set is used to characterize the image quality of the sample picture, the second grouping The vector set includes the first score of each of the plurality of fourth tags corresponding to the sample picture. The terminal receives the user's first input operation of selecting one or more training pictures from the plurality of sample pictures. In response to the first input operation, the terminal may determine the second shooting-related parameter set and the second score vector set corresponding to the one or more training pictures as the sample data.

Exemplarily, the first interface may be the user preference survey interface 440 shown in 4d in FIG. 4 or the user preference survey interface 530 shown in 5b in FIG. 5. The sample picture may be picture a, picture b, picture c, picture d, etc. in the user preference survey interface 440 or the user preference survey interface 530. The first input operation may be the input operation 442 shown in 4d in FIG. 4 or the input operation 532 shown in 5b in FIG. For specific content, reference may be made to the foregoing embodiments of FIG. 4 or FIG. 5, and details are not described herein again. That is to say, the terminal can train the first neural network model through the sample data corresponding to the sample image preselected by the user, so that the terminal can extract one or more second feature tags that meet the user's personalized shooting preferences.

In a possible case, the terminal can determine whether the number of sample pictures is less than the number of trainings. If so, the terminal takes one or more sets of the second shooting-related parameter set from the pre-stored training set database and The second set of score vectors is used as the sample data. For specific content, reference may be made to the foregoing embodiment shown in 10b in FIG. 10, and details are not described herein again. That is to say, the terminal can use the pre-stored training set to train the first neural network when the number of sample images selected by the user is insufficient, which reduces the user's input operation and improves the user experience.

In a possible situation, each first label and each second label jointly have an associated score; the size of the associated score is used to characterize the degree of association between the first label and the second label . The terminal may determine the total association score of each second label according to the one or more first labels and the one or more second labels

Where T _i is the total relevance score of the i-th second label among the one or more second labels, L _{1 is} the weight of the one or more first labels, and L _{2 is} the one The weight of the second label or multiple labels, where W _k is the association score corresponding to the k-th first label and the i-th second label of the one or more first labels, and R is the one Or the number of multiple first labels. The terminal determines the third label according to the total association score of each second label, where the third label is the one with the highest total association score among the one or more second labels.

Exemplarily, the association score may be the score corresponding to the general feature tag and the shooting feature tag in the embodiment shown in FIG. 11 (for example, x1, x2, x3, x4, x5, x6; y1, y2, y3, y4, y5, y6; z1, z2, z3, z4, z5, z6), refer to the embodiment shown in Table 16 above. The third tag may be the smart camera tag in the foregoing embodiment shown in FIG. 11. For the total association score of each second label, reference may be made to the fusion feature label scores T1, T2, and T3 in the foregoing embodiment shown in FIG. 11. For specific content, reference may be made to the foregoing embodiment shown in FIG. 11, and details are not described herein again. That is to say, the terminal may set the weight value for the first label and the second label of the user, and set the corresponding degree of association value for each first label and each second label. In this way, the terminal can make the image quality adjustment parameters recommended by the terminal to the user more in line with the user's personalized preferences and improve the user experience.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they can still The technical solutions described in the embodiments are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

An intelligent photographing method, characterized in that it includes:

The terminal extracts one or more first tags in the user's general data; the general data is used to characterize the identity of the user;

The terminal extracts one or more second tags in the user's shooting related data; the shooting related data is used to characterize the user's shooting preferences;

The terminal determines a third label according to the one or more first labels and the one or more second labels;

The terminal adjusts the image quality of the image captured by the terminal according to the image quality effect parameter set corresponding to the third label.
The method according to claim 1, wherein the terminal extracts one or more first tags in the general data, specifically including:

The terminal extracts one or more first tags corresponding to the general data according to the first mapping relationship; wherein the first mapping relationship includes mappings between multiple sets of general data and multiple first tags.
The method according to claim 1, wherein the terminal extracts one or more second tags in the user's shooting related data, specifically including:

The terminal extracts one or more first shooting related parameter sets from the shooting related data;

The terminal inputs the one or more first shooting-related parameter sets into the first neural network model to obtain the one or more first score vector sets; wherein, the first score vector set includes multiple A first score for each of the fourth tags, and the first score is used to characterize the degree of matching between the first shooting-related parameter set and the fourth tag;

The terminal determines the one or more second tags from the plurality of fourth tags according to the first score vector set corresponding to each of the one or more first shooting related parameter sets.
The method according to claim 3, wherein the one or more second tags include a first set of score vectors corresponding to each of the one or more sets of first shooting related parameters, the first score One or more fourth tags that are greater than the first threshold.
The method according to claim 3, wherein the one or more second tags include a first set of score vectors corresponding to each of the first shooting-related parameter sets, the one with the highest first score Or multiple fourth labels.
The method according to claim 3, wherein before the terminal inputs the one or more first shooting related parameter sets into the first neural network model, the method further comprises:

The terminal acquires sample data; the sample data includes multiple sets of first training sets, where each set of first training sets includes a set of second shooting-related parameter sets and a set of second score vector sets;

Based on the sample data, the terminal trains the first neural network model through a deep learning algorithm.
The method according to claim 6, wherein the terminal acquiring the sample data specifically includes:

The terminal displays a first interface, and the first interface includes multiple sample pictures;

Each sample picture corresponds to a set of the second shooting related parameter set and a set of second score vector set; the second shooting related parameter set is used to characterize the image quality of the sample picture, so The second grouping vector set includes the first score of each of the plurality of fourth tags corresponding to the sample picture;

The terminal receives the first input operation of the user selecting one or more training pictures from the plurality of sample pictures;

In response to the first input operation, the terminal determines the first shooting-related parameter set and the second score vector set corresponding to the one or more training pictures as the sample data.
The method according to claim 7, wherein the method further comprises:

The terminal determines whether the number of the sample pictures is less than the number of trainings. If yes, the terminal takes one or more sets of the first training set from the pre-stored training set database as the sample data.
The method according to claim 1, wherein each of the first tags and each of the second tags corresponds to an associated score; the size of the associated score is used to characterize the first tag and The degree of association between the second tags;

The terminal determines a third label according to the one or more first labels and the one or more second labels, which specifically includes:

The terminal determines the total association score of each second label according to the one or more first labels and the one or more second labels
Where T i is the total relevance score of the i-th second label among the one or more second labels, and L 1 is the weight of the one or more first labels, so The L 2 is the weight of the one or more second labels, and the W k is the common correspondence between the k-th first label and the i-th second label of the one or more first labels The correlation score, the R is the number of the one or more first tags;

The terminal determines the third label according to the total association score of each second label, wherein the third label is the total association score of the one or more second labels The highest one.
A terminal is characterized by comprising: one or more processors and one or more memories; the one or more memories are coupled with the one or more processors, and the one or more memories are used for Store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the terminal executes the smart photographing method according to any one of claims 1-9 .
A computer storage medium, characterized by including computer instructions, when the computer instructions run on a terminal, causing the terminal to execute the intelligent photographing method according to any one of claims 1-9.
A computer program product, characterized in that, when the computer program product runs on a computer, the computer is caused to execute the intelligent photographing method according to any one of claims 1-9.