CN110059211A

CN110059211A - Record the method and relevant apparatus of user feeling

Info

Publication number: CN110059211A
Application number: CN201910245272.0A
Authority: CN
Inventors: 相超; 龚阿世; 张宇涵
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-03-28
Filing date: 2019-03-28
Publication date: 2019-07-26
Anticipated expiration: 2039-03-28
Also published as: WO2020192761A1; CN110059211B

Abstract

This application discloses the methods and relevant apparatus of record user feeling.In the method for record user feeling, electronic equipment can be in preview image under photographed scene, identify the affective state of user, and when shooting image the associated storage shooting image and the user recognized affective state, alternatively, the associated storage shooting image and Emotion element corresponding with the affective state of the user recognized when shooting image.It can recorde user feeling in this way, pay close attention to user feeling, promote the usage experience of user.

Description

Record the method and relevant apparatus of user feeling

Technical field

This application involves emotion recognition and technical field of image processing, in particular to record the method and correlation of user feeling Device.

Background technique

With the promotion of terminal capabilities, most of terminal (such as mobile phone) all has shooting function.User is in using terminal When shooting image, terminal only records the visual effect of image objects, lacks the concern to photographer's emotion.

Specifically, there are many kinds of the factors that influence user feeling, such as current shooting when user's using terminal shoots image Subject matter, shooting when atmosphere, the nearest mood of user etc..Emotion of the user when shooting different images may be different.Currently, After shooting image, user is subsequent when checking the image, oneself feelings when shooting the image can not be perceived from the image The emotion of photographer can not be presented in the mode of sense, i.e., current terminal record image well.

Summary of the invention

This application provides the method and relevant apparatus of record user feeling, the current feelings of the available user of electronic equipment Sense state, and the image checked of associated storage user and the current affective state of user.

In a first aspect, this application provides a kind of methods for recording user feeling, this method comprises: electronic equipment shows the One user interface shows the first image in first user interface；The electronic equipment recognizes user and is currently located in the first feelings Sense state；In response to the first operation detected, the second image of associated storage and first affective state, alternatively, association is deposited Store up second image and the first Emotion element；First Emotion element includes one or more corresponding with first affective state A Emotion element, first Emotion element reflect first affective state；Second image is identical with first image.

The method of first aspect can be applied to photographed scene.Under photographed scene, which can be electronics The user interface that the camera applications installed in equipment provide, the preview that the first image can get for the camera of electronic equipment Image, the second image can be shooting figure picture.When user's preview image and shooting image carry out simultaneously, user is in view-finder The preview image and final shooting figure viewed seem it is identical, i.e. the first image and the second image are identical.

The method of first aspect applies also for checking the scene of picture.Under the scene for checking picture, which can be with From server, electronic equipment local side can be from.

Implement the method for first aspect, electronic equipment in photographed scene or can be checked under the scene of picture and identify user Affective state, and the image for needing associated storage to shoot/check according to user and the current affective state of the user recognized, Alternatively, the image for needing associated storage to shoot/check according to user and emotion corresponding with the current affective state of user member Element.

Second aspect, this application provides a kind of methods for recording user feeling, this method comprises: electronic equipment shows the One user interface shows the first image in first user interface；The electronic equipment recognizes user and is currently located in the first feelings Sense state；In response to the first operation detected, the second image of associated storage and first affective state, alternatively, association is deposited Store up second image and the first Emotion element；First Emotion element includes one or more corresponding with first affective state A Emotion element, first Emotion element reflect first affective state；First image is the preview image before shooting, this Two images are the image of shooting.

The method of second aspect can be applied to photographed scene.Under photographed scene, which can be electronics The user interface that the camera applications installed in equipment provide, the preview that the first image can get for the camera of electronic equipment Image, the second image can be shooting figure picture.Since user's preview image and shooting image may not be to carry out simultaneously, because This, the preview image and final shooting image that user views in view-finder may be identical, it is also possible to different.

Implement the method for second aspect, electronic equipment can identify the affective state of user under photographed scene, and according to User needs associated storage shooting image and the current affective state of the user that recognizes, deposits alternatively, needing to be associated with according to user Storage shooting image and Emotion element corresponding with the current affective state of user.

With reference to first aspect or the method for second aspect, the affective state of user refer to the mood of user, mood or spirit State etc., such as tranquil, angry, detest, sad, pleasant, romantic, happy and fear etc..It is not limited to the above-mentioned several feelings referred to Sense state, the application is in the specific implementation, can also have other affective states, such as pleasure, anger, sorrow, happiness etc., the application to this not It is restricted.

With reference to first aspect or the method for second aspect one or more emotions corresponding with first affective state are first Element include: text information, picture, music, the composition of image, the effect of shadow of image, the saturation degree of image, image tone or The color of image.

With reference to first aspect or the method for second aspect, in some embodiments, the first user interface includes the first interaction Element, first interactive elements are used to monitor the operation for saving second image；First operation includes that the electronic equipment detects To the operation for acting on first interactive elements.Under photographed scene, which can be camera applications offer User interface in shooting control.

With reference to first aspect or the method for second aspect, in some embodiments, which can be by first figure As and the first affective state associated storage in local side or cloud server, alternatively, the electronic equipment is by first figure Picture and the first Emotion element associated storage are in local side or cloud server.

With reference to first aspect or the method for second aspect, in some embodiments, electronic equipment can be according to recognizing User feeling state, subregion store the second image.For example, electronics is set when the first affective state recognized is " happy " It is standby can be by the second image and affective state " happy " associated storage to first area；When the first affective state recognized When for " romance ", electronic equipment can be by the second image and affective state " happy " associated storage to second area；Work as knowledge When the first affective state being clipped to is " happy ", electronic equipment can be deposited the second image and the affective state " calmness " association Store up third region.

With reference to first aspect or the method for second aspect, in some embodiments, electronic equipment can also be according to user's need Reproduce user feeling.Specifically, after electronic device association stores second image and first affective state, alternatively, After the electronic device association stores second image and the first Emotion element, this method may also include that the electronic equipment is shown Second user interface shows one or more third images in the second user interface, which includes second figure Picture；In response to the second operation checked to the second image in the second user interface detected, the electronic equipment is aobvious Show third user interface, also, the second Emotion element is presented in the electronic equipment；The third user interface includes second image； Second Emotion element includes one or more emotion members corresponding with the first affective state of the second image associated storage Element, alternatively, second Emotion element includes first Emotion element.

Here, when electronic equipment shows third user interface, the second Emotion element of presentation can be and second image One of any one or electronic equipment default in the corresponding one or more Emotion elements of the affective state of associated storage, Alternatively, second Emotion element can be and one or more Emotion elements of the second image associated storage in any one Or one of electronic equipment default.

By the method for the reproduction user feeling that the embodiment provides, electronic equipment, can be with after the emotion of record user The purpose of reappearing user emotion when needed promotes the usage experience of user so that user generates affectional sympathetic response.

Electronic equipment can show second user interface in the following two cases, i.e. electronic equipment shows second user circle Before face, this method further includes any one in following two:

1. in one possible implementation, electronic equipment can show fourth user interface, the fourth user interface In show one or more photograph album entries, the one or more photograph album entry include the first photograph album entry, the first photograph album item Mesh corresponds to first affective state；When including that user is in first affective state in the corresponding photograph album of the first photograph album entry The image that the electronic equipment saves；In response to the third operation for acting on the first photograph album entry detected, the electronic equipment Show the second user interface.

2. electronic equipment can show fourth user interface, fourth user circle in alternatively possible implementation One or more photograph album entries are shown in face, which includes the first photograph album entry, first photograph album Entry corresponds to first affective state；It include that user is in first affective state in the corresponding photograph album of first photograph album entry When the electronic equipment save image；The electronic equipment recognizes the second affective state that user is currently at, and shows the 5th User interface；The one or more photograph album entries shown in 5th user interface are by the electronic equipment according to preset recommendation Strategy determines in the one or more photograph album entries shown in the fourth user interface according to second affective state；It should The one or more photograph album entries shown in 5th user interface include the first photograph album entry；It is acted in response to what is detected The third of the first photograph album entry in 5th user interface operates, which shows the second user interface.

Here, preset Generalization bounds can include: 1., when the affective state of active user is " sorrow ", push for user The affective state of user image accessed when being " happiness " or " romance ".Such push strategy can eliminate user not Good mood.2. pushing user or the preset figure of electronic equipment when the affective state of active user is " calmness " for user Picture, such as push image accessed when the affective state of user is " pleasure " or " happy ".

In conjunction with the possible implementation of any one of the above, one or more photograph album entries for being shown in fourth user interface It include: the one or more photograph album entries for corresponding respectively to different affective states.

With reference to first aspect or the method for second aspect, in some embodiments, electronic equipment can save the second figure Before picture, Emotion element corresponding with the affective state of user is presented for user, user is allowed to save the second image Preview before and the corresponding Emotion element of the first affective state.Specifically, electronic equipment is currently located in first recognizing user After affective state, before detecting first operation, this method may also include that electronic equipment presentation and the first emotion shape The corresponding one or more Emotion elements of state.Here, what electronic equipment was presented is one corresponding with first affective state Or multiple Emotion elements, it is that user selects, alternatively, being the electronic equipment default choice, alternatively, being that the electronic equipment is random Selection.One or more Emotion elements corresponding with first affective state in some embodiments, by the electronic equipment Default setting, alternatively, being independently arranged by user.

With reference to first aspect or the method for second aspect, in some embodiments, electronic equipment recognizes user and currently locates Before the first affective state, this method further include: the electronic equipment detects the 4th behaviour for opening emotion recognition service Make, in response to the 4th operation, the affective state that identification user is currently located in；Emotion recognition service is logical for the electronic equipment The hardware device or the hardware device by starting the other equipment for being connected to the electronic equipment of crossing configuration acquire the row of user For data, the affective state that user is currently located in is identified according to the behavioral data of user.

In some embodiments, the 4th operation acts in first user interface including what the electronic equipment detected The operation of second interactive elements, second interactive elements are used to monitor the operation for opening the emotion recognition service.

The third aspect, this application provides a kind of electronic equipment, which includes one or more processors, storage Device and display screen；The memory is coupled with the one or more processors, which, should for storing computer program code Computer program code includes computer instruction, which calls the computer instruction so that the electronics is set It is standby to execute:

The first image is shown in first user interface of display screen display, first user interface；Recognize use Family is currently located in the first affective state；In response to the first operation detected, the second image of associated storage and first emotion State, alternatively, associated storage second image and the first Emotion element；First Emotion element includes and first affective state Corresponding one or more Emotion element, first Emotion element reflect first affective state；Second image and this One image is identical.

The electronic equipment of the third aspect can be applied to photographed scene.Under photographed scene, which can be The user interface that the camera applications installed on electronic equipment provide, what the first image can get for the camera of electronic equipment Preview image, the second image can be shooting figure picture.When user's preview image and shooting image carry out simultaneously, user is finding a view The preview image and final shooting figure viewed in frame seem it is identical, i.e. the first image and the second image are identical.

The electronic equipment of the third aspect applies also for checking the scene of picture.Under the scene for checking picture, the picture It can come from server, can be from electronic equipment local side.

The electronic equipment of the third aspect in photographed scene or can check under the scene of picture the emotion shape for identifying user State, and the image for needing associated storage to shoot/check according to user and the current affective state of the user recognized, alternatively, root The image and Emotion element corresponding with the current affective state of user for needing associated storage to shoot/check according to user.

Fourth aspect, this application provides a kind of electronic equipment, which includes one or more processors, storage Device and display screen；The memory is coupled with the one or more processors, which, should for storing computer program code Computer program code includes computer instruction, which calls the computer instruction so that the electronics is set It is standby to execute:

The first image is shown in first user interface of display screen display, first user interface；Recognize use Family is currently located in the first affective state；In response to the first operation detected, the second image of associated storage and first emotion State, alternatively, associated storage second image and the first Emotion element；First Emotion element includes and first affective state Corresponding one or more Emotion element, first Emotion element reflect first affective state；First image is before shooting Preview image, second image be shooting image.

The electronic equipment of fourth aspect can be applied to photographed scene.Under photographed scene, which can be The user interface that the camera applications installed on electronic equipment provide, what the first image can get for the camera of electronic equipment Preview image, the second image can be shooting figure picture.Since user's preview image and shooting image may not be to carry out simultaneously, Therefore, the preview image and final shooting image that user views in view-finder may be identical, it is also possible to different.

The electronic equipment of fourth aspect can identify the affective state of user under photographed scene, and need to close according to user The connection storage shooting image affective state current with the user recognized, alternatively, needing associated storage to shoot image according to user With Emotion element corresponding with the current affective state of user.

In conjunction with the electronic equipment of the third aspect or fourth aspect, the affective state of user refer to the mood of user, mood or State of mind etc., such as tranquil, angry, detest, sad, pleasant, romantic, happy and fear etc..Be not limited to it is above-mentioned refer to it is several Kind of affective state, the application is in the specific implementation, can also have other affective states, such as pleasure, anger, sorrow, happiness etc., the application couple This is with no restriction.

In conjunction with the electronic equipment of the third aspect or fourth aspect one or more feelings corresponding with first affective state Sense element includes: the color of text information, picture, music, the composition of image, the effect of shadow of image, the saturation degree of image, image The color of tune or image.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, the first user interface includes first Interactive elements, first interactive elements are used to monitor the operation for saving second image；First operation includes the electronic equipment What is detected acts on the operation of first interactive elements.Under photographed scene, which can be camera applications Shooting control in the user interface of offer.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, electronic equipment can by this first Image and the first affective state associated storage in local side or cloud server, alternatively, the electronic equipment by this first Image and the first Emotion element associated storage are in local side or cloud server.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, electronic equipment can be according to identification The user feeling state arrived, subregion store the second image.For example, when the first affective state recognized is " happy ", electricity Sub- equipment can be by the second image and affective state " happy " associated storage to first area；When the first emotion recognized When state is " romance ", electronic equipment can be by the second image and affective state " happy " associated storage to second area； When the first affective state recognized is " happy ", electronic equipment can be closed the second image and the affective state " calmness " Third region is arrived in connection storage.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, electronic equipment can also according to Family needs to reproduce user feeling.Specifically, the one or more processors are also used to call the computer instruction so that the electricity Sub- equipment executes:

After associated storage second image and first affective state, alternatively, associated storage second image and After one Emotion element, at the display screen display second user interface, one or more is shown in the second user interface Third image, the third image include second image；

In response to the second operation checked to the second image in the second user interface detected, in the display Screen display third user interface, also, the second Emotion element is presented；The third user interface includes second image；This Two Emotion elements include one or more Emotion elements corresponding with the first affective state of the second image associated storage, or Person, second Emotion element include first Emotion element.

Electronic equipment can show second user interface in the following two cases, i.e. the one or more processors are also used In the calling computer instruction so that the electronic equipment executes following any one:

1., in the display screen display fourth user interface, this before the display screen display second user interface One or more photograph album entries are shown in four user interfaces, which includes the first photograph album entry, should First photograph album entry corresponds to first affective state；In the corresponding photograph album of first photograph album entry include user be in this first The image that the electronic equipment saves when affective state；It is operated in response to the third for acting on the first photograph album entry detected, At the display screen display second user interface.

2., in the display screen display fourth user interface, this before the display screen display second user interface One or more photograph album entries are shown in four user interfaces, which includes the first photograph album entry, should First photograph album entry corresponds to first affective state；In the corresponding photograph album of first photograph album entry include user be in this first The image that the electronic equipment saves when affective state；The second affective state that user is currently at is recognized, and in the display screen The 5th user interface of upper display；The one or more photograph album entries shown in 5th user interface be by the electronic equipment according to Preset Generalization bounds, in the one or more photograph album entries shown in the fourth user interface according to second affective state Determining；The one or more photograph album entries shown in 5th user interface include the first photograph album entry；In response to detection To the third operation for acting on the first photograph album entry in the 5th user interface, which shows second user circle Face.

The one or more photograph album entries shown in fourth user interface include: to correspond respectively to different affective states One or more photograph album entries.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, electronic equipment can save the Before two images, Emotion element corresponding with the affective state of user is presented for user, user is allowed to save second Preview and the corresponding Emotion element of the first affective state before image.Specifically, the one or more processors are also used to call The computer instruction is so that the electronic equipment executes:

User is recognized to be currently located in after the first affective state, detect this first operation before, present and this first The corresponding one or more Emotion elements of affective state.In some embodiments, the electronic equipment present and first feelings The corresponding one or more Emotion elements of sense state, are that user selects, alternatively, be the electronic equipment default choice, or Person is that the electronic equipment is randomly selected.

In conjunction with the electronic equipment of the third aspect or fourth aspect, in some embodiments, the one or more processors are also For calling the computer instruction so that the electronic equipment executes:

It recognizes user to be currently located in front of the first affective state, detects the 4th behaviour for opening emotion recognition service Make, in response to the 4th operation, the affective state that identification user is currently located in；Emotion recognition service is logical for the electronic equipment The hardware device or the hardware device by starting the other equipment for being connected to the electronic equipment of crossing configuration acquire the row of user For data, the affective state that user is currently located in is identified according to the behavioral data of user.

In some embodiments, the 4th operation acts in first user interface including what the electronic equipment detected The second interactive elements operation, second interactive elements be used for monitors unlatching the emotion recognition service operation.

5th aspect, the application provides a kind of computer program product comprising instruction, when above-mentioned computer program product When running on an electronic device, so that above-mentioned electronic equipment is executed such as any possible realization in first aspect and first aspect The method that mode describes.

6th aspect, the application provides a kind of computer program product comprising instruction, when above-mentioned computer program product When running on an electronic device, so that above-mentioned electronic equipment is executed such as any possible realization in second aspect and second aspect The method that mode describes.

7th aspect, the application provides a kind of computer readable storage medium, including instruction, when above-metioned instruction is set in electronics When standby upper operation, so that above-mentioned electronic equipment is executed such as possible implementation description any in first aspect and first aspect Method.

Eighth aspect, the application provides a kind of computer readable storage medium, including instruction, when above-metioned instruction is set in electronics When standby upper operation, so that above-mentioned electronic equipment is executed such as possible implementation description any in second aspect and second aspect Method.

Implement technical solution provided by the present application, electronic equipment can record or save the feelings of user when obtaining image Sense, can achieve concern user feeling, and the purpose of reappearing user emotion when needed, improving user makes electronic equipment With experience.

Detailed description of the invention

Technical solution in ord to more clearly illustrate embodiments of the present application or in background technique below will be implemented the application Attached drawing needed in example or background technique is illustrated.

Figure 1A is the structural schematic diagram of electronic equipment provided by the embodiments of the present application；

Figure 1B is the software architecture diagram of electronic equipment provided by the embodiments of the present application；

Fig. 2A-Fig. 2 B is a kind of human-computer interaction schematic diagram for opening " emotion recognition " provided by the embodiments of the present application；

Fig. 3 A- Fig. 3 H is the human-computer interaction schematic diagram that user feeling is recorded under photographed scene provided by the embodiments of the present application；

Fig. 4 A- Fig. 4 H is the human-computer interaction signal provided by the embodiments of the present application checked under picture scene and record user feeling Figure；

Fig. 5 A- Fig. 5 K, Fig. 6 A- Fig. 6 B are the human-computer interaction schematic diagram provided by the embodiments of the present application for reproducing user feeling；

Fig. 7 A- Fig. 7 D is the human-computer interaction schematic diagram of user setting Emotion element provided by the embodiments of the present application.

Specific embodiment

Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application is described.

Wherein, in the description of the embodiment of the present application, unless otherwise indicated, "/" indicate or the meaning, for example, A/B can be with Indicate A or B；"and/or" herein is only a kind of incidence relation for describing affiliated partner, indicates that there may be three kinds of passes System, for example, A and/or B, can indicate: individualism A exists simultaneously A and B, these three situations of individualism B.In addition, at this In the description for applying for embodiment, " multiple " refer to two or more.

Hereinafter, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or Implicitly include one or more of the features.In the description of the embodiment of the present application, unless otherwise indicated, " multiples' " contains Justice is two or more.

The application following embodiment provides a kind of method and relevant apparatus for recording user feeling, so that electronic equipment can To record when obtaining image or save the emotion of user, concern user feeling can achieve, and reappearing user feelings when needed The purpose of sense improves user to the usage experience of electronic equipment.

In the application following embodiment, opened at " emotion recognition (the emotion recognition) " of electronic equipment Under conditions of, when the electronic equipment identifies that user checks the scene of image, the available user of the electronic equipment is current Affective state, and the associated storage image and the current affective state of user.If user checks the image, electronic equipment again Can be while the image be presented for user, Emotion element corresponding to the affective state of presentation and the image associated storage, I.e. electronic equipment can go back the emotion of original subscriber, cause user's sympathetic response, give the better usage experience of user.About affective state with And the definition of Emotion element can refer to the associated description of subsequent embodiment.

In the application following embodiment, image that user checks can include: the figure that the camera of electronic equipment is got As (such as image that view-finder includes in the shooting interface of camera applications offer), electronic equipment are from network or other equipment Obtain and store local image, electronic equipment access cloud device on the image that is stored etc..Wherein, which can be with For static images, dynamic picture and video etc..

In the application following embodiment, " emotion recognition " can be a kind of services or functionalities of electronic equipment offer, can To support electronic equipment to get the affective state of user.The affective state of user refers to the mood, mood or spiritual shape of user State etc., such as tranquil, angry, detest, sad, pleasant, romantic, happy and fear etc..It is not limited to the above-mentioned several emotions referred to State, for the application in the specific implementation, can also have other affective states, such as pleasure, anger, sorrow, happiness etc., the application does not make this Limitation.

In some embodiments, " emotion recognition " can support the electronic equipment to pass through artificial intelligence (artificial Intelligence, AI) technology analyzes the behavioral data of user, to recognize the affective state of user.In this Shen Please in, open " emotion recognition " may include enable electronic equipment hardware device and/or starting be connected to the electronic equipment The hardware device of other equipment identifies according to the behavioral data of the user emotion shape of user to acquire the behavioral data of user State.These hardware devices user behavior data collected can include: 1. non-physiologic data.Such as the acquisition of camera 193 Face, facial expression and movement, the voice of microphone 170C acquisition, the ambient light of ambient light sensor 180L acquisition, touch Shield 194 collected user's typing speeds and grammer.2. physiological data.Such as it is connected to the motion bracelet or intelligence of electronic equipment Heart rate, pulse, respiratory rate and the Skin Resistance etc. of energy wrist-watch acquisition.Electronic equipment can be according to collected user behavior number According to, pass through AI technology analyze user affective state.For example, when user's mouth both sides are upturned, canthus it is micro- stick up, heart rate it is flat Surely, when breathing is releived, Skin Resistance is higher, the user mood is pleasant；When user's expression is terrified, heart rate accelerates, it is short of breath, skin Skin impedance is more low, which is in terrified and fears state；When the amplification of user's pupil, both hands are clenched fist, which is in indignation State.

In further embodiments, " emotion recognition " can support behavioral data of the electronic equipment by AI technology to user It is analyzed, the immediate affective state of virtual condition with active user is matched in the presumable affective state of user, i.e., Recognize the current affective state of user.Here, the possible affective state of user refers to the emotion to match with the personality of user State.For example, the possible affective state of the user of optimism type may include pleasant, romantic, happy, the user of careful inhibitable type Possible affective state may include calmness, sorrow etc..Wherein, the personality of user can be passed through AI technology to user by electronic equipment Previous behavioral data is analyzed to obtain.The possible affective state of the user of different characters can be stored in advance in electronic equipment Or in cloud server.

Intelligible, the AI technology that the embodiment of the present application refers to can be machine learning algorithm.The machine learning algorithm can Think deep learning (deep learning) algorithm, it may include one or more of: convolutional neural networks (convolutional neural network, CNN), neighbour (k-NearestNeighbor, KNN) sorting algorithm, circulation mind Through network (recurrent neural network, RNN) or statistic algorithm etc..

" emotion recognition " can support electronic equipment to receive the affective state of user's input or selection in yet other embodiments, Instruction information.Opening " emotion recognition " may include the finger that electronic equipment receives the affective state that user directly inputs or selects Show information, to recognize the current affective state of user according to the instruction information.User can be inputted on touch screen 194 or Person selects the instruction information of affective state, can also input voice by microphone 170C to input or select affective state Instruction information, can also be by rocking or gesture is come instruction information for inputting or selecting affective state etc., the application is real It is without limitation to apply example.Here, the instruction information of affective state can be icon, text or voice etc..

It is understood that " emotion recognition ", " emotion recognition " are one used in the present embodiment A word, the meaning represented have been recorded in the present embodiment, and title can not constitute any restrictions to the present embodiment.Separately Outside, in some other embodiment of the application, " emotion recognition " can also be referred to as such as other nouns such as " mental state identification ".Together Sample, " the emotion recognition " mentioned in the embodiment of the present application can also be referred to as in some other embodiment Such as other names such as " affective recognition ".

In the embodiment of the present application, Emotion element refers to the element that can reflect the affective state of user.Emotion element can Including but not limited to: text information, picture (such as expression packet), music, the composition of image, effect of shadow, saturation degree, tone and Color etc..Wherein, the mode of composition of image may include nine grids mode of composition, guide line mode of composition, diagonal line mode of composition, Three points of modes of composition, the mode of composition etc. for being left white mode of composition, foreground/background virtualization.The effect of shadow of image may include strong light Shadow, shadow of ashing processing etc..Saturation degree may include high saturation, low saturation etc..Tone may include cool tone, warm tones, Neutralc tint etc..Color may include gorgeous, dim, gray scale, normal etc..

Different Emotion elements can give different impressions, i.e., different Emotion elements can correspond to different emotion shapes State.For example, (picture is divided in the form of golden section), image color are gorgeous when the mode of composition of image is three points of compositions When, visual pleasant feeling and lively sense can be promoted, can correspond to affective state " happy ".In another example the mode of composition of image When to be left white composition, shadow ashing, color dimness, feeling of lonely can be given, can correspond to affective state " sorrow ".It is appreciated that , when user sees these Emotion elements, it can be perceive intuitively that the corresponding affective state of the Emotion element.At this Apply in embodiment, a kind of affective state can be corresponding with multiple Emotion elements.Here, the member of emotion corresponding to different emotions state The determination method or setting method of element can refer to the associated description of subsequent embodiment, wouldn't repeat here.

In the embodiment of the present application, the corresponding Emotion element of different emotions state can store in electronic equipment local End, also can store server beyond the clouds, the application is without limitation.

In the following, introducing electronic equipment involved in the embodiment of the present application first.Class of the application to the electronic equipment referred to Type is not specifically limited, and electronic equipment can be mobile phone, tablet computer, personal digital assistant (personal digital Assistant, PDA), wearable device, the portable electronic devices such as laptop computer (laptop).Portable electronic device Exemplary embodiment include but is not limited to the portable electric for carrying iOS, android, microsoft or other operating systems Sub- equipment.Above-mentioned portable electronic device is also possible to other portable electronic devices, such as with touch sensitive surface (such as touch-control Panel) laptop computer (laptop) etc..It is to be further understood that electronics is set in some other embodiment of the application It is standby to may not be portable electronic device, but there is the desktop computer of touch sensitive surface (such as touch panel), either Intelligent TV set etc..

The structural schematic diagram of example electronic device 100 provided by the embodiments of the present application is shown referring to Figure 1A, Figure 1A.

Figure 1A shows the structural schematic diagram of electronic equipment 100.

Electronic equipment 100 may include processor 110, external memory interface 120, internal storage 121, general serial Bus (universal serial bus, USB) interface 130, charge management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio-frequency module 170, loudspeaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, key 190, motor 191, indicator 192, camera 193, display screen 194 and Subscriber Identity Module (subscriber identification module, SIM) card interface 195 Deng.Wherein sensor module 180 may include pressure sensor 180A, gyro sensor 180B, baroceptor 180C, magnetic Sensor 180D, acceleration transducer 180E, range sensor 180F, close to optical sensor 180G, fingerprint sensor 180H, temperature Spend sensor 180J, touch sensor 180K, ambient light sensor 180L, bone conduction sensor 180M etc..

It is understood that the structure of the embodiment of the present application signal does not constitute the specific restriction to electronic equipment 100.? In other embodiments of the application, electronic equipment 100 may include than illustrating more or fewer components, or the certain portions of combination Part perhaps splits certain components or different component layouts.The component of diagram can be with hardware, software or software and hardware Combination realize.

Processor 110 may include one or more processing units, such as: processor 110 may include application processor (application processor, AP), modem processor, graphics processor (graphics processing Unit, GPU), image-signal processor (image signal processor, ISP), controller, Video Codec, number Signal processor (digital signal processor, DSP), baseband processor and/or neural network processor (neural-network processing unit, NPU) etc..Wherein, different processing units can be independent device, It can integrate in one or more processors.In some embodiments of the application, processor 110 can be according to the user got Behavioral data recognizes the affective state of user.Specific identification method can refer to associated description above.

Controller can generate operating control signal according to instruction operation code and clock signal, complete instruction fetch and execution The control of instruction.

Memory can also be set in processor 110, for storing instruction and data.In some embodiments, processor Memory in 110 is cache memory.The memory can save the instruction that processor 110 is just used or is recycled Or data.If processor 110 needs to reuse the instruction or data, can be called directly from the memory.It avoids Repeated access, reduces the waiting time of processor 110, thus improves the efficiency of system.

In some embodiments, processor 110 may include one or more interfaces.Interface may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit Sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiving-transmitting transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), universal input export (general-purpose Input/output, GPIO) interface, Subscriber Identity Module (subscriber identity module, SIM) interface, and/or Universal serial bus (universal serial bus, USB) interface etc..

I2C interface is a kind of bi-directional synchronization universal serial bus, including serial data line (serial data line, SDA) He Yigen serial time clock line (derail clock line, SCL).In some embodiments, processor 110 may include Multiple groups I2C bus.Processor 110 can by different I2C bus interface distinguish coupled with touch sensors 180K, charger, Flash lamp, camera 193 etc..Such as: processor 110 can make processor by I2C interface coupled with touch sensors 180K 110 are communicated with touch sensor 180K by I2C bus interface, realize the touch function of electronic equipment 100.

I2S interface can be used for voice communication.In some embodiments, processor 110 may include multiple groups I2S bus. Processor 110 can be coupled by I2S bus with audio-frequency module 170, be realized logical between processor 110 and audio-frequency module 170 Letter.In some embodiments, audio-frequency module 170 can transmit audio signal to wireless communication module 160 by I2S interface, real The function of now being received calls by bluetooth headset.

Pcm interface can be used for voice communication, by analog signal sampling, quantization and coding.In some embodiments, sound Frequency module 170 can be coupled with wireless communication module 160 by pcm bus interface.In some embodiments, audio-frequency module 170 Audio signal can also be transmitted to wireless communication module 160 by pcm interface, realize the function to receive calls by bluetooth headset Energy.The I2S interface and the pcm interface may be used to voice communication.

UART interface is a kind of Universal Serial Bus, is used for asynchronous communication.The bus can be bidirectional communications bus. The data that it will be transmitted are converted between serial communication and parallel communications.In some embodiments, UART interface usually by with In connection processor 110 and wireless communication module 160.Such as: processor 110 passes through UART interface and wireless communication module 160 In bluetooth module communication, realize Bluetooth function.In some embodiments, audio-frequency module 170 can be by UART interface to nothing Line communication module 160 transmits audio signal, realizes the function that music is played by bluetooth headset.

MIPI interface can be used to connect the peripheral components such as processor 110 and display screen 194, camera 193.MIPI connects Mouth includes camera serial line interface (camera serial interface, CSI), display screen serial line interface (display Serial interface, DSI) etc..In some embodiments, processor 110 and camera 193 are communicated by CSI interface, real The shooting function of existing electronic equipment 100.Processor 110 and display screen 194 realize electronic equipment 100 by DSI interface communication Display function.

GPIO interface can pass through software configuration.GPIO interface can be configured as control signal, may be alternatively configured as counting It is believed that number.In some embodiments, GPIO interface can be used for connecting processor 110 and camera 193, display screen 194, wirelessly Communication module 160, audio-frequency module 170, sensor module 180 etc..GPIO interface can be additionally configured to I2C interface, and I2S connects Mouthful, UART interface, MIPI interface etc..

Usb 1 30 is the interface for meeting USB standard specification, specifically can be Mini USB interface, and Micro USB connects Mouthful, USB Type C interface etc..Usb 1 30 can be used for connecting charger for the charging of electronic equipment 100, can be used for Data are transmitted between electronic equipment 100 and peripheral equipment.It can be used for connection earphone, audio played by earphone.The interface It can be also used for connecting other electronic equipments, such as AR equipment etc..

It is understood that the interface connection relationship of each intermodule of the embodiment of the present application signal, only schematically illustrates, The structure qualification to electronic equipment 100 is not constituted.In other embodiments of the application, electronic equipment 100 can also be used The combination of different interface connection type or multiple interfaces connection type in above-described embodiment.

Charge management module 140 is used to receive charging input from charger.Wherein, charger can be wireless charger, It is also possible to wired charger.In the embodiment of some wired chargings, charge management module 140 can pass through usb 1 30 Receive the charging input of wired charger.In the embodiment of some wireless chargings, charge management module 140 can pass through electronics The Wireless charging coil of equipment 100 receives wireless charging input.While charge management module 140 is that battery 142 charges, may be used also To be power electronic equipment by power management module 141.

Power management module 141 is for connecting battery 142, charge management module 140 and processor 110.Power management mould Block 141 receives the input of battery 142 and/or charge management module 140, is processor 110, internal storage 121, display screen 194, the power supply such as camera 193 and wireless communication module 160.Power management module 141 can be also used for monitoring battery capacity, Circulating battery number, the parameters such as cell health state (electric leakage, impedance).In some other embodiment, power management module 141 Also it can be set in processor 110.In further embodiments, power management module 141 and charge management module 140 can also To be set in the same device.

The wireless communication function of electronic equipment 100 can pass through antenna 1, antenna 2, mobile communication module 150, wireless communication Module 160, modem processor and baseband processor etc. are realized.

Antenna 1 and antenna 2 electromagnetic wave signal for transmitting and receiving.Each antenna in electronic equipment 100 can be used for covering Cover single or multiple communication bands.Different antennas can also be multiplexed, to improve the utilization rate of antenna.Such as: it can be by antenna 1 It is multiplexed with the diversity antenna of WLAN.In other embodiments, antenna can be used in combination with tuning switch.

Mobile communication module 150, which can provide, applies wirelessly communicating on electronic equipment 100 including 2G/3G/4G/5G etc. Solution.Mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (low noise amplifier, LNA) etc..Mobile communication module 150 can receive electromagnetic wave by antenna 1, and to received electricity Magnetic wave is filtered, and the processing such as amplification is sent to modem processor and is demodulated.Mobile communication module 150 can also be right The modulated modulated signal amplification of demodulation processor, switchs to electromagenetic wave radiation through antenna 1 and goes out.In some embodiments, it moves At least partly functional module of dynamic communication module 150 can be arranged in processor 110.In some embodiments, mobile logical At least partly functional module of letter module 150 can be arranged in the same device at least partly module of processor 110.

Modem processor may include modulator and demodulator.Wherein, modulator is used for low frequency base to be sent Band signal is modulated into high frequency signal.Demodulator is used to received electromagnetic wave signal being demodulated into low frequency baseband signal.Then solution Adjust device that the low frequency baseband signal that demodulation obtains is sent to baseband processor.Low frequency baseband signal is through baseband processor Afterwards, it is delivered to application processor.Application processor is defeated by audio frequency apparatus (being not limited to loudspeaker 170A, receiver 170B etc.) Voice signal out, or image or video are shown by display screen 194.In some embodiments, modem processor can be Independent device.In further embodiments, modem processor can be independently of processor 110, with mobile communication module 150 or other function module be arranged in the same device.

It includes WLAN (wireless that wireless communication module 160, which can be provided and be applied on electronic equipment 100, Local area networks, WLAN) (such as Wireless Fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), Global Navigation Satellite System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), the short distance wireless communication technology (near field communication, NFC) are red The solution of the wireless communications such as outer technology (infrared, IR).Wireless communication module 160 can be integrated into few communication One or more devices of processing module.Wireless communication module 160 receives electromagnetic wave via antenna 2, by electromagnetic wave signal frequency modulation And filtering processing, by treated, signal is sent to processor 110.Wireless communication module 160 can also connect from processor 110 Signal to be sent is received, frequency modulation is carried out to it, is amplified, is switched to electromagenetic wave radiation through antenna 2 and go out.

In some embodiments, the antenna 1 of electronic equipment 100 and mobile communication module 150 couple, antenna 2 and channel radio Believe that module 160 couples, allowing electronic equipment 100, technology is communicated with network and other equipment by wireless communication.It is described Wireless communication technique may include global system for mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), CDMA access (code Division multiple access, CDMA), wideband code division multiple access (wideband code division multiple Access, WCDMA), time division CDMA (time-division code division multiple access, TD- SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM and/or IR technology etc..Institute Stating GNSS may include GPS (global positioning system, GPS), global navigational satellite system It unites (global navigation satellite system, GLONASS), Beidou satellite navigation system (beidou Navigation satellite system, BDS), quasi- zenith satellite system (quasi-zenith satellite System, QZSS) and/or satellite-based augmentation system (satellite based augmentation systems, SBAS).

Electronic equipment 100 realizes display function by GPU, display screen 194 and application processor etc..GPU is at image The microprocessor of reason connects display screen 194 and application processor.GPU is calculated for executing mathematics and geometry, is used for figure wash with watercolours Dye.Processor 110 may include one or more GPU, execute program instructions to generate or change display information.

Display screen 194 is for showing image, video etc..Display screen 194 includes display panel.Display panel can use liquid Crystal display screen (liquid crystal display, LCD), Organic Light Emitting Diode (organic light-emitting Diode, OLED), active matrix organic light-emitting diode or active-matrix organic light emitting diode (active-matrix Organic light emitting diode's, AMOLED), Flexible light-emitting diodes (flex light-emitting Diode, FLED), Miniled, MicroLed, Micro-oLed, light emitting diode with quantum dots (quantum dot light Emitting diodes, QLED) etc..In some embodiments, electronic equipment 100 may include 1 or N number of display screen 194, N For the positive integer greater than 1.

Electronic equipment 100 can be by ISP, camera 193, Video Codec, GPU, display screen 194 and at It manages device etc. and realizes shooting function.

ISP is used to handle the data of the feedback of camera 193.For example, opening shutter when taking pictures, light is passed by camera lens It is delivered on camera photosensitive element, optical signal is converted to electric signal, and camera photosensitive element passes to the electric signal at ISP Reason, is converted into macroscopic image.ISP can also be to the noise of image, brightness, colour of skin progress algorithm optimization.ISP can be with Exposure to photographed scene, the parameter optimizations such as colour temperature.In some embodiments, ISP can be set in camera 193.

Camera 193 is for capturing still image or video.Object generates optical imagery by camera lens and projects photosensitive member Part.Photosensitive element can be charge-coupled device (charge coupled device, CCD) or complementary metal oxide is partly led Body (complementary metal-oxide-semiconductor, CMOS) phototransistor.Photosensitive element turns optical signal It changes electric signal into, electric signal is passed into ISP later and is converted into data image signal.Data image signal is output to DSP by ISP Working process.Data image signal is converted into the RGB of standard, the picture signal of the formats such as YUV by DSP.In some embodiments, Electronic equipment 100 may include 1 or N number of camera 193, and N is the positive integer greater than 1.In some embodiments, camera 193 can be used for acquiring face, facial expression and the movement of user, and the affective state of user is recognized with electronic device.

Digital signal processor, in addition to can handle data image signal, can also handle it for handling digital signal His digital signal.For example, digital signal processor is used to carry out Fu to frequency point energy when electronic equipment 100 is when frequency point selects In leaf transformation etc..

Video Codec is used for compression of digital video or decompression.Electronic equipment 100 can be supported one or more Video Codec.In this way, electronic equipment 100 can play or record the video of a variety of coded formats, and such as: dynamic image is special Family's group (moving picture experts group, MPEG) 1, MPEG2, mpeg 3, MPEG4 etc..

NPU is neural network (neural-network, NN) computation processor, by using for reference biological neural network structure, Such as transfer mode between human brain neuron is used for reference, it, can also continuous self study to input information fast processing.Pass through NPU The application such as intelligent cognition of electronic equipment 100 may be implemented, such as: image recognition, recognition of face, speech recognition, text understanding Deng.

External memory interface 120 can be used for connecting external memory card, such as Micro SD card, realize that extension electronics is set Standby 100 storage capacity.External memory card is communicated by external memory interface 120 with processor 110, realizes that data store function Energy.Such as by music, the files such as video are stored in external memory card.

Internal storage 121 can be used for storing computer executable program code, and the executable program code includes Instruction.Internal storage 121 may include storing program area and storage data area.Wherein, storing program area can store operation system It unites, application program (such as sound-playing function, image player function etc.) needed at least one function etc..It storage data area can The data (such as audio data, phone directory etc.) etc. created in storage 100 use process of electronic equipment.In addition, storage inside Device 121 may include high-speed random access memory, can also include nonvolatile memory, for example, at least a disk storage Device, flush memory device, generic flash memory (universal flash storage, UFS) etc..Processor 110 passes through operation It is stored in the instruction of internal storage 121, and/or is stored in the instruction for the memory being set in processor, electronics is executed and sets Standby 100 various function application and data processing.In some embodiments of the application, internal storage 121 can be used for storing The corresponding one or more Emotion elements of different emotions state.In some embodiments, internal storage 121 can also be used in The affective state of user when image and acquisition image that associated storage electronic equipment 100 is got.In further embodiments, interior Portion's memory 121 can also be used in the image that associated storage electronic equipment 100 is got and the emotion shape with user when obtaining image The corresponding Emotion element of state.

Electronic equipment 100 can pass through audio-frequency module 170, loudspeaker 170A, receiver 170B, microphone 170C, earphone Interface 170D and application processor etc. realize audio-frequency function.Such as music, recording etc..

Audio-frequency module 170 is used to for digitized audio message to be converted into analog audio signal output, is also used for analogue audio frequency Input is converted to digital audio and video signals.Audio-frequency module 170 can be also used for audio-frequency signal coding and decoding.In some embodiments In, audio-frequency module 170 can be set in processor 110, or the partial function module of audio-frequency module 170 is set to processor In 110.

Loudspeaker 170A, also referred to as " loudspeaker ", for audio electrical signal to be converted to voice signal.Electronic equipment 100 can be with Music is listened to by loudspeaker 170A, or listens to hand-free call.In some embodiments, loudspeaker 170A can be used for exporting audio Class Emotion element (such as background music).In further embodiments, loudspeaker 170A can also be used to input speech prompt information, example Such as prompting user to open the voice of " emotion recognition ".

Receiver 170B, also referred to as " earpiece ", for audio electrical signal to be converted into voice signal.When electronic equipment 100 connects It answers a call or when voice messaging, it can be by the way that receiver 170B be answered voice close to human ear.

Microphone 170C, also referred to as " microphone ", " microphone ", for voice signal to be converted to electric signal.When making a phone call Or when sending voice messaging, voice signal can be input to microphone by mouth close to microphone 170C sounding by user 170C.At least one microphone 170C can be set in electronic equipment 100.In further embodiments, electronic equipment 100 can be set Two microphone 170C are set, in addition to collected sound signal, can also realize decrease of noise functions.In further embodiments, electronics is set Standby 100 can also be arranged three, four or more microphone 170C, realize that collected sound signal, noise reduction can also identify sound Directional recording function etc. is realized in source.

Earphone interface 170D is for connecting wired earphone.Earphone interface 170D can be usb 1 30, be also possible to Opening mobile electronic device platform (open mobile terminal platform, OMTP) standard interface of 3.5mm, the U.S. Cellular telecommunication industrial association (cellular telecommunications industry association of the USA, CTIA) standard interface.

Pressure signal can be converted into electric signal for experiencing pressure signal by pressure sensor 180A.In some implementations In example, pressure sensor 180A be can be set in display screen 194.There are many type of pressure sensor 180A, such as resistive pressure Sensor, inductance pressure transducer, capacitance pressure transducer, etc..Capacitance pressure transducer, can be including at least two Parallel-plate with conductive material.When effectively acting on pressure sensor 180A, the capacitor between electrode changes.Electronic equipment 100 determine the intensity of pressure according to the variation of capacitor.When there is touch operation to act on display screen 194, electronic equipment 100 is according to pressure Force snesor 180A detects the touch operation intensity.Electronic equipment 100 can also be believed according to the detection of pressure sensor 180A Number calculate touch position.In some embodiments, identical touch location, but the touch behaviour of different touch operation intensity are acted on Make, different operational orders can be corresponded to.Such as: when the touch operation for having touch operation intensity to be less than first pressure threshold value acts on When short message application icon, the instruction for checking short message is executed.When have touch operation intensity be greater than or equal to first pressure threshold When the touch operation of value acts on short message application icon, the instruction of newly-built short message is executed.

Gyro sensor 180B is determined for the athletic posture of electronic equipment 100.It in some embodiments, can be with Determine that electronic equipment 100 surrounds the angular speed of three axis (that is, x, y and z-axis) by gyro sensor 180B.Gyro sensors Device 180B can be used for shooting stabilization.Illustratively, when pressing shutter, gyro sensor 180B detection electronic equipment 100 is trembled Dynamic angle goes out the distance that lens module needs to compensate according to angle calculation, camera lens is allowed to offset electronic equipment by counter motion Stabilization is realized in 100 shake.Gyro sensor 180B can be also used for navigating, somatic sensation television game scene.

Baroceptor 180C is for measuring air pressure.In some embodiments, electronic equipment 100 passes through baroceptor The atmospheric pressure value that 180C is measured calculates height above sea level, auxiliary positioning and navigation.

Magnetic Sensor 180D includes Hall sensor.Electronic equipment 100 can use Magnetic Sensor 180D flip cover skin The folding of set.In some embodiments, when electronic equipment 100 is liding machine, electronic equipment 100 can be according to Magnetic Sensor The folding of 180D flip cover.And then according to the folding condition of the leather sheath detected or the folding condition of flip lid, setting flip lid is certainly The characteristics such as dynamic unlock.

Acceleration transducer 180E can detect the big of (the generally three axis) acceleration in all directions of electronic equipment 100 It is small.It can detect that size and the direction of gravity when electronic equipment 100 is static.It can be also used for identification electronic equipment posture, answer Switch for horizontal/vertical screen, the application such as pedometer.

Range sensor 180F, for measuring distance.Electronic equipment 100 can pass through infrared or laser distance measuring.? In some embodiments, photographed scene, electronic equipment 100 can use range sensor 180F ranging to realize rapid focus.

It may include such as light emitting diode (LED) and photodetector, such as photodiode close to optical sensor 180G. Light emitting diode can be infrared light-emitting diode.Electronic equipment 100 launches outward infrared light by light emitting diode.Electronics is set Standby 100 detect the infrared external reflection light from neighbouring object using photodiode.It, can be true when detecting sufficient reflected light Determining electronic equipment 100 nearby has object.When detecting insufficient reflected light, electronic equipment 100 can determine electronic equipment 100 do not have object nearby.Electronic equipment 100 can use to be pasted close to optical sensor 180G detection user's hand-hold electronic equipments 100 Nearly ear call, so that automatic distinguishing screen achievees the purpose that power saving.It can also be used for leather sheath mode, mouth close to optical sensor 180G Bag mode automatic unlocking and screen locking.

Ambient light sensor 180L is for perceiving environmental light brightness.Electronic equipment 100 can be according to the environment bright of perception Spend 194 brightness of automatic adjusument display screen.Automatic white balance adjustment when ambient light sensor 180L can also be used for taking pictures.Environment light Sensor 180L can also cooperate with close to optical sensor 180G, electronic equipment 100 be detected whether in pocket, with false-touch prevention.

Fingerprint sensor 180H is for acquiring fingerprint.The fingerprint characteristic that electronic equipment 100 can use acquisition realizes fingerprint Unlock accesses application lock, and fingerprint is taken pictures, fingerprint incoming call answering etc..

Temperature sensor 180J is for detecting temperature.In some embodiments, electronic equipment 100 utilizes temperature sensor The temperature of 180J detection, executes Temperature Treatment strategy.For example, when the temperature sensor 180J temperature reported is more than threshold value, electronics Equipment 100 executes the performance for reducing the processor being located near temperature sensor 180J, implements Thermal protection to reduce power consumption.? In other embodiments, when temperature is lower than another threshold value, electronic equipment 100 heats battery 142, leads to electricity to avoid low temperature The abnormal shutdown of sub- equipment 100.In some other embodiment, when temperature is lower than another threshold value, electronic equipment 100 is to battery 142 output voltage executes boosting, to avoid shutting down extremely caused by low temperature.

Touch sensor 180K, also referred to as " touch panel ".Touch sensor 180K can be set in display screen 194, by touching It touches sensor 180K and display screen 194 forms touch screen, also referred to as " touch screen ".Touch sensor 180K acts on it for detecting On or near touch operation.The touch operation that touch sensor can will test passes to application processor, to determine touching Touch event type.Visual output relevant to touch operation can be provided by display screen 194.In further embodiments, it touches Touching sensor 180K also can be set in the surface of electronic equipment 100, different from the location of display screen 194.

The available vibration signal of bone conduction sensor 180M.In some embodiments, bone conduction sensor 180M can be with Obtain the vibration signal of human body part vibration bone block.Bone conduction sensor 180M can also contact human pulse, receive blood pressure and jump Dynamic signal.In some embodiments, bone conduction sensor 180M also can be set in earphone, be combined into bone conduction earphone.Sound Frequency module 170 can parse voice based on the vibration signal for the part vibration bone block that the bone conduction sensor 180M is obtained Signal realizes phonetic function.The blood pressure jitter solution that application processor can be obtained based on the bone conduction sensor 180M Heart rate information is analysed, realizes heart rate detecting function.

Key 190 includes power button, volume key etc..Key 190 can be mechanical key.It is also possible to touch-key. Electronic equipment 100 can receive key-press input, generate key letter related with the user setting of electronic equipment 100 and function control Number input.

Motor 191 can produce vibration prompt.Motor 191 can be used for calling vibration prompt, can be used for touching vibration Dynamic feedback.For example, acting on the touch operation of different application (such as taking pictures, audio broadcasting etc.), different vibrations can be corresponded to Feedback effects.The touch operation of 194 different zones of display screen is acted on, motor 191 can also correspond to different vibrational feedback effects. Different application scenarios (such as: time alarm receives information, alarm clock, game etc.) different vibrational feedback effects can also be corresponded to Fruit.Touch vibrational feedback effect can also be supported customized.

Indicator 192 can be indicator light, can serve to indicate that charged state, electric quantity change can be used for instruction and disappear Breath, missed call, notice etc..

SIM card interface 195 is for connecting SIM card.SIM card can be by being inserted into SIM card interface 195, or from SIM card interface 195 extract, and realization is contacting and separating with electronic equipment 100.Electronic equipment 100 can support 1 or N number of SIM card interface, N For the positive integer greater than 1.SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc..The same SIM Card interface 195 can be inserted into multiple cards simultaneously.The type of multiple cards may be the same or different.SIM card interface 195 Different types of SIM card can also be compatible with.SIM card interface 195 can also be with compatible external storage card.Electronic equipment 100 passes through SIM Card and network interaction realize the functions such as call and data communication.In some embodiments, electronic equipment 100 uses eSIM, That is: embedded SIM card.ESIM card can cannot separate in electronic equipment 100 with electronic equipment 100.

The software systems of electronic equipment 100 can use layer architecture, event-driven framework, micronucleus framework, micro services frame Structure or cloud framework.The embodiment of the present application by taking the android system of layer architecture as an example, exemplary illustration electronic equipment 100 it is soft Part structure.

Figure 1B is the software architecture diagram of the electronic equipment 100 of the embodiment of the present application.

Software is divided into several layers by layer architecture, and each layer has clearly role and the division of labor.Pass through between layers Software interface communication.In some embodiments, android system is divided into four layers, from top to bottom respectively application layer, answered With process block rack-layer, (Android runtime) and system library and inner nuclear layer when Android is run.

Application layer may include a series of application packages.

As shown in Figure 1B, application package may include camera, picture library, calendar, call, map, navigation, WLAN, bluetooth, Music, video, the application programs such as short message.

Application framework layer provides Application Programming Interface (application for the application program of application layer Programming interface, API) and programming framework.Application framework layer includes some functions predetermined.

As shown in Figure 1B, application framework layer may include window manager, Content Provider, view system, phone Manager, resource manager, notification manager etc..

Window manager is for managing window writing routine.The available display screen size of window manager, judges whether there is shape State column, lock-screen, screen printing etc..

Content Provider is used to store and obtains data, and accesses these data by application program.The data It may include video, image, audio, the phone dialed and answered, browsing history and bookmark, telephone directory etc..

View system includes visible controls, such as the control of display text, shows the control etc. of picture.View system is available In building application program.What display interface can be made of one or more views.E.g., including the display of short massage notice icon Interface may include the view for showing text and the view for showing picture.

Telephone supervisor is for providing the communication function of electronic equipment 100.Such as talking state management (including connect, It hangs up).

Resource manager provides various resources, such as localized strings for application program, icon, picture, topology file, Video file etc..

Notification manager allows application program to show notification information in status bar, can be used for conveying and informs type Message, can be to disappear, without user's interaction automatically after short stay.For example notification manager be used to inform that downloading is completed, and disappear Breath prompting etc..Notification manager, which can also be, appears in the logical of system head status bar with chart or scroll bar textual form Know, for example, running background application program notice, can also be occur notice on the screen in the form of dialog box.Such as Text information is prompted in status bar, issues prompt tone, vibration of electronic equipment, indicator light flashing etc..

Android Runtime includes core library and virtual machine.Android runtime be responsible for Android system scheduling and Management.

Core library includes two parts: a part is the power function that java language needs to call, and another part is Android Core library.

Application layer and application framework layer operate in virtual machine.Virtual machine is by application layer and application program It is binary file that the java file of ccf layer, which executes,.Virtual machine is used to execute the management of Object Life Cycle, stack management, line Thread management, safety and the functions such as abnormal management and garbage reclamation.

System library may include multiple functional modules.Such as: surface manager (surface manager), media library (Media Libraries), three-dimensional graph process library (such as: OpenGL ES), 2D graphics engine (such as: SGL) etc..

Surface manager provides 2D and 3D figure layer for being managed to display subsystem for multiple application programs Fusion.

Media library supports a variety of common audios, video format playback and recording and static image file etc..Media library It can support a variety of audio/video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG etc..

Three-dimensional graph process library is for realizing 3-D graphic drawing, image rendering, synthesis and figure layer process etc..

2D graphics engine is the drawing engine that 2D draws.

Inner nuclear layer is the layer between hardware and software.Inner nuclear layer includes at least display driving, webcam driver, and audio is driven It is dynamic, sensor driving.

It takes pictures scene below with reference to capture, the workflow of 100 software of exemplary illustration electronic equipment and hardware.

When touch sensor 180K receives touch operation, corresponding hardware interrupts are sent to inner nuclear layer.Inner nuclear layer will touch It touches operation and is processed into original input event (including touch coordinate, the information such as timestamp of touch operation).Original input event quilt It is stored in inner nuclear layer.Application framework layer obtains original input event from inner nuclear layer, identifies control corresponding to the incoming event Part.It is to touch single-click operation with the touch operation, for control corresponding to the single-click operation is the control of camera applications icon, Camera applications call the interface of application framework layer, start camera applications, and then by calling inner nuclear layer to start webcam driver, lead to It crosses camera 193 and captures still image or video.

The exemplary patterns of the application program for showing the installation of electronic equipment 100 on electronic equipment 100 are described below User interface.

Fig. 2A illustrates showing for the application program for showing the installation of electronic equipment 100 on electronic equipment 100 Example property user interface 21.

User interface 21 can include: status bar 201, calendar indicator 202, weather indicator 203 have common using journey The pallet 204 of sequence icon, navigation bar 205 and other applications icon.Wherein:

Status bar 201 can include: one or more signal strengths of mobile communication signal (and can be described as cellular signal) refer to Show symbol 201A, operator name (such as " China Mobile ") 201B, Wireless Fidelity (wireless fidelity, Wi-Fi) letter Number one or more signal strength indicator 201C, battery status indicator 201D, time indicator 201E.

Calendar indicator 202 may be used to indicate current time, such as date, what day, time division information etc..

Weather indicator 203 may be used to indicate weather pattern, such as cloudy turn to fine, light rain etc., may be used to indicate that gas The information such as temperature.

It can be shown with common application program image target pallet 204: phone icon 204A, contact icon 204B, short message Icon 204C, camera icon 204D.

Navigation bar 205 can include: the systems navigation key such as return key 205A, main screen key 205B, multitask key 205C.Work as inspection When measuring user click return key 205A, electronic equipment 100 can show a upper page for current page.When detecting user's point When hitting main screen key 205B, electronic equipment 100 can show main interface.When detecting that user clicks multitask key 205C, electronics Equipment 100 can show the task that user opens recently.The name of each navigation key can also be other, and the application does not limit this System.It is not limited to virtual key, each navigation key in navigation bar 205 also can be implemented as physical button.

Other applications icon can be such as: the icon 206 of wechat (Wechat), QQ icon 207, push away spy (Twitter) the shared icon 211 of icon 208, the icon 209 of facebook (Facebook), the icon 210 of mailbox, cloud, memorandum The icon 212 of record, the icon 213 of Alipay, picture library icon 214, setting icon 215.User interface 21 may also include page Face indicator 216.Other applications icon can be distributed in multiple pages, and it is current that page indicator 216 may be used to indicate user What is browsed is the application program in which page.User can horizontally slip other applications figure target area, to browse Application icon in other pages.

In some embodiments, Fig. 2A it is exemplary shown in user interface 21 can be main interface (Home screen).

In some other embodiment, electronic equipment 100 can also include the main screen key of entity.The main screen key is available In the instruction for receiving user, the UI currently shown is returned into main interface, user is can be convenient in this way and checks main screen at any time.On Stating instruction specifically can be the operational order that user's single presses main screen key, be also possible to user in a short time twice in succession The operational order for pressing main screen key can also be the operational order of user's long-pressing main screen key in the given time.In this Shen Please in some other embodiment, main screen key can with integrated fingerprint identifier, for use in when pressing main screen key, Fingerprint collecting and identification are carried out therewith.

It is understood that the merely illustrative user interface shown on electronic equipment 100 of Fig. 2A, should not be constituted to this Apply for the restriction of embodiment.

Fig. 2A and Fig. 2 B illustrates a kind of unlatching on electronic equipment 100 " emotion recognition " operation.

As shown in Figure 2 B, when electronic equipment detects the slide downward gesture on status bar 201, in response to the hand Gesture, electronic equipment 100 can in user interface 21 display window 217.As shown in Figure 2 B, " feelings can be shown in window 217 Perception is other " switch control 217A, the switch control of other function (such as Wi-Fi, bluetooth, flashlight) can also be shown Part.When detecting operation (touch operation such as on switch control 217A) on the switch control 217A in window 217, In response to the operation, electronic equipment 100 can be opened " emotion recognition ".

In other words, user can do the gesture of a slide downward at status bar 201 to open window 217, and can " emotion recognition " is easily opened to click the switch control 217A of " emotion recognition " in window 217." emotion recognition " The form of expression of switch control 217A can be text information or icon.

It is not limited to open " emotion recognition " in window 217, in some embodiments, user can also check image (the cloud of the image in view-finder, the image being stored in electronic equipment or the access of electronic equipment 100 that such as camera applications provide Image etc. on the server of end) when be then turned on " emotion recognition ", be discussed in greater detail in subsequent embodiment, do not repeat first here.? In other embodiments, electronic equipment 100 can also be automatically turned on " emotion recognition ", such as check image recognizing user It is automatically turned on when scene " emotion recognition " etc..In further embodiments, user can open from the setting options of electronic equipment Open " emotion recognition " function.

In some embodiments of the application, after electronic equipment opens " emotion recognition ", it can also be shown in status bar 201 Show the prompt information that " emotion recognition " has turned on.For example, showing the icon or directly aobvious of " emotion recognition " in status bar 201 Show text " emotion recognition " etc..

Describe separately below this application involves application scenarios and electronic equipment 100 on the graphic user interface realized Some embodiments.

(1) application scenarios one: photographed scene

In the UI embodiment that following figure 3 A-3H is illustrated, user can image in preview when taking pictures from view-finder Head acquired image, and the image of preview in the current view-finder of shooting may be selected, that is, save the image.In the embodiment of the present application In, electronic equipment can identify affective state when user views the image, and associated storage feelings when storing the image Sense state.It is intelligible, the image for the camera acquisition that user's preview in view-finder is arrived and the figure of user's final choice shooting Picture, may be identical, it is also possible to different.

" interface of taking pictures " that the UI embodiment that explanatory diagram 3A- Fig. 3 H is illustrated below provides.

In some embodiments, " interface of taking pictures " is displayed for camera acquired image, one or more bats Take the photograph related control when image.Wherein, it is collected to can be front camera for camera acquired image, after being also possible to It is collected to set camera.Related control when shooting image can be used for receiving user's operation (such as touch operation), in response to The user's operation, electronic equipment can execute following one or more: the ratio of change display image opens flash lamp, opens Corresponding screening-mode, switching camera etc..

The user interface 31 that Fig. 3 A- Fig. 3 C is illustrated can be a kind of implementation at " interface of taking pictures ".The user Interface 31 can be what " camera " application provided." camera " is mounted to one on the electronic equipments such as smart phone, tablet computer Money is used to shoot the application program of image, the embodiment of the present application to the title of the application program with no restrictions.The application program can To support to show the collected all kinds of images of camera 193.The object that i.e. " camera " is shown is the collected figure of camera 193 Picture.The user interface 31 can be user and click the user interface that the camera icon 204D in Fig. 2A is opened, without being limited thereto, user The user interface 31 for shooting image can also be opened in other applications, such as user clicks in " wechat " application Control is shot to open the user interface 31 for shooting image.

As shown in figs. 3 a-3 c, user interface 31 can include: shot image echo control 301, shooting control 302, used In the control 303 of switching camera, screening-mode menu 304, view-finder 305 and other some controls for example for adjusting The control 306A of the image size shown in section view-finder 305, recommended information for showing screening-mode control 306B, use The control 306C and control 306D for opening/closing flash lamp of all kinds of parameters when shooting is set.

The image that image echo control 301 can be used for showing that 100 the last time of electronic equipment takes has been shot, can also have been supervised Listen the user's operation for being used to open photograph album.Electronic equipment 100 can detecte user's operation (such as the point for acting on control 301 Hit operation), in response to the operation, the last image saved of electronic equipment display.

Shooting control 302 can monitor the user's operation for shooting image.What electronic equipment can detecte acts on bat The user's operation (such as clicking operation) of control 302 is taken the photograph, and is shot in response to the user's operation, the image of the shooting is saved And the image is shown in control 301.In other words, user can click shooting control 302 to shoot image.

Control 303 for switching camera can monitor the user's operation for switching camera.Electronic equipment 100 can be with It detects the user's operation (such as clicking operation) for acting on control 303, and switches camera in response to the user's operation, such as Rear camera is switched to front camera, or front camera is switched to rear camera.

View-finder 305 can be used for showing the image that camera obtains.That is, view-finder 305 can be used for showing preview Image.Electronic equipment 100 can refresh display content therein in real time.Wherein, it can be for obtaining the camera of the image Rear camera is also possible to front camera.

Screening-mode menu 304 can show one or more screening-mode options.Screening-mode option may be implemented For icon, text or other forms.As shown in Figure 3A, the screening-mode option in screening-mode menu 304 can include: large aperture The option 304A of mode, the option 304B of night scene mode, the option 304C of portrait mode of figure, the option 304D of photographing mode, video recording Option 304E, option 304F of professional mode of mode etc..Electronic equipment, which can detecte, acts on screening-mode menu 304 Slide (such as leftward or rightward slide), and screening-mode menu 304 is shown in for operating switching in response to this In screening-mode option, so that users can browse more screening-mode options.Electronic equipment 100, which can also detect, acts on bat The user's operation of the mode option is taken the photograph, and opens corresponding screening-mode in response to the user's operation.Electronic equipment 100 is different Under screening-mode, the image of different-effect can be shot.Locate for example, can be done to the image taken under large aperture mode Reason makes it that Deep Canvas be presented, and can beautify etc. to the portrait in the image of shooting under portrait mode of figure.

In the case where " emotion recognition " is not opened, as shown in Figure 3A, " emotion knowledge can be shown in user interface 31 Switch control 307 not ".The switch control 307 can be icon.In some embodiments, user interface 31 can also be shown There is the prompt information for prompting user to open " emotion recognition ", which " can for example click here for text information Unlatching emotion recognition ", picture, link or other forms, the present embodiment do not do any restrictions to this.

As shown in Figure 3A, electronic equipment, which can detecte, acts on the operation of control 307 (such as user is on control 307 Touch, the operation such as weight or long-pressing), in response to the operation, which can open " emotion recognition ".In some embodiments In, after electronic equipment opens " emotion recognition ", the display mode of switch control 307 can be updated, updated control 307 can be such as Shown in Fig. 3 B.That is, control 307 can be changed to shadow-free by shadow state after electronic equipment opens " emotion recognition " State.It is not limited to the display mode of change control 307, electronic equipment can also prompt user to have turned on " feelings by other means Perception is other ", such as electronic equipment can also play voice or generation is vibrated to prompt user to have turned on " emotion recognition " etc., this Apply for embodiment with no restriction.

It is not limited to user shown in Fig. 3 A and inputs user's operation on control 307 to open the mode of " emotion recognition ", electricity Sub- equipment can also open by other means " emotion recognition ".It " opens emotion to know for example, user can also input voice Not ", electronic equipment can collect the voice of user by microphone 170C, and open " emotion recognition " according to the voice.For example, User can also open " emotion recognition " etc. in window 217 as shown in Figure 2 B, and the application is without limitation.

It is had turned in " emotion recognition " but in the case where the electronic equipment also unidentified affective state to user, such as Fig. 3 B institute Show, prompt information 308 can be shown in view-finder 305.Wherein, prompt information 308 can be used for prompting consumer electronic devices Identifying the affective state of user, such as prompt information 308 can be text information " identifying user feeling ".It is not limited to Text information, prompt information 308 can also be without limitation for other forms, the application such as picture, links.Some other In embodiment, the prompt information 308 in this user interface 31 may not be the interactive elements of display on the touchscreen, but logical Cross the audio that loudspeaker 170A is played back.

In some embodiments, in the also unidentified affective state to user of electronic equipment, which can also be with Any other content, i.e. blank is not presented in view-finder 305, can indicate the currently also unidentified emotion shape to user with this State.

If electronic equipment has recognized the affective state of user by preset time, then the electronic equipment can be updated and be taken Content shown by scape frame 305 can show prompt information 309 in updated view-finder 305.The preset time can be by electricity The performance of sub- equipment determines, can also be independently arranged by user, the embodiment of the present application is without limitation.Wherein, prompt information The affective state of 309 users that can be used for that consumer electronic devices is prompted to have identified that, such as prompt information 309 can be text This information " recognizes user and is in happy state, click is checked ".It is not limited to text information, prompt information 309 can also be figure The other forms such as piece, link, the application are without limitation.In some embodiments, user shoots video using electronic equipment During, if the affective state of user is updated, the prompt information 309 in user interface 31 as shown in Figure 3 C can also It is corresponding to update.It is understood that being not limited to the interactive elements (prompt information 309) that Fig. 3 C is illustrated, electronic equipment is also It can indicate that electronic equipment has identified that the affective state of user, such as electronic equipment by the interactive elements of other forms Voice can also be played to prompt user to recognize affective state etc..Here, electronic equipment identifies the affective state of user Specific embodiment can refer to associated description above, and details are not described herein.

In some embodiments, after electronic equipment has recognized the affective state of user, which can directly exist Emotion element corresponding with the affective state of user is shown in view-finder 305.

In further embodiments, after electronic equipment has recognized the affective state of user, user interface shown in Fig. 3 C It is opposite with the affective state of user that prompt information 309 some or all of in 31 can be used for receiving triggering electronic equipment presentation The operation for the Emotion element answered.For example, the part in prompt information 309 " recognize user and be in happy state, click and check " Text " click is checked " can be used for receiving the behaviour that Emotion element corresponding with the affective state of user is presented in triggering electronic equipment Make.The operation can be touch, the operation such as weight or long-pressing.Here corresponding Emotion element can with the affective state of user To be stored in electronic equipment local side, server beyond the clouds also can store.That is, electronic equipment recognizes the feelings of user After sense state, Emotion element corresponding with the affective state can be got from local side or from cloud server.

The user interface 31 that Fig. 3 D-3H is illustrated can present for electronic equipment and the current affective state phase of user Several implementations of corresponding Emotion element.

Fig. 3 D shows the mode that a kind of electronic equipment shows an Emotion element corresponding with the affective state of user. As shown in Figure 3D, the corresponding Emotion element of affective state " happy " is shown in user interface 31, which is table Feelings packet 310A.Here, the corresponding Emotion element of affective state " happy " can have one or more, and expression packet 310A can be this One of any one or electronic equipment default in the corresponding one or more Emotion elements of affective state " happy ".

Fig. 3 E shows the mode that a kind of electronic equipment shows multiple Emotion elements corresponding with the affective state of user. As shown in FIGURE 3 E, corresponding multiple Emotion elements, including expression packet are shown with affective state " happy " in user interface 31 The height that image in 310A, text bubble 310B, text bubble 310C and text information 310D and view-finder 305 is presented Saturation degree.Multiple Emotion element can be the part or complete with affective state " happy " in corresponding multiple Emotion elements Portion.

It is intelligible, it is not limited to the Emotion element presented in user interface 31 in Fig. 3 D or Fig. 3 E, electronic equipment may be used also Other Emotion elements are presented.Such as electronic equipment can also adjust the mode of composition of image in view-finder 305, effect of shadow, satisfy With degree, tone and color, background music etc. can also be played by loudspeaker 170A, to reflect the current emotion shape of user State.

Fig. 3 F- Fig. 3 H shows the side that Emotion element corresponding with the affective state of user is presented in another electronic equipment Formula.As illustrated in Figure 3 F, wicket 311 is shown in user interface 31.Wicket 311 can be used for user and select corresponding emotion member Element is presented.It may include icon 311A, text information " Emotion element " 311B, multiple feelings of Emotion element in wicket 311 Sense element option such as " original image " 311C, high saturation 311D, text bubble " live up！" 311E, expression packet 311F, text Information " daytime, which sings, to indulge in drinking, and the youth, which keeps company, returns to one's native " 311G, background music " song ' today be an auspicious day ' " 311H etc.. Here, Emotion element corresponding to multiple Emotion element option can be the corresponding one or more feelings of affective state " happy " Some or all of in sense element.It is not limited to the Emotion element option in wicket 311 shown in Fig. 3 F included, in other embodiments In, it may also include other Emotion element options in the wicket 311, such as the option, full of the option of mode of composition, effect of shadow With the option of degree, the option of tone and the option of color etc..

Each Emotion element option can be used for monitoring triggering electronic equipment presentation feelings corresponding with the option in wicket 311 Feel the operation of element.In response to operation (such as the touching on Emotion element option for acting on Emotion element option detected Touch operation), Emotion element corresponding with the Emotion element option can be presented in electronic equipment.For example, in response to the work detected For the touch operation on option 311C, any Emotion element can not be presented in electronic equipment；It is acted in response to what is detected Touch operation on option 311E, electronic equipment can show that text bubble " lives up in user interface 31！", such as Fig. 3 G It is shown；In response to the touch operation acted on option 311H detected, electronic equipment enables loudspeaker 170A and plays song " today is an auspicious day ".In some embodiments, Emotion element option, should in wicket 311 after receiving user's operation The display mode of Emotion element option can be modified, for example, option 311E shown in Fig. 3 G frame by overstriking.

In a kind of possible embodiment, it is only capable of that an Emotion element is presented in user interface 31 as shown in Figure 3 G, When electronic equipment, which detects, acts on the operation of next Emotion element option in wicket 311, electronic equipment stops presenting A upper Emotion element simultaneously starts that next Emotion element is presented.In alternatively possible embodiment, as shown in Figure 3 G User interface 31 in multiple Emotion elements can be presented, act on the operation of Emotion element option when electronic equipment detects When (such as touch operation), corresponding with Emotion element option Emotion element is presented in electronic equipment, when electronic equipment again When detecting operation (such as touch operation) for acting on the Emotion element option, electronic equipment can stop presentation and the Emotion element The corresponding Emotion element of option.

In some embodiments, control 311I and/or control 311J can also be shown in wicket 311.Control 311I Or control 311J can be used for user's switching or update the Emotion element option shown in wicket 311, can check so more Emotion element corresponding with the affective state of active user.It is not limited to above-mentioned control, other interactive elements can be used for using The Emotion element option shown in family switching wicket 311.In other cases, user can also be in wicket 311 Leftward or rightward slip gesture is made to switch or update the Emotion element option shown in wicket 311.

As shown in Fig. 3 F and Fig. 3 G, wicket 311 may also include control 311K and control 311L.Control 311L can be used for supervising Triggering is listened to stop the operation of display wicket 311.Illustratively, as shown in Fig. 3 G and Fig. 3 H, user selectes Emotion element option After 311E, user's operation (touch operation such as in text " determination ") can be inputted on control 311L.It is grasped in response to the user Make, electronic equipment stops display wicket 311 and continues Emotion element that presentation user selectes (as shown in continuously display Fig. 3 H Text bubble " live up！").Control 311K, which can be used for monitoring triggering, to be stopped display wicket 311 and stops presenting any The operation of Emotion element.(the touch behaviour such as in text " cancellation " of the operation on control 311K is acted in response to what is detected Make), electronic equipment stops display wicket 311 and stops that any Emotion element is presented.

In some embodiments, after electronic equipment has recognized the affective state of user, which can be associated with and deposit The image and the current affective state of the user that storage electronic equipment is got.Specifically, electronic equipment can obtain camera The current affective state associated storage of the image and user arrived can also work as image that camera is got and user to local For preceding affective state associated storage to cloud server, the application is without limitation.Illustratively, as shown in Figure 3 C, electronics After equipment has recognized the affective state of user, user can shooting control 302 on input user's operation (such as touch behaviour Make), in response to the user's operation detected, electronic equipment can be with the people in associated storage view-finder 305 as shown in Figure 3 C Picture and the current affective state of the user recognized " happy ".It is not limited to the user's operation inputted on shooting control 302, electronics Equipment may also respond to other users and operate the image that associated storage the is got affective state current with the user recognized, Such as user can also trigger electronic device association storage by input voice " taking pictures ", in face of modes such as camera blinks and obtain To the current affective state of image and the user that recognizes.In the embodiment of the present application, electronic equipment can with JPEG, TIFF, RAW, PNG, GIF, AVI, 3GP, MOV or other picture formats store image, with text (such as text " happy "), refer to Indicating knows modes such as (such as signs " 1 ", indicate that corresponding affective state is " happy ") and stores the current emotion shape of user State, the application are without limitation.

In further embodiments, it after electronic equipment has recognized the affective state of user, can be set with associated storage electronics The standby image and Emotion element corresponding with the affective state of the user recognized got.Electronic device association storage obtains To image and Emotion element corresponding with the affective state of the user recognized when, which can be one or more It is a.Specifically, electronic equipment can be by image and Emotion element corresponding with the affective state of user that camera is got Associated storage can also close image that camera is got and Emotion element corresponding with the affective state of user to local Cloud server is arrived in connection storage, and the application is without limitation.

Illustratively, as shown in Figure 3D, electronic equipment has recognized the affective state of user and in user interface 31 When showing Emotion element expression packet 310A corresponding with affective state " happy ", user can be on shooting control 302 It inputs user's operation (such as touch operation), in response to the user's operation detected, electronic equipment can be with associated storage as schemed Portrait and Emotion element expression packet 310A in view-finder 305 shown in 3D.

Again illustratively, as shown in figure 3h, electronic equipment has recognized the affective state of user and in user interface 31 On show Emotion element text bubble corresponding with affective state " happy " and " live up！" 310B when, user can be with User's operation (such as touch operation) is inputted on shooting control 302, in response to the user's operation detected, electronic equipment can With in the view-finder 305 of associated storage as shown in figure 3h portrait and Emotion element text bubble " live up！"310B.

Again illustratively, as shown in FIGURE 3 E, electronic equipment has recognized the affective state of user, and in user interface 31 On show multiple Emotion elements corresponding with affective state " happy " such as expression packet 310A, text bubble 310B, text bubble 310C and text information 310D has also been played Emotion element song " today is an auspicious day ", and has been shown and found a view with high saturation When image in frame 305, user can input user's operation (such as touch operation) on shooting control 302, in response to detection The user's operation arrived, electronic equipment can be with the portraits and Emotion element table in the view-finder 305 of associated storage as shown in FIGURE 3 E Feelings packet 310A, text bubble 310B, text bubble 310C, text information 310D, song " today is an auspicious day " and height are full With the instruction information of degree.

In above-mentioned Fig. 3 D, Fig. 3 E and Fig. 3 H example, it is not limited to the user's operation inputted on shooting control 302, electronics Equipment may also respond to the image and Emotion element that other users operation associated storage is got.For example, user can also lead to It crosses input voice " taking pictures ", trigger the image and emotion member that electronic device association storage is got in face of modes such as camera blinks Element, the embodiment of the present application are without limitation.

In above-mentioned Fig. 3 C, Fig. 3 D, Fig. 3 E and Fig. 3 H example, it is not limited to carry out associated storage in response to user's operation to get Image and affective state/Emotion element, electronic equipment can also deposit auto-associating after the affective state for recognizing user It stores up the image got and affective state/Emotion element, the embodiment of the present application is without limitation.

After electronic equipment has recognized the affective state of user, image that associated storage electronic equipment is got and with identification The corresponding Emotion element of the affective state of the user arrived, alternatively, the image that gets of associated storage electronic equipment and recognizing User affective state when, the electronic equipment storage the image got be shooting image.Due to user's preview image and Shooting image may not be to carry out simultaneously, therefore, the preview image and final shooting figure that user views in view-finder As (image saved) may be identical, it is also possible to different.

It is intelligible, preview image that UI embodiment shown in figure 3 above C- Fig. 3 H is viewed with user in view-finder and It is illustrated for final shooting image is identical, i.e., user is in view-finder in the UI embodiment shown in figure 3 above C- Fig. 3 H In the preview image that views and final shooting image be all portrait in view-finder 305.In some other implementation of the application In example, the preview image and final shooting image that user views in view-finder can not also be identical.If user is in view-finder The preview image and final shooting image viewed be not identical, then user checks preview image and final shooting figure in view-finder Duration as between is no more than preset duration, which can preset, such as can be 1 minute, 2 minutes etc..It should Preset duration can guarantee and shoot that the affective state of image associated storage is affective state when user sees the shooting image, Alternatively, the preset duration can guarantee and shoot that the Emotion element of image associated storage is able to reflect user and views the shooting figure As when affective state.

In some embodiments, if electronic equipment obtains video by camera, and user obtains video in electronic equipment During affective state change, i.e. electronic equipment multiple affective states that user is recognized during obtaining video When, electronic equipment can be with each segment of associated storage video and the corresponding affective state of each segment or Emotion element. For example, electronic equipment, which gets a length of 20 minutes videos, electronic equipment at one section, recognizes the emotion of 1-10 minutes users State is " romance ", and the emotion state of 11-20 minutes users is " happy ", then electronic equipment can be by the preceding 1-10 of the video Minute and emotion state " romance " associated storage, by rear 11-20 minutes of the video and emotion state " happy " associated storage.

In some embodiments, electronic equipment can be according to the image storage that different affective states will acquire to difference Region.For example, all affective states in user can be the image got when " happy " and the feelings by electronic equipment All affective states in user can be the figure got when " romance " to first area by sense state " happy " associated storage Picture and affective state " romance " associated storage are to second area, when can be " calmness " by all affective states in user The image got and affective state " calmness " associated storage are to third region.In another example electronic equipment can will own The corresponding Emotion element association of the image and the affective state " happy " got when the affective state of user is " happy " The fourth region is stored, can be the image and the affective state got when " romance " by all affective states in user " romance " corresponding Emotion element associated storage obtains when can be " calmness " by all affective states in user to the 5th region The image got and the corresponding Emotion element associated storage of affective state " calmness " are to the 6th region.Above-mentioned first area is extremely 6th region can be located at electronic equipment local side, can also be located at cloud server.

It is not limited to shown in Fig. 3 A- Fig. 3 H, other forms can also be presented in the page layout at " interface of taking pictures ", and the application is implemented Example is without limitation.

The UI embodiment illustrated by Fig. 3 A- Fig. 3 H, electronic equipment can identify that user works as when shooting image Preceding affective state, and the image for needing associated storage to take according to user and the current affective state of the user recognized, Alternatively, the image and Emotion element corresponding with the current affective state of user that need associated storage to take according to user. The emotion that can recorde user in this way can achieve concern user feeling, convenient for the purpose of reappearing user emotion when needed, mention User has been risen to the usage experience of electronic equipment.

This application involves photographed scene in, " interface of taking pictures " is properly termed as the first user interface, and preview image can be with Referred to as the first image, shooting image are properly termed as the second image.First image and the second image may be the same or different. The example implementations for being somebody's turn to do " interface of taking pictures " can refer to user interface 31 shown in Fig. 3 A- Fig. 3 H.

This application involves photographed scene in, can electronic equipment recognizes when checking preview image by user use The affective state at family is known as the first affective state.When electronic device association store the second image and with the first emotion shape for recognizing When the corresponding Emotion element of state, should Emotion element corresponding with the first affective state recognized can be referred to as the first feelings When feeling element.

This application involves photographed scene in, can by include in the first user interface be used to monitor save the second figure The control of picture is known as the shooting control 302 in the first interactive elements, such as Fig. 3 C- Fig. 3 E, Fig. 3 H；Can will act on this first The operation of interactive elements is known as the first operation, such as acts on the touch operation on shooting control 302.

(2) application scenarios two: picture scene is checked

In the UI embodiment that following figure 4 A- Fig. 4 H is illustrated, when checking picture, electronic equipment can identify user The current affective state of user, and associated storage affective state when storing the picture.

" checking picture interface " that the UI embodiment that explanatory diagram 4A- Fig. 4 H is illustrated below provides.

In some embodiments, it " checks picture interface " and is displayed for picture, is one or more for handling the figure The related control of piece.Wherein, which can be the picture stored in the cloud device that electronic equipment accesses, electronic equipment sheet The picture etc. that ground terminal is stored.The exemplary embodiment of the picture stored in the cloud device of electronic equipment access may include using Family " wechat (WeChat) " application in check circle of friends picture, user " facebook (Facebook) " application in check good friend The picture etc. of publication.The exemplary embodiment for the picture that electronic equipment local side is stored may include that user looks into " photograph album " application The picture etc. seen.Related control for handling the picture can be used for monitoring users operation (such as touch operation), in response to this User's operation, electronic equipment can execute following one or more: share picture, save picture, collection picture etc..

The user interface 41 that Fig. 4 A- Fig. 4 D is illustrated can be a kind of implementation of " checking picture interface ".It should User interface 41 can be " wechat (WeChat) " application, " facebook (Facebook) " is applied, " photograph album " is applied or other are answered With offer.The user interface 41 can be user and click wechat icon 206, facebook icon 209 or picture library figure in Fig. 2A The user interface opened when checking certain picture after mark 214.

As illustrated in figs. 4 a-4d, user interface 41 can include: status bar 401 and picture 402.Status bar 401 can refer to figure Status bar 201 in user interface 21 shown in 2A, which is not described herein again.Picture 402 is the picture that user currently checks, the figure Piece can be the picture stored in the cloud device that electronic equipment accesses, be also possible to the figure that electronic equipment local side is stored Piece, the application are without limitation.In some embodiments, user interface 41 may also include suppressible navigation bar (in figure not Show), which can refer to the navigation bar 205 in user interface 21 shown in Fig. 2A, and which is not described herein again.

In the case where " emotion recognition " is not opened, user can open " emotion recognition ".A and Fig. 4 B referring to fig. 4 shows A kind of possible mode for opening " emotion recognition " is gone out.

As shown in Figure 4 A, electronic equipment, which can detecte, acts on the operation of picture 402 (such as user is on picture 402 The operation such as long-pressing, weight or double-click), in response to the operation, which can show menu 403 as shown in Figure 4 B.Dish It may include multiple in single 403 for triggering the control that electronic equipment handles the picture, such as sharing control 403A, preservation control 403B, collection control 403C, emotion recognition control 403D.Control in menu 403 can be used for monitoring users operation (as touched behaviour Make), in response to the user's operation detected, electronic equipment can do corresponding processing to the picture, such as share, save, receive Hiding or emotion recognition etc..In some embodiments, it may also include more control 403E in menu 403.More control 403E can be used (such as touch operation) is operated in monitoring users, in response to the user's operation detected, electronic equipment can show more use The control of the picture is handled in triggering electronic equipment, such as deletes control, mobile control.Here, the table of the control in menu 403 Existing form can be icon and/or text information.

Emotion recognition control 403D in menu 403 can be used for monitoring the operation that " emotion recognition " is opened in triggering.In response to The operation (touch operation such as on emotion recognition icon) for acting on emotion recognition control 403D detected, electronic equipment is opened Open " emotion recognition ".

It is not limited to user shown in Fig. 4 B and inputs user's operation on emotion recognition control 403D to open " emotion recognition " Mode, electronic equipment can also open by other means " emotion recognition ".For example, user can also be as shown in Figure 2 B Window 217 in open " emotion recognition " etc., the application is without limitation.

" emotion recognition " have turned on but electronic equipment it is also unidentified to user feeling in the case where, as shown in Figure 4 C, figure Prompt information 404 can be shown on piece 402.Wherein, prompt information 404 can be used for that consumer electronic devices is prompted to identify The affective state of user.Prompt information 404 can be with reference to prompt information 308 shown in Fig. 3 B, and details are not described herein.

In some embodiments, in the also unidentified affective state to user of electronic equipment, which can also be with Any other content, i.e. blank are not presented on picture 402, can indicate the currently also unidentified emotion shape to user with this State.

If the electronic equipment can update figure after a period of time, electronic equipment has recognized the affective state of user Shown content on piece 402 can show prompt information 405 in updated picture 402.Wherein, prompt information 405 can be with Affective state for the user for prompting consumer electronic devices to have identified that.Prompt information 405 can be with reference to shown in Fig. 3 C Prompt information 309, details are not described herein.Here, the specific embodiment of the affective state of electronic equipment identification user can refer to Associated description above, details are not described herein.

In some embodiments, after electronic equipment has recognized the affective state of user, which can directly exist Emotion element corresponding with the affective state of user is shown on picture 402.

In further embodiments, after electronic equipment has recognized the affective state of user, user interface shown in Fig. 4 D It is opposite with the affective state of user that prompt information 405 some or all of in 41 can be used for receiving triggering electronic equipment presentation The operation for the Emotion element answered.For example, the part in prompt information 405 " recognize user and be in happy state, click and check " Text " click is checked " can be used for receiving the behaviour that Emotion element corresponding with the affective state of user is presented in triggering electronic equipment Make.The operation can be touch, the operation such as weight or long-pressing.Here corresponding Emotion element can with the affective state of user To be stored in electronic equipment local side, server beyond the clouds also can store.That is, electronic equipment recognizes the feelings of user After sense state, Emotion element corresponding with the affective state can be got from local side or from cloud server.

The user interface 41 that Fig. 4 E- Fig. 4 H is illustrated can present opposite with the affective state of user for electronic equipment Several implementations for the Emotion element answered.

Fig. 4 E shows the mode that a kind of electronic equipment shows an Emotion element corresponding with the affective state of user. As shown in Figure 4 E, the corresponding Emotion element of affective state " happy " is shown in user interface 41, which is table Feelings packet 406A.Here, the corresponding Emotion element of affective state " happy " can have one or more, and expression packet 406A can be this One of any one or electronic equipment default in the corresponding one or more Emotion elements of affective state " happy ".

Fig. 4 F shows the mode that a kind of electronic equipment shows multiple Emotion elements corresponding with the affective state of user. As illustrated in figure 4f, the corresponding multiple Emotion elements of affective state " happy " are shown in user interface 41, including expression packet 406A, The high saturation that text bubble 406B, text bubble 406C and text information 406D and picture 402 are presented.Multiple feelings Sense element can be some or all of in the corresponding multiple Emotion elements of affective state " happy ".

It is intelligible, it is not limited to the Emotion element shown in user interface 41 in Fig. 4 E and Fig. 4 F, electronic equipment may be used also Other Emotion elements are presented.Such as electronic equipment can also adjust the mode of composition of image in view-finder 305, effect of shadow, satisfy With degree, tone and color, background music etc. can also be played by loudspeaker 170A, to reflect the current emotion shape of user State.

In some embodiments, user interface 41 shown in Fig. 4 E and Fig. 4 F may also include control 407.Control 407 can be used In the operation for monitoring any Emotion element of triggering electronic equipment stopping presentation.In response to the user's operation, electronic equipment stopping is in What incumbent Emotion element.

Fig. 4 G-4H shows the side that Emotion element corresponding with the affective state of user is presented in another electronic equipment Formula.As shown in Figure 4 G, wicket 409 is shown in user interface 41.It may include the icon of Emotion element in wicket 409 409A, text information " Emotion element " 409B, multiple Emotion element options such as " original image " 409C, high saturation 409D, text Bubble " lives up！" 409E, expression packet 409F, text information " daytime, which sings, to indulge in drinking, and the youth, which keeps company, returns to one's native " 409G, back Scape music " song ' today is an auspicious day ' " 409H etc..Each Emotion element option in wicket 409 can be used for monitoring touching Send out the operation that Emotion element corresponding with the option is presented in electronic equipment.Each Emotion element option in wicket 409 can join Each Emotion element option in the wicket 311 in user interface 31 shown in Fig. 3 F is examined, details are not described herein.

In a kind of possible embodiment, it is only capable of that an Emotion element is presented in user interface 41 as shown at figure 4h, When electronic equipment, which detects, acts on the operation of next Emotion element option in wicket 409, electronic equipment stops presenting A upper Emotion element simultaneously starts that next Emotion element is presented.In alternatively possible embodiment, as shown at figure 4h User interface 41 in multiple Emotion elements can be presented, act on the operation of Emotion element option when electronic equipment detects When (such as touch operation), corresponding with Emotion element option Emotion element is presented in electronic equipment, when electronic equipment again When detecting operation (such as touch operation) for acting on the Emotion element option, electronic equipment can stop presentation and the Emotion element The corresponding Emotion element of option.

In some embodiments, control 409I and/or control 409J can also be shown in wicket 409.Control 409I Or control 409J can be used for user's switching or update the Emotion element option shown in wicket 409, can check so more Emotion element corresponding with the affective state of active user.It is not limited to above-mentioned control, other interactive elements can be used for using The Emotion element option shown in family switching wicket 409.In other cases, user can also be in wicket 409 Leftward or rightward slip gesture is made to switch or update the Emotion element option shown in wicket 409.

As shown in Fig. 4 G and Fig. 4 H, wicket 409 may also include control 409K.Control 409K, which can be used for monitoring triggering, to be stopped It shows wicket 409 and stops presenting the operation of any Emotion element.In response to the behaviour acted on control 409K detected Make (touch operation such as in text " cancellation "), electronic equipment stops display wicket 409 and stops that any emotion member is presented Element.

In some embodiments, after electronic equipment has recognized the affective state of user, which can be associated with and deposit The picture and the current affective state of the user that storage user checks.In the embodiment of the present application, electronic equipment can look into user The picture and the current affective state associated storage of the user seen can also work as the picture that user checks with the user to locally For preceding affective state associated storage to cloud server, the application is without limitation.Illustratively, D, electronics are set referring to fig. 4 After the standby affective state for having recognized user, user can input the user's operation (length such as on picture 402 on picture 402 Press, the operation such as weight or double-click), in response to the user's operation detected, electronic equipment can be in the picture of user interface 41 Wicket 403 as shown in Figure 4 B is shown on 402.User can input user on the preservation control 403B in wicket 403 It operates (such as touch operation), in response to the user's operation detected, figure that electronic equipment can be as shown in Figure 4 D with associated storage Piece 402 and the current affective state of the user recognized " happy ".It is not limited to defeated on the preservation control 403B in wicket 403 The user's operation entered, electronic equipment may also respond to the other users picture checked of operation associated storage user and recognize The current affective state of user, such as user can also trigger electronic device association storage by modes such as inputs voice " preservation " The picture that user the checks affective state current with the user recognized.Here, electronic equipment storage picture and user are current The mode of affective state can refer to the associated description in application scenarios one, details are not described herein.

In further embodiments, it after electronic equipment has recognized the affective state of user, can be looked into associated storage user The picture and Emotion element corresponding with the affective state of user seen.The electronic device association picture checked of storage user and with When the corresponding Emotion element of the affective state of user, which can be one or more.In the embodiment of the present application, Electronic equipment can store picture that user checks and Emotion element corresponding with the affective state of user to local, can also With the picture of checking user and Emotion element associated storage corresponding with the affective state of user to cloud server, this Shen It please be without limitation.

Illustratively, as shown in Figure 4 E, electronic equipment has recognized the affective state of user and in user interface 41 Show Emotion element expression packet 406A corresponding with affective state " happy ".User interface 41 shown in Fig. 4 E can also wrap Control 408 is included, which can be used for monitoring the picture that triggering electronic device association storage user checks and the emotion with user The operation of the corresponding Emotion element of state.It (is grasped as touched in response to the user's operation acted on control 408 detected Make), electronic equipment can be with associated storage picture 402 as shown in Figure 4 E and Emotion element expression packet 406A.Here, control 408 The form of expression can be for text or icon etc., the application is without limitation.

Again illustratively, as shown at figure 4h, electronic equipment has recognized the affective state of user and in user interface 41 On show Emotion element text bubble corresponding with affective state " happy " and " live up！"406B.Shown in Fig. 4 H It may also include control 409L in wicket 407 in user interface 41, control 409L can be used for monitoring triggering electronic equipment and close The operation of picture and Emotion element corresponding with the affective state of user that connection storage user checks.In response to the work detected User's operation (such as touch operation) on control 409L, electronic equipment can be with the picture 402 of associated storage as shown at figure 4h With Emotion element expression packet 406A.Here, the form of expression of control 409L can be text or icon etc., and the application does not do this Limitation.

Again illustratively, as illustrated in figure 4f, electronic equipment has recognized the affective state of user, and in user interface 41 On show multiple Emotion elements corresponding with affective state " happy " such as expression packet 406A, text bubble 406B, text bubble 406C and text information 406D has also played Emotion element song " today is an auspicious day ", and has shown picture with high saturation 402.User interface 41 shown in Fig. 4 F may also include control 408, which can be used for monitoring triggering electronic device association and deposit The operation of picture and Emotion element corresponding with the affective state of user that storage user checks.It is acted in response to what is detected User's operation (such as touch operation) on control 408, electronic equipment can be with the picture 402 of associated storage as illustrated in figure 4f and emotions Element expression packet 406A, text bubble 406B, text bubble 406C, text information 406D, a song " today be an auspicious day " with And the instruction information of high saturation.Here, the form of expression of control 408 can be text or icon etc., and the application does not do this Limitation.

In above-mentioned Fig. 4 E, Fig. 4 F and Fig. 4 H example, it is not limited to input on saving control 408 or preservation control 409L User's operation, electronic equipment may also respond to other users operation the associated storage image and Emotion element that get.Example Such as, user can also trigger the image and emotion member that electronic device association storage is got by modes such as inputs voice " preservation " Element, the embodiment of the present application are without limitation.

In above-mentioned Fig. 4 D, Fig. 4 E, Fig. 4 F and Fig. 4 H example, it is not limited to carry out associated storage in response to user's operation to get Image and affective state/Emotion element, electronic equipment can also deposit auto-associating after the affective state for recognizing user It stores up the image got and affective state/Emotion element, the embodiment of the present application is without limitation.

It is not limited to shown in Fig. 4 A- Fig. 4 H, other forms, this implementation can also be presented in the page layout of " checking picture interface " Example is without limitation.

The UI embodiment illustrated by Fig. 4 A- Fig. 4 H, electronic equipment can identify use when user checks picture The current affective state in family, and the picture for needing associated storage user to check according to user and the current emotion of the user recognized State, alternatively, the picture for needing associated storage user to check according to user and feelings corresponding with the current affective state of user Feel element.The emotion that can recorde user in this way can achieve concern user feeling, convenient for reappearing user emotion when needed Purpose improves user to the usage experience of electronic equipment.

It is not limited to the scene for checking picture that Fig. 4 A- Fig. 4 H is illustrated, electronic equipment can also check view in user The affective state of user, and the image that associated storage user can be needed to check according to user are identified when frequency or other kinds of image The current affective state with the user that recognizes, alternatively, the image that associated storage user can be needed to check according to user and identification The corresponding Emotion element of the current affective state of the user arrived.

This application involves check in picture scene, " checking picture interface " is properly termed as the first user interface, checks Picture be properly termed as the first image, the picture of preservation is properly termed as the second image, and the first image and the second image are identical.It should The example implementations of " checking picture interface " can refer to user interface 41 shown in Fig. 4 A- Fig. 4 H.First image and Two images for example can be the picture 402 in user interface shown in Fig. 4 A- Fig. 4 H.

This application involves check in picture scene, can electronic equipment recognizes when checking picture by user use The affective state at family is known as the first affective state.When electronic device association store the second image and with the first emotion shape for recognizing It, should one or more emotions corresponding with the first affective state recognized when the corresponding one or more Emotion elements of state Element can be referred to as the first Emotion element.

This application involves check in picture scene, can be used to monitor preservation figure for include in the first user interface The control of piece is known as the control 408 in the first interactive elements, such as Fig. 4 E- Fig. 4 F, the control 409L in Fig. 4 H；It can will act on It is known as the first operation in the operation of first interactive elements, such as acts on the touch operation on control 408 or 409L.

In this application, the operation for being used to open " emotion recognition " can be known as to the 4th operation.In response to the 4th behaviour Make, i.e., after unlatching " emotion recognition ", electronic equipment can be connected to the electricity by the hardware device that configures or by starting The behavioral data of the hardware device acquisition user of the other equipment of sub- equipment, identifies that user is current according to the behavioral data of user The affective state being in.The specific implementation of the affective state of electronic equipment identification user can refer to associated description above.

It in some embodiments, may include the second interactive elements in the first user interface, second interactive elements are for supervising Listen the operation for opening the emotion recognition service.4th operation can be the second interactive elements acted in the first user interface On operation (such as touch operation).The example implementations of second interactive elements may include control 307 in Fig. 3 A or Control 403D in Fig. 4 B.

(3) application scenarios three: electronic equipment reproduces the scene of the affective state of user

After the electronic device association storage image affective state current with the user recognized, alternatively, electronic equipment closes After connection storage image and Emotion element corresponding with the current affective state of the user recognized, electronic equipment can be in user Corresponding Emotion element is presented when checking the image again.The Emotion element is opposite with the affective state of the image associated storage The Emotion element answered, alternatively, the Emotion element is and the Emotion element of the image associated storage.User is seeing these emotions member After element, corresponding affective state can be experienced, affectional sympathetic response is generated, that is, reproduces affective state when obtaining the image. Wherein, which can be static images, dynamic picture and video etc..The image and feelings that electronic device association storage is got The specific implementation of sense state/Emotion element can refer to the associated description of application scenarios one and application scenarios two above.

In the embodiment that following figure 5 A- Fig. 5 K is illustrated, electronic device association stores image and affective state/emotion After element, it can need to present corresponding Emotion element according to user when user checks the image again.

The relevant user interface that explanation selects image to be checked for user first below.

Fig. 5 A- Fig. 5 C illustrates the relevant user interface for selecting image to be checked for user.

The user interface 51 that Fig. 5 A is illustrated can be used for showing multiple photograph albums.The user interface 51 can be " figure Library " application provides." picture library " is mounted to a for showing image on the electronic equipments such as smart phone, tablet computer Application program, the embodiment of the present application to the title of the application program with no restrictions.The application program can be supported to show and is stored in Image on electronic equipment local side or cloud server.The user interface 51 can be user and click picture library figure in Fig. 2A The user interface that mark 214 is opened.

As shown in Figure 5A, user interface 51 can include: status bar 501, application tile column, one or more photograph album items Mesh, menu 504 and navigation bar 505.

Status bar 501 can refer to the status bar 201 in user interface 21 shown in Fig. 2A, and which is not described herein again.

Application tile column may include current page indicator 502.Current page indicator 502 may be used to indicate currently The page, such as text information " photograph album ".It is not limited to text information, current page indicator 502 can also be icon.

Photograph album entry can include: all photo album entries, all video photograph album entries, camera photograph album entry, screenshotss mesh Picture recording volume entry, wechat photograph album entry, microblogging photograph album entry, and with the affective state point of user when electronic equipment acquisition image For example happy photograph album entry 503A of one or more photograph album entries of class, romantic photograph album entry 503B, angry photograph album entry 503C With sad photograph album entry 503D.Wherein, the image under happy photograph album entry 503A be user affective state be " happy " when obtain It gets, the image under romantic photograph album entry 503A is got when being affective state " romance " of user, angry photograph album item Image under mesh 503A is got when being affective state " anger " of user, and the image under sad photograph album entry 503A is to use It is got when affective state " sorrow " at family.Each photograph album entry can be used for monitoring triggering electronic equipment and show corresponding photograph album Under image operation (such as touch operation), in response to the operation, electronic equipment is openable for showing figure under corresponding photograph album The user interface of picture.

User can input the gesture slided upward or downward in photograph album entry, check more photograph album entries.It is each Photograph album entry can be used for monitoring the user's operation (such as touch operation) for the image that triggering is shown under corresponding photograph album.It is not limited to Fig. 5 A Shown in photograph album entry, electronic equipment can also show more or fewer photograph album entries in user interface 51, and the application is real It is without limitation to apply example.Here, the title of each photograph album entry can be what user was independently arranged in user interface 51, can also To be electronic equipment default setting, the embodiment of the present application is without limitation.

Menu 504 includes one or more related controls for processing to photograph album, such as control 504A, control 504B With control 504C.Control 504A can be used for monitoring the user's operation (such as touch operation) of triggering search photograph album, and control 504B can For monitoring the user's operation (such as touch operation) for triggering newly-built photograph album, it is more that control 504C can be used for monitoring triggering display User's operation (such as touch operation) for being processed to photograph album.Here, the table of the related control for being processed to photograph album Existing form can be icon or text information.

Navigation bar 505 can refer to the navigation bar 205 in user interface 21 shown in Fig. 2A, and which is not described herein again.

The user interface 52 that Fig. 5 B is illustrated can be used for showing the image under happy photograph album.The user interface 52 can be with It is the user interface that user clicks that the happy photograph album entry 503A in Fig. 5 A is opened.As shown in Figure 5 B, which can wrap It includes: status bar 506, application tile column, image-region 510.

Status bar 506 can refer to the status bar 201 in user interface 21 shown in Fig. 2A, and which is not described herein again.

Application tile column can include: return key 509, current page indicator 507 and 508.Return key 509 is APP Grade return key, can be used for returning menu upper level.The upper level page of user interface 52 is user interface 51 as shown in Figure 5A. It will be understood by those skilled in the art that the logic upper level of a page be it is fixed, just had determined that in application programming. Current page indicator 507 and 508 may be used to indicate current page, such as text information " photograph album " " happy " may be used to indicate and work as The preceding page is not limited to text information, current page indicator 507 and 508 can also be for showing the image under happy photograph album Icon.

One or more images can be shown in image-region 510, which may include picture or video, such as scheme Piece 510A, 510B, 510D-510H, 510J-510L and video 510C, 510I.The image shown in picture region 510 can be Thumbnail.At this point, the corresponding original image of thumbnail in image-region 510 can store on electronic equipment, it is stored in On cloud server.

In some embodiments, can be with display reminding information on the image in image-region 510, the prompt information is available There are the affective state or Emotion element of user in prompt user's image associated storage.Illustratively, which can be with For the icon 510M in Fig. 5 B.

In some embodiments, it may also include control 510N in image-region 510, control 510N can be used for user's switching figure Image as shown in region 510 can check the image under more current photograph albums in this way.In other cases, user Slip gesture upward or downward can also be made in image-region 510 to browse more images.

Each image in image-region 510 can be used for monitoring triggering and show the operation of corresponding image (as touched behaviour Make).Illustratively, image 510E can receive the user's operation (such as touch operation) of input, grasp in response to the user detected Make, electronic equipment can show user interface 53 as shown in Figure 5 C.As shown in Figure 5 C, user interface 53 can include: status bar 511, application tile column, image-region 514 and one or more controls for being used to handle image.

Status bar 511 can refer to the status bar 201 in user interface 21 shown in Fig. 2A, and which is not described herein again.

Application tile column can include: return key 512, image information indicator 513.Return key 512 can be used for returning Menu upper level.The upper level page of user interface 53 is user interface 52 as shown in Figure 5 B.Image information indicator 513 can The place being used to indicate when electronic equipment obtains the image and time, such as image information indicator 513 can be text information " the 9 days 2 months 9:00 in Shenzhen-Longgang District ".

Image-region 514 can be used for showing the image of user's selection.The image can be to be stored in electronic equipment local side Or the original image on cloud server.

Control for handling image can be used for monitoring triggering electronic equipment and execute phase to the image that image-region 514 is shown The operation (such as touch operation) for the processing answered.Control for handling image for example may include control 515A, control 515B, control 515C and control 515D.

Control 515A can be used for monitoring triggering electronic equipment and set images share to the other users that image-region 514 is shown The operation (such as touch operation) of standby (such as smart phone) or cloud server.In response to the user's operation detected, electronics Equipment can be by the images share to other users equipment or cloud server.In some other embodiment, in response to detection The user's operation arrived, electronic equipment can also by the image and with affective state/Emotion element of the image associated storage Share to other users equipment or cloud server.

Control 515B can be used for monitoring the operation for the image that triggering electronic equipment collection image region 514 is shown (as touched Operation).In response to the user's operation detected, electronic equipment can collect the image.In some other embodiment, response In the user's operation detected, electronic equipment can also collect the image and with the affective state of the image associated storage/ Emotion element.

Control 515C can be used for monitoring triggering electronic equipment and delete the operation for the image that image-region 514 is shown (as touched Operation).In response to the user's operation detected, electronic equipment can delete the image.In some other embodiment, response In the user's operation detected, electronic equipment can also delete the image and with the affective state of the image associated storage/ Emotion element.Here, electronic equipment can execute the delete operation in local side or cloud server.

Control 515D can be used for monitoring the operation (such as touch operation) that Emotion element is presented in triggering electronic equipment.In response to inspection Corresponding Emotion element can be presented in the user's operation measured, electronic equipment, which is to be associated with to deposit with the image The corresponding Emotion element of the affective state of storage, alternatively, the Emotion element is and the Emotion element of the image associated storage.

In the other some embodiments of the application, it is not limited to the user's operation in response to receiving on control 515D, Electronic equipment corresponding Emotion element can also be presented when user checks image by other interactive elements.For example, electronics Equipment can also detect operation (such as touch operation, double click operation, the long-pressing acted on picture 510E as shown in Figure 5 B Operation etc.) when, corresponding Emotion element is presented.

When electronic device association stores the affective state of the image got and the user recognized, electronic equipment can To know the affective state of the user for the image associated storage checked with active user, and looked on local side or cloud server The corresponding one or more Emotion elements of the affective state are found, and when user checks the image according to certain presentation strategy The one or more Emotion element is presented.

When electronic device association stores the image got and corresponding with the affective state of the user recognized one When a or multiple Emotion elements, electronic equipment can be presented when user checks the image according to certain presentation strategy this one A or multiple Emotion elements.

The user interface 53 that Fig. 5 D- Fig. 5 K is illustrated can be electronic equipment in associated storage image and emotion shape After state/Emotion element, several implementations of corresponding Emotion element are presented when user checks the image again.Below in detail Strategy of the thin description electronic equipment when corresponding Emotion element is presented.

(1) the first strategy: an Emotion element can be presented in electronic equipment when user checks image.One emotion Element is any one or electronics in one or more Emotion elements corresponding with the affective state of the image associated storage Equipment default one, alternatively, an Emotion element be and one or more Emotion elements of the image associated storage in One of any one or electronic equipment default.

Fig. 5 D shows the mode that a kind of electronic equipment shows an Emotion element.As shown in Figure 5 D, in user interface 53 An Emotion element is shown, which is expression packet 516A.Expression packet 516A is the emotion with 514 associated storage of picture One of any one or electronic equipment default in one or more Emotion elements corresponding to state " happy ", alternatively, Expression packet 516A is any one or electronic equipment default in one or more Emotion elements with 514 associated storage of picture One.

(2) second of strategy: multiple Emotion elements can be presented in electronic equipment when user checks image.Multiple emotion Element is some or all of in one or more Emotion element corresponding with the affective state of the image associated storage, alternatively, One Emotion element be and one or more Emotion elements of the image associated storage in some or all of.

Fig. 5 E shows the mode that a kind of electronic equipment shows multiple Emotion elements.As shown in fig. 5e, in user interface 53 Multiple Emotion elements are shown, multiple Emotion element includes expression packet 516A, text bubble 516B, text bubble 516C, text The high saturation that this information 516D and the image are presented.Expression packet 516A, text bubble 516B, text bubble 516C, text The high saturation that information 516D and the image are presented is one corresponding to the affective state " happy " with 514 associated storage of picture Some or all of in a or multiple Emotion elements, alternatively, expression packet 516A, text bubble 516B, text bubble 516C, text Information 516D be and 514 associated storage of picture one or more Emotion elements in some or all of.

It is intelligible, it is not limited to the Emotion element shown in user interface 53 in Fig. 5 D or Fig. 5 E, electronic equipment may be used also Other Emotion elements are presented.Such as electronic equipment can also adjust the mode of composition of image in view-finder 305, effect of shadow, satisfy With degree, tone and color, background music etc. can also be played by loudspeaker 170A, to reflect the current emotion shape of user State.

In some embodiments, user interface 53 shown in Fig. 5 D or Fig. 5 E may also include control 517.Control 517 can be used In the operation (such as touch operation) for monitoring triggering electronic equipment stopping presentation Emotion element.In response to the operation detected, electricity Sub- equipment can stop that all Emotion elements are presented.In some embodiments, in response to the operation detected, electronic equipment can be jumped Go to user interface 52 as shown in Figure 5 C.Such mode, which can be convenient user's switching, to be had under Emotion element and ameleia element Image-region 514.

In other some embodiments, it is not limited to above-mentioned control 517, other interactive elements can be used for user's switching There is the image-region 514 under Emotion element and ameleia element.For example, user can also the image district shown in Fig. 5 D or Fig. 5 E Long-pressing, double-click are made on domain 514, draw the gestures such as circle to switch the image-region 514 having under Emotion element and ameleia element.

(3) the third strategy: electronic equipment can need that one or more Emotion elements are presented according to user.This or Multiple Emotion elements are one or more Emotion elements corresponding with the affective state of the image associated storage, alternatively, should One or more Emotion elements are and one or more Emotion elements of the image associated storage.

Fig. 5 F- Fig. 5 H shows the mode that corresponding Emotion element is presented in another electronic equipment.As illustrated in figure 5f, user Wicket 518 is shown in interface 53.Wicket 518 can be used for user and corresponding Emotion element selected to present.Wicket It may include icon 518A, text information " Emotion element " 518B, multiple Emotion element options such as " original of Emotion element in 518 Figure " 518C, high saturation 518D, text bubble " live up！" 518E, expression packet 518F, " daytime sings must be vertical for text information Wine, the youth, which keeps company, returns to one's native " 518G, background music " song ' today be an auspicious day ' " 518H etc..Here, icon 518A, text This information " Emotion element " 518B, multiple Emotion element option 518C-518H can refer in the wicket 311 of Fig. 3 F- Fig. 3 H Icon 311A, text information " Emotion element " 311B, multiple Emotion element option 311C-311H, which is not described herein again.

In some embodiments, wicket 518 may also include control 518I and/or control 518J.Control 518I, control 518J can refer to control 311I, the control 311J in the wicket 311 in Fig. 3 F- Fig. 3 H, and which is not described herein again.

In some embodiments, wicket 518 may also include control 518K and control 518L.Control 518K, control 518L It can refer to control 311K, the control 311L in the wicket 311 in Fig. 3 F- Fig. 3 H, which is not described herein again.

(4) the 4th kinds of strategies: electronic equipment can show the image under Emotion element and ameleia element simultaneously.The feelings Sense element can be one or more Emotion elements corresponding with the affective state of the image associated storage, alternatively, the emotion is first Element is and one or more Emotion elements of the image associated storage.

The user interface 53 that Fig. 5 I- Fig. 5 K is illustrated can be electronic equipment while show Emotion element and mercilessness Feel the several ways of the image under element.

As shown in fig. 5i, the picture 519 of ameleia element, which can suspend, is shown on the picture 520 of Emotion element, i.e., The part display area of the covering picture 520 of picture 519.

As indicated at figure 5j, there is the picture 520 of Emotion element that can suspend to be shown on the picture 519 of ameleia element, i.e., The part display area of the covering picture 519 of picture 520.

As it can be seen from figure 5k, the picture 519 of ameleia element and there is the picture 520 of Emotion element that can be shown in user circle The different zones in face 53.

Here, there is the Emotion element on the picture 520 of Emotion element in the user interface 53 shown in Fig. 5 I- Fig. 5 K, it can be with For one or more Emotion elements corresponding with the affective state of picture associated storage in the image-region 514, alternatively, for and One or more Emotion elements of picture associated storage in image-region 514.

In some embodiments, having in user interface 53 shown in Fig. 5 I- Fig. 5 K on the picture 520 of Emotion element can be with Including prompt information, Emotion element is presented on user's picture 520 for prompting.The prompt information can show as icon, Text information or other forms.

In some embodiments, user interface 53 shown in Fig. 5 I- Fig. 5 K can also receive user's operation (as touched behaviour Work, double click operation etc.), in response to the user's operation, electronic equipment can replace the display position of picture 519 and picture 520.It should User's operation can act on picture 519 or picture 520.

Above-mentioned Fig. 5 I- Fig. 5 K embodiment can user side just compare to have checked under Emotion element and ameleia element Image.

It is not limited to the mode of several presentations and the corresponding Emotion element of image that above-mentioned Fig. 5 D- Fig. 5 K is illustrated, this Emotion element corresponding with image can also be presented in application embodiment by other means, such as electronic equipment can also be with fixation Time interval toggles the image etc. shown under Emotion element and ameleia element, and the embodiment of the present application does not limit this System.

Emotion element can be presented for user in the user interface that above-mentioned Fig. 5 D- Fig. 5 K is illustrated, and reproduce electronic equipment The affective state recognized when obtaining picture promotes the usage experience of user so that user generates affectional sympathetic response.

In some embodiments, when electronic equipment recognizes user and wants to check the scene of image, it can identify and work as front court The affective state of user under scape, and be that user pushes specific image with certain push strategy according to the affective state.

The user interface that Fig. 6 A- Fig. 6 B is illustrated can recognize user for electronic equipment and want to check the field of image Jing Shi identifies the affective state of user and pushes a kind of mode of specific image for user.

The user interface 51 that Fig. 6 A is illustrated can be used for showing multiple photograph albums.Shown in the user interface 51 and Fig. 5 A User interface 51 it is identical, can refer to associated description.When electronic equipment shows user interface 51 as shown in Figure 6A, electronics is set It is standby to can be confirmed that user is currently at the scene for wanting to check picture.

When electronic equipment recognizes user and is currently at the scene for wanting to check picture, if being not turned on " emotion recognition ", User can be then prompted to open " emotion recognition ".Prompting mode may include but be not limited in user interface 51 as shown in Figure 6A Display reminding information, playing alert tones etc..The prompt information can be implemented as icon, text etc..In some embodiments, as schemed The switch control of " emotion recognition " can be shown in user interface 51 shown in 6A, user can opening in " emotion recognition " Close input operation (such as touch operation) Lai Kaiqi " emotion recognition " on control.In further embodiments, user can also pass through The mode as shown in Fig. 2A-Fig. 2 B opens " emotion recognition ".

After electronic equipment opens " emotion recognition ", the affective state of user under current scene can be identified.Electronic equipment is known The operation of the affective state of user can refer to the associated description of embodiment above under other current scene, and details are not described herein.

It can be to use with certain push strategy according to the affective state after electronic equipment recognizes the affective state of user Family pushes specific image.The push strategy can include but is not limited to: 1. when the affective state of active user is " sorrow ", Image accessed when the affective state of user is " happiness " or " romance " is pushed for user.Such push strategy can disappear Except the unhealthy emotion of user.2. pushing user when the affective state of active user is " calmness " for user or electronic equipment being pre- The image first set, such as push image accessed when the affective state of user is " pleasure " or " happy ".

The user interface 61 that Fig. 6 B is illustrated can be electronic equipment be user push specific image when a kind of side Formula.As shown in Figure 6B, user interface 61 can include: status bar 601, return key, current page indicator, one or more photograph albums Entry.

Status bar 601 can refer to the status bar 201 in user interface 21 shown in Fig. 2A, and which is not described herein again.

Return key 604 is APP grades of return keys, can be used for returning menu upper level.The upper level page of user interface 61 can Think user interface 51 as shown in Figure 6A.

Current page indicator 602 and 603 may be used to indicate current page, such as text information " photograph album " " recommendation " can be used It is used to show the photograph album recommended in instruction current page.It is not limited to text information, current page indicator 602 and 603 can be with It is icon.

Photograph album entry can include: it is that electronic equipment is recommended according to the affective state of active user, with electronic equipment obtain scheme As when user affective state classify for example happy photograph album entry 605A of one or more photograph album entries, romantic photograph album entry 605B.Here, happy photograph album entry 605A, romance photograph album entry 605B can refer to opening in user interface 51 as shown in Figure 5A Heart photograph album entry 503A, romantic photograph album entry 503B.Happy photograph album entry 605A, romance photograph album entry 605B can receive user It operates (such as touch operation), in response to the user's operation, electronic equipment can show the image under corresponding photograph album.Electronic equipment is aobvious Show that the implementation of the image under corresponding photograph album can refer to Fig. 5 B- Fig. 5 K and associated description, which is not described herein again.

In this application, the user interface for being used to show the image of some photograph album can be known as second user interface, it will Image in the photograph album is known as third image, may include the second image that electronic equipment saves in third image.The second user The example implementations at interface can be user interface 52 shown in Fig. 5 B, and the example implementations of third image may include Picture 510A, 510B, 510D-510H, 510J-510L and video 510C, 510I in user interface 52 shown in Fig. 5 B, the second figure The example implementations of picture can be picture 510E.

In this application, the operation for being used to check the second image can be known as the second operation, can will be responsive to this Two operations and show for showing that the user interface of the second image is known as third user interface, this can be will be responsive to and second grasped The Emotion element made and presented is known as the second Emotion element.Second Emotion element includes first with the second image associated storage The corresponding one or more Emotion elements of affective state, alternatively, second Emotion element includes the first Emotion element.

The example implementations of second operation are on the picture 510E acted in user interface 52 shown in Fig. 5 B Operation (such as touch, double-click or long-pressing etc.).The example implementations of the third user interface may include shown in Fig. 5 D- Fig. 5 K User interface 53.The example implementations of second Emotion element can refer to electronic equipment and show shown in Fig. 5 D- Fig. 5 K User interface 53 when the Emotion element that is presented, can refer to the associated description of embodiment above.

In some embodiments of the application, the user interface for being used to show one or more photograph album entries is known as the 4th use Family interface, the one or more photograph album entry include the first photograph album entry；It will act on being used to open in the first photograph album entry The operation at second user interface is known as third operation；Entry corresponding to the photograph album of second user showing interface is the first photograph album item Mesh.The example implementations of the fourth user interface may include user interface 51 as shown in Figure 5A.The first photograph album entry Example implementations can be " happy " photograph album entry 503A in Fig. 5 A.The example implementations of third operation can Operation (such as touching, long press operation) including acting on " happy " photograph album entry 503A.

In other embodiments of the application, the user interface for being used to show one or more photograph album entries can be known as Fourth user interface, the one or more photograph album entry include the first photograph album entry；It will be used to show and be preset according to electronic equipment Generalization bounds, according to the affective state recognized in one or more photograph album entries in fourth user interface determine phase The user interface of volume entry is known as the 5th user interface, and the first photograph album entry is shown in the 5th user interface；It will act on The operation for being used to open second user interface on five interfaces in the first photograph album entry is known as third operation；The exhibition of second user interface Entry corresponding to the photograph album shown is the first photograph album entry.The example implementations of the fourth user interface may include such as Fig. 6 A Shown in user interface 51.The example implementations of 5th user interface can be user interface 61 shown in Fig. 6 B.It should The example implementations of first photograph album entry can be " happy " photograph album entry 605A in Fig. 6 B.The example of third operation Property implementation may include the operation (such as touch, long press operation) for acting on " happy " photograph album entry 605A.

The determination method or setting of Emotion element corresponding to different emotions state in the embodiment of the present application are described below Method.

In some embodiments, Emotion element corresponding to different emotions state is deposited by electronic equipment default setting and in advance Storage is in electronic equipment local side or cloud server, alternatively, by being stored in electronic equipment sheet after cloud server default setting Ground terminal or cloud server.It is corresponding that electronic equipment can get some affective state from local side or cloud server Emotion element.

In further embodiments, Emotion element corresponding to different emotions state is independently arranged by user.Fig. 7 A- Fig. 7 D The user interface illustrated can be a kind of implementation of Emotion element corresponding to user setting different emotions state.

The user interface 71 that Fig. 7 A is illustrated can be a kind of implementation of " set interface ".The user interface 71 It can be what " setting " application provided." setting " application is mounted to a on the electronic equipments such as smart phone, tablet computer For being arranged the application program of the various functions of electronic equipment, the embodiment of the present application to the title of the application program with no restrictions. The user interface 71 can be user and click the user interface that the setting icon 215 in Fig. 2A is opened.

As shown in Figure 7 A, user interface 71 can include: status bar 701, current page indicator 702, search box 703, figure Mark 704, one or more setting options.

Status bar 701 can refer to the status bar 201 in user interface 21 shown in Fig. 2A, and which is not described herein again.

Current page indicator 702 may be used to indicate current page, such as text information " setting " may be used to indicate currently The page is for showing one or more setting options.It is not limited to text information, current page indicator 702 can also be icon.

Search box 703 can be used for monitoring the operation (such as touch operation) for passing through text search setting option.In response to the operation, Electronic equipment can display text input frame so that user shown in input frame want search setting option.

Icon 704 can be used for monitoring the operation (such as touch operation) for passing through phonetic search setting option.In response to the operation, electricity Sub- equipment can show voice input interface, so that user inputs voice in the voice input interface, to search for setting option.

Setting option can include: log in Huawei's account setting option, wirelessly connect setting option, application with network settings item, equipment It is set with notice setting option, battery setting option, display setting option, sound setting option, emotion recognition setting option 705, safety and privacy Set item, user and account setting option etc..The form of expression of each setting option may include icon and/or text, the application to this not It is limited.Each setting option can be used for monitoring the operation (such as touch operation) that triggering display is correspondingly arranged the set content of item, ring Should be in the operation, electronic equipment is openable for showing the user interface for the set content for being correspondingly arranged item.

Illustratively, the operation (such as touch operation) on emotion recognition setting option 705 is detected, electronic equipment can be shown User interface 72 as shown in Figure 7 B.

As shown in Figure 7 B, user interface 72 is used to show the corresponding content of emotion recognition setting option.The user interface 72 can It include: status bar, return key 706, current page indicator 707 and 708, affective state display area 709.

Return key 706 is APP grades of return keys, can be used for returning menu upper level.The upper level page of user interface 72 can Think user interface 71 as shown in Figure 7 A.

Current page indicator 707 and 708 may be used to indicate current page, such as text information " setting " " emotion recognition " Current page be may be used to indicate for showing the corresponding content of emotion recognition setting option, be not limited to text information, current page refers to Show that symbol 707 and 708 can also be icon.

Affective state display area 709 is for showing one or more affective state options.In some embodiments, this one A or multiple affective state options can be electronic equipment default setting, such as can be default setting when electronic equipment dispatches from the factory , in another example can be the corresponding option of the affective state to match with the personality of user that electronic equipment is analyzed.At it In his some embodiments, which can be what user was independently arranged.

Illustratively, as shown in Figure 7 B, which includes: " happy " option 709A, " wave It is unrestrained " option 709D, " sorrow " option 709G.The form of expression of the one or more affective state option can be text information (such as text information " happy "), or icon, the embodiment of the present application are without limitation.

In some embodiments, it may also include the corresponding deletion control of each affective state option in region 709, such as " open The corresponding deletion control 709B of the heart " option 709A, the corresponding deletion control 709E of " romance " option 709D, " sorrow " option 709A Corresponding deletion control 709B.The operation (such as touch operation) for deleting corresponding affective state option can be monitored by deleting control, In response to the operation, electronic equipment can delete corresponding affective state option.The form of expression for deleting control can be text Information (such as text information " deletion "), or icon, the embodiment of the present application are without limitation.

In some embodiments, user interface 72 may also include control 710.Control 710 can be monitored for adding emotion shape The operation (such as touch operation) of state option, in response to the operation, electronic equipment can be shown for adding affective state option Interface, user can select on the surface or customized affective state option to be added.User adds affective state option Afterwards, the affective state option can be increased newly in the region 709 in user interface 72.The form of expression of control 710 can be text envelope Breath, or icon, the embodiment of the present application are without limitation.

It may also include the control for the corresponding Emotion element of each affective state option to be arranged in region 709, such as " open The corresponding setting control 709C of the heart " option 709A, the corresponding setting control 709F of " romance " option 709D, " sorrow " option 709A Corresponding setting control 709I.The form of expression of setting control can be text information (such as text information " setting emotion member Element "), or icon, the embodiment of the present application are without limitation.Setting control can be monitored for corresponding emotion member to be arranged The operation (such as touch operation) of element, in response to the operation, electronic equipment can open the use for corresponding Emotion element to be arranged Family interface.

Illustratively, the operation (such as touch operation) on setting control 709C is detected, electronic equipment can be shown such as User interface 73 shown in Fig. 7 C.

As seen in figure 7 c, user interface 73 is for showing the corresponding Emotion element setting option of affective state " happy ".The use Family interface 73 can include: status bar, return key 711, current page indicator 712 and 713, one or more Emotion element classification Option.

Return key 706 is APP grades of return keys, can be used for returning menu upper level.The upper level page of user interface 73 can Think user interface 72 as shown in Figure 7 B.

Current page indicator 712 and 713 may be used to indicate current page, such as text information " setting Emotion element " " happy " may be used to indicate current page for showing the corresponding Emotion element setting option of affective state " happy ", be not limited to text Information, current page indicator 712 and 713 can also be icon.

Emotion element classification options can include: shadow option 714A, composition option 714B, color option 714C, picture choosing Item 714D, the text option 714E and background music option 714F.Emotion element classification options as seen in figure 7 c are not limited to, may be used also To include more or less or other Emotion element classification options, such as saturation degree option, tone option etc., the embodiment of the present application It is without limitation.

Each Emotion element classification options can monitor the user interface for being used to open the Emotion element that corresponding classification is arranged It operates (such as touch operation), which selects for user or customized corresponding one or more Emotion elements.Example Such as, in response to the operation (such as touch operation) detected on shadow option 714A, electronic equipment is openable for emotion to be arranged The user interface of the corresponding effect of shadow of state " happy "；It (is such as touched in response to the operation detected on composition option 714B Operation), electronic equipment is openable for the user interface of the corresponding patterning effect of affective state " happy " to be arranged；In color option The operation (such as touch operation) detected on 714C, electronic equipment are openable for the corresponding color of affective state " happy " to be arranged The user interface of effect；The operation (such as touch operation) detected on picture option 714D, electronic equipment are openable for setting Set the user interface of the corresponding picture of affective state " happy "；The operation detected on the text option 714E is (as touched behaviour Make), electronic equipment is openable for the user interface of the corresponding text of affective state " happy " to be arranged；In background music option The operation (such as touch operation) detected on 714F, electronic equipment are openable for the corresponding background of affective state " happy " to be arranged The user interface of music.

With reference to Fig. 7 D, Fig. 7 D shows the use for being used to be arranged the corresponding color of affective state " happy " of electronic equipment opening Family interface 74.The user interface 74 can include: status bar, return key 716, current page indicator 717 and 718, one or more A color option.

Return key 716 is APP grades of return keys, can be used for returning menu upper level.The upper level page of user interface 74 can Think user interface 73 as seen in figure 7 c.

Current page indicator 717 and 718 may be used to indicate current page, such as text information " setting Emotion element " " color " may be used to indicate current page for " color " this kind of other Emotion elements to be arranged, and be not limited to text information, currently Page indicator 717 and 718 can also be icon.

Color option can include: bright-coloured 719, gray scale 720, dimness 721, normal 722.It is not limited to as illustrated in fig. 7d several Color option can also include that more or less or other color options, the embodiment of the present application are without limitation.Each color Option can receive user's operation (such as touch operation), and in response to the user's operation, electronic equipment can be corresponding by the color option Color be set as the corresponding Emotion element with affective state " happy ".In some embodiments, the good Emotion element of user setting Afterwards, prompt information can also be shown in user interface 74, to prompt user's Emotion element to be arranged to " open with affective state The corresponding Emotion element of the heart ".The prompt information for example can be icon 723 as illustrated in fig. 7d, and icon 723 can prompt user's color Color " bright-coloured " is arranged to the corresponding Emotion element with affective state " happy ".

Intelligible, the emotion of other classifications can also be arranged in user on the other users interface of similar user interfaces 74 Element.Here, the Emotion element of user setting can store in electronic equipment local side, be stored in cloud server. For example, user can select background music in electronic equipment local side, background music, this Shen can also be selected by server beyond the clouds Please embodiment it is without limitation.

Each embodiment of the application can be arbitrarily combined, to realize different technical effects.

In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof real It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.The computer program Product includes one or more computer instructions.When loading on computers and executing the computer program instructions, all or It partly generates according to process described herein or function.The computer can be general purpose computer, special purpose computer, meter Calculation machine network or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or It is transmitted from a computer readable storage medium to another computer readable storage medium, for example, the computer instruction can To pass through wired (such as coaxial cable, optical fiber, digital subscriber from a web-site, computer, server or data center Line) or wireless (such as infrared, wireless, microwave etc.) mode to another web-site, computer, server or data center into Row transmission.The computer readable storage medium can be any usable medium or include one that computer can access Or the data storage devices such as integrated server, data center of multiple usable mediums.The usable medium can be magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disk Solid State Disk) etc..

Claims

1. a kind of method for recording user feeling characterized by comprising

Electronic equipment shows the first user interface, shows the first image in first user interface；

The electronic equipment recognizes user and is currently located in the first affective state；

In response to the first operation detected, the second image of associated storage and first affective state, alternatively, associated storage Second image and the first Emotion element；First Emotion element includes one corresponding with first affective state Or multiple Emotion elements, first Emotion element reflect first affective state；

Second image is identical with the first image, alternatively, the first image is the preview image before shooting, second figure As the image for shooting.

2. the method according to claim 1, wherein first user interface includes the first interactive elements, institute The first interactive elements are stated for monitoring the operation for saving second image；First operation includes that the electronic equipment detects To the operation for acting on first interactive elements.

3. method according to claim 1 or 2, which is characterized in that the second image described in associated storage and described first After affective state, alternatively, after the second image and the first Emotion element described in associated storage, the method also includes:

The electronic equipment shows second user interface, shows one or more third images in the second user interface, The third image includes second image；

In response to the second operation checked to the second image in the second user interface detected, the electronics is set Standby display third user interface, also, the second Emotion element is presented in the electronic equipment；The third user interface includes described Second image；Second Emotion element includes one corresponding with the first affective state of the second image associated storage Or multiple Emotion elements, alternatively, second Emotion element includes first Emotion element.

4. according to the method described in claim 3, it is characterized in that, the electronic equipment show second user interface before, institute State method further include:

The electronic equipment shows fourth user interface, shows one or more photograph album entries in the fourth user interface, One or more of photograph album entries include the first photograph album entry, and the first photograph album entry corresponds to the first emotion shape State；It include that user be in electronic equipment preservation when first affective state in the corresponding photograph album of the first photograph album entry Image；

In response to the third operation for acting on the first photograph album entry detected, the electronic equipment shows that described second uses Family interface.

5. according to the method described in claim 3, it is characterized in that, the electronic equipment show second user interface before, institute State method further include:

The electronic equipment recognizes the second affective state that user is currently at, and shows the 5th user interface；Described 5th The one or more photograph album entries shown in user interface are by the electronic equipment according to preset Generalization bounds, according to described It is determined in one or more photograph album entries that second affective state is shown in the fourth user interface；5th user The one or more photograph album entries shown in interface include the first photograph album entry；

In response to the third operation for acting on the first photograph album entry in the 5th user interface detected, the electronics is set It is standby to show the second user interface.

6. method according to claim 4 or 5, which is characterized in that one or more shown in the fourth user interface A photograph album entry includes: to correspond respectively to one or more photograph album entries of different affective states.

7. method according to claim 1-6, which is characterized in that the electronic equipment recognizes user and currently locates After the first affective state, before detecting first operation, the method also includes:

One or more Emotion elements corresponding with first affective state are presented in the electronic equipment.

8. the method according to the description of claim 7 is characterized in that first affective state that the electronic equipment is presented and described Corresponding one or more Emotion element, is that user selects, alternatively, being the electronic equipment default choice, alternatively, being The electronic equipment is randomly selected.

9. method according to claim 1-8, which is characterized in that with first affective state corresponding one A or multiple Emotion elements include: the saturation of text information, picture, music, the composition of image, the effect of shadow of image, image The color of degree, the tone of image or image.

10. -9 described in any item methods according to claim 1, which is characterized in that it is current that the electronic equipment recognizes user It is in front of the first affective state, the method also includes:

The electronic equipment detects the 4th operation for opening emotion recognition service, in response to the 4th operation, identification The affective state that user is currently located in；Emotion recognition service for the electronic equipment pass through the hardware device configured or Hardware device by starting the other equipment for being connected to the electronic equipment acquires the behavioral data of user, according to the row of user Affective state that user is currently located in is identified for data.

11. according to the method described in claim 10, it is characterized in that, the 4th operation includes that the electronic equipment detects The operation for acting on the second interactive elements in first user interface, second interactive elements for monitor open institute State the operation of emotion recognition service.

12. a kind of electronic equipment characterized by comprising one or more processors, memory and display screen；

The memory is coupled with one or more of processors, and the memory is for storing computer program code, institute Stating computer program code includes computer instruction, and one or more of processors call the computer instruction so that institute State electronic equipment execution:

The first user interface is shown on the display screen, shows the first image in first user interface；

It recognizes user and is currently located in the first affective state；

13. electronic equipment according to claim 12, which is characterized in that first user interface includes the first interaction member Element, first interactive elements are for monitoring the operation for saving second image；First operation includes that the electronics is set The standby operation for acting on first interactive elements detected.

14. electronic equipment according to claim 12 or 13, which is characterized in that one or more of processors are also used to The computer instruction is called so that the electronic equipment executes:

After second image described in associated storage and first affective state, alternatively, the second image described in associated storage and After first Emotion element, second user interface is shown on the display screen, shows one in the second user interface Or multiple third images, the third image include second image；

In response to the second operation checked to the second image in the second user interface detected, in the display Screen display third user interface, also, the second Emotion element is presented；The third user interface includes second image； Second Emotion element includes one or more feelings corresponding with the first affective state of the second image associated storage Element is felt, alternatively, second Emotion element includes first Emotion element.

15. electronic equipment according to claim 14, which is characterized in that one or more of processors are also used to call The computer instruction is so that the electronic equipment executes:

Before showing second user interface on the display screen, fourth user interface is shown on the display screen, described One or more photograph album entries are shown in four user interfaces, one or more of photograph album entries include the first photograph album entry, The first photograph album entry corresponds to first affective state；It include at user in the corresponding photograph album of the first photograph album entry The image that the electronic equipment saves when first affective state；

In response to the third operation for acting on the first photograph album entry detected, described second is shown on the display screen User interface.

16. electronic equipment according to claim 14, which is characterized in that one or more of processors are also used to call The computer instruction is so that the electronic equipment executes:

The second affective state that user is currently at is recognized, and shows the 5th user interface on the display screen；Described The one or more photograph album entries shown in five user interfaces are by the electronic equipment according to preset Generalization bounds, according to institute State determination in one or more photograph album entries that the second affective state is shown in the fourth user interface；Described 5th uses The one or more photograph album entries shown in the interface of family include the first photograph album entry；

17. electronic equipment according to claim 15 or 16, which is characterized in that one shown in the fourth user interface A or multiple photograph album entries include: to correspond respectively to one or more photograph album entries of different affective states.

18. the described in any item electronic equipments of 2-17 according to claim 1, which is characterized in that one or more of processors It is also used to call the computer instruction so that the electronic equipment executes:

It recognizes user to be currently located in after the first affective state, before detecting first operation, present and described first The corresponding one or more Emotion elements of affective state.

19. electronic equipment according to claim 18, which is characterized in that first feelings that the electronic equipment is presented and described The corresponding one or more Emotion elements of sense state, are that user selects, alternatively, be the electronic equipment default choice, Alternatively, being that the electronic equipment is randomly selected.

20. the described in any item electronic equipments of 2-19 according to claim 1, which is characterized in that and the first affective state phase Corresponding one or more Emotion element includes: text information, picture, music, the composition of image, the effect of shadow of image, figure The color of the saturation degree of picture, the tone of image or image.

21. the described in any item electronic equipments of 2-20 according to claim 1, which is characterized in that one or more of processors It is also used to call the computer instruction so that the electronic equipment executes:

It recognizes user to be currently located in front of the first affective state, detects the 4th operation for opening emotion recognition service, In response to the 4th operation, the affective state that identification user is currently located in；The emotion recognition service is set for the electronics The standby hardware device by configuring or the acquisition of the hardware device by starting the other equipment for being connected to the electronic equipment are used The behavioral data at family identifies the affective state that user is currently located according to the behavioral data of user.

22. electronic equipment according to claim 21, which is characterized in that the 4th operation includes that the electronic equipment is examined What is measured acts on the operation of the second interactive elements in first user interface, and second interactive elements are opened for monitoring Open the operation of the emotion recognition service.

23. a kind of computer program product comprising instruction, which is characterized in that when the computer program product is in electronic equipment When upper operation, so that the electronic equipment executes such as method of any of claims 1-11.

24. a kind of computer readable storage medium, including instruction, which is characterized in that when described instruction is run on an electronic device When, so that the electronic equipment executes such as method of any of claims 1-11.