WO2020151580A1 - 一种屏幕控制和语音控制方法及电子设备 - Google Patents

一种屏幕控制和语音控制方法及电子设备 Download PDF

Info

Publication number
WO2020151580A1
WO2020151580A1 PCT/CN2020/072610 CN2020072610W WO2020151580A1 WO 2020151580 A1 WO2020151580 A1 WO 2020151580A1 CN 2020072610 W CN2020072610 W CN 2020072610W WO 2020151580 A1 WO2020151580 A1 WO 2020151580A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
electronic device
yaw
face
display screen
Prior art date
Application number
PCT/CN2020/072610
Other languages
English (en)
French (fr)
Inventor
辛志华
陈涛
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020151580A1 publication Critical patent/WO2020151580A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification
    • G10L17/22—Interactive procedures; Man-machine interfaces
    • G10L17/24—Interactive procedures; Man-machine interfaces the user being prompted to utter a password or a predefined phrase
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering

Definitions

  • the embodiments of the present application relate to the field of electronic technology, and in particular to a screen control and voice control method and electronic equipment.
  • the above-mentioned display screen may be a touch screen.
  • large household appliances such as refrigerators, washing machines, air conditioners, and small household appliances such as speakers, air purifiers, and kitchen and bathroom supplies can be equipped with displays.
  • the display screen can display one or more of the operating parameters of the corresponding household equipment, home monitoring, clock and calendar, digital photo albums and news information.
  • the display screen is generally always on or lights up in response to a user's operation of a physical button or a display screen (such as a touch screen).
  • the always-on display will increase the energy consumption of home equipment and cause unnecessary energy loss.
  • the constant light of the display will also accelerate the wear of the display and shorten the service life of the display. And lighting the display screen in response to the user's operation will increase the time required for the home equipment, and affect the user experience.
  • the embodiments of the present application provide a screen control and voice control method and an electronic device, which can automatically light up the display screen of the electronic device when the display screen of the electronic device is used or is highly likely to be viewed. In this way, the possibility of the display screen being illuminated by mistake can be reduced, and the waste of energy consumption of the electronic device can be reduced.
  • an embodiment of the present application provides a screen control method, which can be applied to an electronic device.
  • the electronic device includes a display screen and a camera.
  • the screen control method may include: when the display screen is black, the electronic device collects a first picture through a camera; the electronic device recognizes that the first picture includes a face image, and obtains the face yaw of the first user; If the user's human face yaw degree is within the first preset angle range, the electronic device automatically lights up the display screen.
  • the first user is the user corresponding to the face image in the first picture; the face yaw of the first user is the left and right rotation angle of the first user's face with respect to the first link. It is the connection between the camera and the head of the first user.
  • the electronic device can automatically light up the display screen.
  • the electronic device can automatically light up the display screen of the electronic device when the display screen is more likely to be used or viewed. In this way, the possibility of the display screen being illuminated by mistake can be reduced, and the waste of energy consumption of the electronic device can be reduced.
  • the electronic device in response to determining that the yaw of the first user's face is within the first preset angle range, the electronic device automatically lights the display screen, including: The yaw of the human face of a user is within the first preset angle range, and the first user's eyes are opened, and the electronic device automatically lights up the display screen.
  • the electronic device can automatically light up the display.
  • the electronic device does not light up the display screen. In this way, the possibility of the display screen being illuminated by mistake can be reduced, the waste of energy consumption of the electronic device can be reduced, and the intelligence of the interaction can be improved.
  • the electronic device in response to determining that the first user's face yaw degree is within the first preset angle range, the electronic device automatically lights the display screen, including: responding to determining The yaw degree of the human face of the first user is within the first preset angle range, and the eyes of the first user look at the display screen, and the electronic device automatically lights the display screen.
  • the electronic device can automatically light up the display.
  • the electronic device does not light up the display screen. In this way, the possibility of the display screen being illuminated by mistake can be reduced, the waste of energy consumption of the electronic device can be reduced, and the intelligence of interaction can be improved.
  • the electronic device in response to determining that the first user's face yaw degree is within the first preset angle range, the electronic device automatically lights the display screen, including: responding to determining The face yaw of the first user is within the first preset angle range, and the duration of the first user's face yaw within the first preset angle range exceeds the preset time threshold, the electronic device automatically lights up Display screen.
  • the duration of the human face yaw within the first preset angle range does not exceed the preset time threshold, it means that the user is not paying attention to the display screen. It may just be that the user is facing the display screen when turning around or turning his head.
  • the yaw of the human face is within the first preset angle range. In this case, the electronic device will not light up the display.
  • the duration of the human face yaw degree within the first preset angle range exceeds the preset time threshold, it means that the user is paying attention to the display screen, and the electronic device can automatically light up the display screen. Therefore, the accuracy of judgment can be improved, and the intelligence of interaction can be improved.
  • the electronic device before the electronic device automatically lights up the display screen, the electronic device can also obtain the position yaw of the first user.
  • the position yaw of the first user is determined by the camera and The angle between the line of the head of the first user and the first line, the first line being perpendicular to the display screen, and the first line passing the camera.
  • the electronic device In response to determining that the human face yaw of the first user is within the first preset angle range, the electronic device automatically lights the display screen, including: responding to determining that the human face yaw of the first user is within the first preset angle Within the range, and the position yaw of the first user is within the second preset angle range, the electronic device automatically lights up the display screen.
  • the position yaw degree is not within the second preset angle range, it means that the user paying attention to the display screen is on both sides of the electronic device, and is in a relatively remote direction from the front of the electronic device.
  • the user may not be the owner of the electronic device, or the user may be operating or viewing the electronic device without the owner's consent.
  • the user may be triggering the electronic device to light up the display screen through the method of the embodiment of the present application; or, the user may be stealing the content displayed on the display screen of the electronic device.
  • the electronic device is currently black, the electronic device will not light up the screen; if the electronic device is currently on the screen, the electronic device can automatically black out the screen. In this way, the data stored in the electronic device can be protected from being stolen.
  • the method of the embodiment of the present application may further include: in response to determining that the position yaw of the first user is not within the second preset angle range, the electronic device issues an alarm indication .
  • the alarm indication can remind the owner that other users are paying attention to the display screen.
  • the method of the embodiment of the present application may further include: the electronic device performs face recognition on the first user.
  • the electronic device In response to determining that the human face yaw of the first user is within the first preset angle range, the electronic device automatically lights the display screen, including: responding to determining that the human face yaw of the first user is within the first preset angle Within the range, and the face recognition of the first user passes, the electronic device automatically lights up the display screen.
  • the electronic device can determine that a user is paying attention to the display screen of the electronic device. If a user is following the display screen and the face recognition fails, it means that the user following the display screen is not an authorized user. At this time, if the electronic device is currently black, the electronic device will not light up the screen; if the electronic device is currently on the screen, the electronic device can automatically black out the screen. In this way, the data stored in the electronic device can be protected from being stolen.
  • the method of the embodiment of the present application may further include: the electronic device collects voice data through a microphone; the electronic device acquires the sound source yaw degree of the voice data, and the sound source yaw degree Is the angle between the connection line between the camera and the sound source of the voice data and the first straight line; in response to the first user’s human face yaw in the first preset angle range, and the first user’s position yaw and When the difference in the yaw of the sound source is within the third preset angle range, the electronic device executes the voice control event corresponding to the voice data.
  • the electronic device can determine that the voice data is paying attention (Looking or gazing) from the user. At this time, the electronic device can directly execute the event corresponding to the aforementioned voice data (ie, voice command).
  • the electronic device may start the voice assistant and directly start the voice assistant when the yaw of the human face is within the first preset angle range, and the difference between the yaw of the position and the yaw of the sound source is within the third preset angle range. Recognize the aforementioned voice data, and execute the voice control event corresponding to the voice data (ie, voice command).
  • the sound source yaw degree is the angle between the connection between the camera and the sound source of the voice data and the first straight line; in response to determining the human face
  • the yaw degree is within the first preset angle range, and the difference between the position yaw degree and the sound source yaw degree is within the third preset angle range, and the electronic device executes the voice control event corresponding to the voice data.
  • the detailed description of the human face yaw degree, the first line, the position yaw degree, and the first straight line described in the second aspect may refer to the description in the first aspect and its possible design methods. This application implements Examples are not repeated here.
  • the electronic device can determine the aforementioned voice data It is sent by a user who is paying attention (looking or staring). At this time, the electronic device can directly execute the event corresponding to the aforementioned voice data (ie, voice command); there is no need to start the voice assistant to recognize the voice data and execute the voice control event corresponding to the voice data after the wake word is recognized.
  • the event corresponding to the aforementioned voice data ie, voice command
  • the method of the embodiment of the present application may further include: responding to determining that the yaw degree of the human face is not within the first preset angle range, or the position yaw The difference between the degree and the yaw degree of the sound source is not within the third preset angle range, and the electronic device recognizes the voice data; in response to determining that the voice data is the preset wake word, the electronic device activates the voice control function of the electronic device. Wherein, after the voice control function is activated, the electronic device executes the corresponding voice control event in response to the voice data collected by the microphone.
  • multiple position parameters and the position yaw corresponding to each position parameter are pre-stored in the electronic device; the position parameters are used to characterize the face image The position in the corresponding picture.
  • the electronic device acquiring the position yaw degree of the first user includes: the electronic device acquires the position parameter of the face image in the first picture; the electronic device searches for the position yaw degree corresponding to the acquired position parameter; and finds it The position yaw degree of is used as the position yaw degree.
  • the method of the embodiment of the present application may further include: in response to determining that the yaw of the human face is within the first preset angle range, the electronic device uses the microphone When the voice data is collected, the voice data from the sound source corresponding to the position yaw degree is enhanced.
  • the microphone array can adjust the filter coefficient of each channel in the desired direction (the position yaw corresponds to the Direction) effectively forms a beam pointing to the target sound source, enhances the signal in the beam and suppresses the signal outside the beam, so as to achieve the purpose of extracting the sound source and suppressing noise at the same time.
  • the electronic device when it collects voice data through a microphone, it can also perform attenuation processing on the voice data emitted by other sound sources.
  • the other orientation may be an orientation whose deviation from the yaw degree of the position is outside the preset angle range (such as the first preset angle range or the third preset angle range).
  • the electronic device can determine that a user is paying attention to the display screen of the electronic device. If a user is paying attention to the display screen, the electronic device can enhance the voice data sent by the user who is paying attention to the display screen (that is, the sound source whose position yaw corresponds to the direction). In this way, the electronic device can specifically collect the voice data sent by the user who pays attention to the display screen.
  • the method of the embodiment of the present application may further include: when the electronic device plays multimedia data, wherein the multimedia data includes audio data, in response to determining the human face The yaw degree is within the first preset angle range, and the electronic device lowers the playback volume of the electronic device.
  • the electronic device can determine that a user is paying attention to the display screen of the electronic device. In the process of playing audio data on the electronic device, if a user is paying attention to the display screen, the user is more likely to control the electronic device through voice commands (ie, voice data). At this time, the electronic device can lower the playback volume of the electronic device and be ready to collect voice commands.
  • voice commands ie, voice data
  • an embodiment of the present application provides an electronic device that includes a processor, a memory, a display screen, and a camera; the memory, the display screen, and the camera are coupled with the processor, and the memory is used to store computer program codes. Including computer instructions.
  • the processor executes the computer instructions, if the display screen is black, the camera is used to collect the first picture; the processor is used to recognize that the first picture includes a face image and obtain the first user's face deviation Navigation, the first user is the user corresponding to the face image in the first picture; the yaw of the first user’s face is the left and right rotation angle of the first user’s face relative to the first connection, the first connection It is the connection between the camera and the head of the first user; in response to determining that the yaw of the first user's face is within the first preset angle range, the display screen is automatically turned on.
  • the above-mentioned processor is configured to automatically light up the display screen in response to determining that the first user's face yaw is within the first preset angle range, including: The above-mentioned processor is configured to automatically light up the display screen in response to determining that the yaw of the first user's face is within the first preset angle range and the first user's eyes are opened.
  • the above-mentioned processor is configured to automatically light up the display screen in response to determining that the yaw of the first user's face is within the first preset angle range, including :
  • the above-mentioned processor is configured to automatically light up the display screen in response to determining that the yaw of the first user's face is within the first preset angle range and the first user's eyes are looking at the display screen.
  • the above-mentioned processor is specifically configured to respond to determining that the yaw degree of the first user's face is within a first preset angle range and that the face of the first user If the duration of the yaw degree within the first preset angle range exceeds the preset time threshold, the display screen is automatically turned on.
  • the above-mentioned processor is also used to obtain the position yaw of the first user before the display screen is automatically turned on.
  • the position yaw of the first user is the camera The angle between the line with the head of the first user and the first line, the first line being perpendicular to the display screen, and the first line passing the camera.
  • the processor is specifically configured to automatically light up the display in response to determining that the yaw degree of the first user's face is within the first preset angle range and the position yaw degree of the first user is within the second preset angle range Screen.
  • the above-mentioned processor is further configured to issue an alarm indication in response to determining that the position yaw of the first user is not within the second preset angle range.
  • the above-mentioned processor is further configured to perform face recognition on the first user before automatically lighting the display screen.
  • the foregoing processor is configured to automatically light up the display screen in response to determining that the yaw of the first user's face is within the first preset angle range, and includes: the foregoing processor is configured to respond to determining the first user's If the face yaw is within the first preset angle range, and the first user's face recognition passes, the display screen is automatically turned on.
  • the aforementioned camera is also used to collect a second picture after the processor automatically lights up the display screen.
  • the processor is also used to identify whether the second picture includes a human face image; in response to determining that the second picture does not include a human face image, the screen is automatically blacked out.
  • the above-mentioned processor is further configured to obtain the face yaw of the second user in response to determining that the second picture includes a face image, and the second user is the first user. 2.
  • the user corresponding to the face image in the picture; the face yaw of the second user is the left and right rotation angle of the second user’s face with respect to the second connection.
  • the second connection s camera and the second user’s head
  • the screen will be blacked out automatically.
  • the above electronic device further includes a microphone.
  • Microphone is used to collect voice data.
  • the processor is also used to obtain the sound source yaw degree of the voice data.
  • the sound source yaw degree is the angle between the connection between the camera and the sound source of the voice data and the first straight line; If the navigation is within the first preset angle range, and the difference between the position yaw of the first user and the sound source yaw is within the third preset angle, the voice control event corresponding to the voice data is executed.
  • the above-mentioned processor is further configured to respond to determining that the yaw of the first user's face is not within the first preset angle range, or the position of the first user is deviated If the difference between the navigation degree and the sound source yaw degree is not within the third preset angle range, the voice data is recognized; in response to determining that the voice data is the preset wake-up word, the voice control function of the electronic device is activated.
  • the processor is also used to start the voice control function and execute the corresponding voice control event in response to the voice data collected by the microphone.
  • multiple position parameters and the position yaw corresponding to each position parameter are pre-stored in the above-mentioned memory; the position parameters are used to characterize the face image in the corresponding picture s position.
  • the above-mentioned processor is configured to obtain the position yaw degree of the first user, including: a processor, configured to obtain the position parameter of the face image of the first user in the first picture; and find the position corresponding to the obtained position parameter Yaw degree; and the found position yaw degree is used as the position yaw degree of the first user.
  • the above-mentioned processor is further configured to respond to determining that the yaw of the first user's face is within the first preset angle range, when the voice data is collected through the microphone , The voice data from the sound source corresponding to the azimuth of the position yaw of the first user is enhanced.
  • the above-mentioned electronic device may further include a multimedia playing module.
  • the above-mentioned processor is also used for the multimedia playback module to play multimedia data, the multimedia data needs to include audio data, and in response to determining that the first user's face yaw is within the first preset angle range, the multimedia playback module is lowered Playback volume.
  • an embodiment of the present application provides an electronic device that includes a processor, a memory, a display screen, a camera, and a microphone; the memory, the display screen, and the camera are coupled to the processor, and the memory is used to store computer program codes.
  • the program code includes computer instructions.
  • the camera is used to collect the first picture;
  • the microphone is used to collect voice data;
  • the processor is used to recognize that the first picture includes a face image and obtain the corresponding face image
  • the user’s face yaw of the user, and the user’s position yaw is obtained;
  • the human face yaw is the left and right rotation angle of the user’s face relative to the first link, which is the camera and the user’s head
  • the position yaw is the angle between the connection between the camera and the user’s head and the first straight line, the first straight line is perpendicular to the display screen, and the first straight line passes through the camera;
  • the sound source for obtaining voice data Yaw degree, the sound source yaw degree is the angle between the connection between the camera and the sound source of the voice data and the first straight line; in response to determining that the human face yaw degree is within the first preset angle range, and the position yaw If the difference between the degree and the ya
  • the above-mentioned processor is also used for determining that the human face yaw degree is not within the first preset angle range, or the position yaw degree and the sound source yaw degree If the difference is not within the third preset angle range, the voice data is recognized; in response to determining that the voice data is the preset wake-up word, the voice control function of the electronic device is activated.
  • the processor is also used to start the voice control function and execute the corresponding voice control event in response to the voice data collected by the microphone.
  • the above-mentioned processor pre-stores multiple position parameters, and the position yaw corresponding to each position parameter; the position parameter is used to characterize the face image in the corresponding picture In the location.
  • the processor is used to obtain the position yaw of the user, including: the processor is used to obtain the position parameter of the face image in the first picture; search for the position yaw corresponding to the obtained position parameter, and search
  • the yaw degree of the arrived position is regarded as the position yaw degree.
  • the above-mentioned processor is further configured to respond to determining that the yaw degree of the human face is within the first preset angle range, when the voice data is collected through the microphone, the position deviation The voice data from the sound source with the corresponding azimuth of the voyage is enhanced.
  • the above electronic device further includes a multimedia playing module.
  • the processor is also used for when the multimedia playback module plays multimedia data, the multimedia data includes audio data, and in response to determining that the first user's face yaw is within the first preset angle range, the playback of the multimedia playback module is lowered volume.
  • an embodiment of the present application provides a computer storage medium that includes computer instructions, which when the computer instructions are executed on an electronic device, cause the electronic device to perform operations such as the first aspect or the second aspect and The method described in any of its possible design methods.
  • the embodiments of the present application provide a computer program product, when the computer program product is run on a computer, the computer is caused to execute as described in the first aspect or the second aspect and any of its possible design methods. The method described.
  • the electronic equipment described in the third aspect, the fourth aspect and any one of their possible design methods provided above, the computer storage medium described in the fifth aspect, and the computer program product described in the sixth aspect are all It is used to implement the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.
  • FIG. 1 is a schematic diagram of an example scenario where a screen control method provided by an embodiment of the application is applied;
  • FIG. 2 is a schematic diagram of an example of a display screen and a camera provided by an embodiment of the application;
  • FIG. 3 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of a camera imaging principle provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of another camera imaging principle provided by an embodiment of this application.
  • FIG. 6 is a schematic diagram of a voice control scenario provided by an embodiment of the application.
  • FIG. 7 is a schematic diagram of a position yaw degree and a sound source yaw degree provided by an embodiment of the application;
  • FIG. 8A is a schematic diagram of another voice control scenario provided by an embodiment of this application.
  • 8B is a logical block diagram of the interaction principle of each module in an electronic device provided by an embodiment of the application.
  • 9A is a schematic diagram of the relationship between the included angle ⁇ and the position parameter x according to an embodiment of the application.
  • 9B is a schematic diagram of another principle of the relationship between the included angle ⁇ and the position parameter x provided by an embodiment of the application;
  • 9C is a schematic diagram of another principle of the relationship between the included angle ⁇ and the position parameter x according to an embodiment of the application.
  • FIG. 10 is a schematic diagram of an example of the relationship between the included angle ⁇ and the position parameter x provided by an embodiment of the application;
  • FIG. 11 is a schematic diagram of an example of the relationship between the included angle ⁇ and the position parameter x provided by an embodiment of the application;
  • FIG. 12 is a schematic diagram of the principle of a method for calculating a position parameter x according to an embodiment of the application.
  • FIG. 13 is a schematic diagram of the principle of another method for calculating a position parameter x provided by an embodiment of the application.
  • the embodiment of the present application provides a screen control method, which can be applied in the process of automatically lighting the display screen of an electronic device.
  • the electronic device includes a display screen and a camera, and the electronic device can detect whether a user is paying attention to the display screen through the camera (for example, the user is watching or staring at the display screen). If a user is paying attention to the display screen, the electronic device can automatically light up the display screen. For example, as shown in (a) of Figure 1, when a user pays attention to the display screen, the display screen of the electronic device is lit. Among them, when a user pays attention to the display screen, the display screen is more likely to be used or viewed. At this time, automatically lighting the display screen of the electronic device can reduce the possibility of the display screen being illuminated by mistake, reduce the waste of energy consumption of the electronic device, and improve the intelligence of interaction.
  • the electronic device After the display screen of the electronic device is lit, if no user pays attention to the display screen of the electronic device within a preset time, the electronic device can automatically black out the screen. For example, as shown in (b) of Fig. 1, when no user pays attention to the display screen, the display screen of the electronic device is black.
  • the camera is set above the display screen.
  • the camera 201 may be arranged on the upper frame of the display screen 200.
  • the camera can be set in another position of the electronic device, as long as the electronic device can accurately detect whether a user is paying attention to the display screen through the camera.
  • the electronic devices in the embodiments of the present application may be household devices such as smart speakers, smart TVs, refrigerators, washing machines, air conditioners, air purifiers, kitchen and bathroom products, including display screens and camera modules.
  • the electronic device in the embodiment of the present application may also be a portable computer (such as a mobile phone), a tablet computer, a desktop computer, a laptop, a handheld computer, a notebook computer, and an ultra-mobile personal computer including a display screen and a camera module ( ultra-mobile personal computer (UMPC), netbooks, and equipment such as cellular phones, personal digital assistants (PDA), augmented reality (AR) ⁇ virtual reality (VR) devices, media players, etc.
  • PDA personal digital assistants
  • AR augmented reality
  • VR virtual reality
  • the embodiment of the application does not impose special restrictions on the specific form of the electronic device.
  • FIG. 3 shows a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display 194, And subscriber identification module (subscriber identification module, SIM) card interface 195 and so on.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and the environment Light sensor 180L, bone conduction sensor 180M, sound sensor and many other sensors.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100.
  • the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc.
  • AP application processor
  • modem processor modem processor
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the different processing units may be independent devices or integrated in one or more processors.
  • the controller may be the nerve center and command center of the electronic device 100.
  • the controller can generate operation control signals according to the instruction operation code and timing signals, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (PCM) interface, and a universal asynchronous transmitter (universal asynchronous transmitter) interface.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART mobile industry processor interface
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB Universal Serial Bus
  • the I2C interface is a two-way synchronous serial bus, which includes a serial data line (SDL) and a serial clock line (SCL).
  • the processor 110 may include multiple sets of I2C buses.
  • the processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the electronic device 100.
  • the I2S interface can be used for audio communication.
  • the processor 110 may include multiple sets of I2S buses.
  • the processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170.
  • the audio module 170 may transmit audio signals to the wireless communication module 160 through an I2S interface, so as to realize the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communication to sample, quantize and encode analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • the UART interface is generally used to connect the processor 110 and the wireless communication module 160.
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function.
  • the audio module 170 may transmit audio signals to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with the display screen 194, the camera 193 and other peripheral devices.
  • the MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on.
  • the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100.
  • the processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and so on.
  • GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. This interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely illustrative, and does not constitute a structural limitation of the electronic device 100.
  • the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger can be a wireless charger or a wired charger.
  • the charging management module 140 may receive the charging input of the wired charger through the USB interface 130.
  • the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.
  • the power management module 141 is used for connecting the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance).
  • the power management module 141 may also be provided in the processor 110.
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
  • the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100.
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.
  • the mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic wave radiation via the antenna 1.
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low-frequency baseband signal is processed by the baseband processor and then passed to the application processor.
  • the application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194.
  • the modem processor may be an independent device.
  • the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (FM), NFC, infrared technology (infrared, IR) and other wireless communication solutions.
  • WLAN wireless local area networks
  • Bluetooth blue, BT
  • global navigation satellite system global navigation satellite system
  • FM frequency modulation
  • NFC infrared technology
  • IR infrared technology
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
  • the wireless communication module 160 may also receive the signal to be sent from the processor 110, perform frequency modulation, amplify, and convert it into electromagnetic waves to radiate through the antenna 2.
  • the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite-based augmentation systems
  • the electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor.
  • the GPU is a microprocessor for image processing, connected to the display 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • the processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos, etc.
  • the display screen 194 includes a display panel.
  • the display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc.
  • the electronic device 100 may include one or N display screens 194, and N is a positive integer greater than one.
  • the electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
  • the ISP is mainly used to process the data fed back from the camera 193. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transfers the electrical signal to the ISP for processing and is converted into an image visible to the naked eye.
  • ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193.
  • the camera 193 is used to capture still images or videos.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats.
  • the electronic device 100 may include one or N cameras 193, and N is a positive integer greater than one.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in a variety of encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • MPEG moving picture experts group
  • NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
  • the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
  • the internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.
  • UFS universal flash storage
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal.
  • the audio module 170 can also be used to encode and decode audio signals.
  • the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.
  • the speaker 170A also called a "speaker” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
  • the receiver 170B also called a "handset", is used to convert audio electrical signals into sound signals.
  • the button 190 includes a power button, a volume button, and so on.
  • the button 190 may be a mechanical button. It can also be a touch button.
  • the electronic device 100 may receive key input, and generate key signal input related to user settings and function control of the electronic device 100.
  • the motor 191 can generate vibration prompts.
  • the motor 191 can be used for incoming call vibration notification, and can also be used for touch vibration feedback.
  • touch operations that act on different applications can correspond to different vibration feedback effects.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.
  • the screen control method provided by the embodiment of the present application may be implemented in the above electronic device 100.
  • the electronic device 100 includes a display screen and a camera.
  • the camera is used to capture images.
  • the image collected by the camera is used for the electronic device 100 to detect whether a user is paying attention to the display screen.
  • the display screen is used to display images generated by the processor of the electronic device 100 or images from other devices.
  • the embodiment of the present application provides a screen control method.
  • This screen control method can be applied to the process in which the electronic device 100 automatically lights up the display screen when the display screen of the electronic device 100 is black.
  • the display screen is in sleep mode or power saving mode; in the embodiments of the present invention, the electronic device black screen is a black screen when the display screen is powered on and the switch is turned on, that is, the display screen can display but no display content.
  • the electronic device 100 can collect the first picture through a camera.
  • the electronic device 100 recognizes that the first picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image.
  • the electronic device 100 may automatically light up the display screen.
  • the human face yaw is the deviation angle between the user's face orientation and the "line between the camera and the user's head" (that is, the first line).
  • the human face yaw degree may also be the left-right rotation angle of the user's face orientation relative to the first line.
  • the connection between the camera and the user's head may be a connection between the camera and any organ of the user's head (such as the nose or mouth).
  • FIG. 4 Take user A as an example.
  • a human face yaw degree [alpha] is the angle between the user A and X A O A O P O A's.
  • O P O B is the connection with the camera head of the user B
  • X B O B represents a B face towards the user.
  • Human face user F is yaw angle of [alpha] F and X F O F O P O F a.
  • the value range of human face yaw is [-90°, 90°].
  • the value range of the yaw degree of the face is [-90°, 0°).
  • the face orientation of user A rotates to the left relative to the connection between the camera and the user's head, and the angle of the left rotation is ⁇ A , ⁇ A ⁇ [-90°, 0°).
  • the face orientation of the user D rotates to the left relative to the connection between the camera and the user's head, and the angle of the left rotation is ⁇ D , ⁇ D ⁇ [-90°, 0°).
  • the value range of the yaw degree of the human face is (0°, 90°).
  • the face of user B rotates to the right relative to the connection between the camera and the head of user B, and the angle of rotation to the right is ⁇ B , ⁇ B ⁇ (0°, 90°].
  • the face of user E rotates to the right relative to the connection between the camera and the head of user E, and the angle of rotation to the right is ⁇ E , ⁇ E ⁇ (0°, 90°].
  • the face orientation of the user F rotates to the right relative to the line connecting the camera and the head of the user F, and the angle of the right rotation is ⁇ F , ⁇ F ⁇ (0°, 90°].
  • the foregoing first preset angle range may be an angle range with a value around 0°.
  • the first preset angle range may be [-n°, n°].
  • the value range of n can be (0, 10) or (0, 5).
  • the electronic device 100 can automatically light up the display screen.
  • the electronic device 100 can automatically light up the display screen of the electronic device 100 when the display screen is more likely to be used or viewed. In this way, the possibility of the display screen being illuminated by mistake can be reduced, and the waste of energy consumption of the electronic device can be reduced.
  • the method for the electronic device 100 to recognize whether a face image is included in the first picture can refer to the specific method for recognizing a face image in the conventional technology, and the examples of this application will not be repeated here.
  • the electronic device 100 may acquire the facial features of the facial image in the first picture by means of facial detection.
  • the facial features may include the aforementioned yaw degree of the human face.
  • the face feature may also include face position information (faceRect), face feature point information (landmarks), and face pose information.
  • face posture information may include a pitch angle (pitch) of a face, an in-plane rotation angle (roll), and a yaw degree (that is, a left-right rotation angle, yaw).
  • the electronic device 100 may provide an interface (such as a Face Detector interface), and the interface may receive the first picture taken by the camera. Then, the processor (such as the NPU) of the electronic device 100 can perform face detection on the first picture to obtain the aforementioned face features. Finally, the electronic device 100 may return the detection result (JSON Object), that is, the aforementioned facial features.
  • an interface such as a Face Detector interface
  • the processor such as the NPU
  • the electronic device 100 may return the detection result (JSON Object), that is, the aforementioned facial features.
  • JSON detection result
  • one picture (such as the first picture) may include one or more face images.
  • the electronic device 100 may allocate the one or more different IDs of the face images to identify the face images.
  • the electronic device 100 can determine that the user is more likely to pay attention to (looking or staring) the display screen, and the electronic device 100 can automatically light up Display screen.
  • the electronic device 100 can also determine whether the user's eyes are open. For example, the electronic device 100 may determine whether at least one eye of the user is open. In response to determining that the yaw degree of the human face is within the first preset angle range and at least one eye of the user is opened, the electronic device 100 may automatically light up the display screen. It can be understood that if the aforementioned human face yaw is within the first preset angle range and at least one eye of the user is open, it means that the user is paying attention to the display screen. At this time, the electronic device 100 can automatically light up the display screen.
  • the electronic device 100 does not light up the display screen. In this way, the possibility of the display screen being illuminated by mistake can be reduced, the waste of energy consumption of the electronic device can be reduced, and the intelligence of the interaction can be improved.
  • the electronic device 100 may determine whether the user’s eyes are open by the following method: when the electronic device 100 detects the user’s face, it determines whether the camera has collected the user’s iris information; if the camera has collected the iris information, the electronic The device 100 determines that the user's eyes are open; if the camera does not collect iris information, the electronic device 100 determines that the user's eyes are not open.
  • the electronic device 100 may determine whether the user’s eyes are open by the following method: when the electronic device 100 detects the user’s face, it determines whether the camera has collected the user’s iris information; if the camera has collected the iris information, the electronic The device 100 determines that the user's eyes are open; if the camera does not collect iris information, the electronic device 100 determines that the user's eyes are not open.
  • other existing technologies can also be used to detect whether the eyes are open.
  • the electronic device 100 can also determine whether the user's eyes are looking at the display screen. In response to determining that the yaw of the human face is within the first preset angle range and the user's eyes are looking at the display screen, the electronic device 100 may automatically light up the display screen. It can be understood that if the aforementioned human face yaw is within the first preset angle range and the user's eyes are looking at the display screen, it means that the user is paying attention to the display screen. At this time, the electronic device 100 can automatically light up the display screen.
  • the electronic device 100 does not light up the display screen. In this way, the possibility of the display screen being illuminated by mistake can be reduced, the waste of energy consumption of the electronic device can be reduced, and the intelligence of the interaction can be improved.
  • the method for the electronic device 100 to determine whether the user's eyes are looking at the display screen can refer to conventional techniques, for example, by determining the position relationship between the user's pupil and the display screen; or by using an eye tracker. The method of judging whether the user's eyes are looking at the display screen is not described in detail in the embodiment of the present application.
  • the electronic device 100 may also determine whether the duration of the human face yaw degree within the first preset angle range exceeds a preset time threshold. If the duration of the human face yaw within the first preset angle range does not exceed the preset time threshold, it means that the user is not paying attention to the display. It may just be that the user is facing the display when turning or The surface yaw degree is within the first preset angle range. In this case, the electronic device 100 will not light up the display screen.
  • the duration of the human face yaw degree within the first preset angle range exceeds the preset time threshold, it means that the user is paying attention to the display screen, and the electronic device 100 can automatically light up the display screen. Therefore, the accuracy of judgment can be improved, and the intelligence of interaction can be improved.
  • the electronic device 100 may continue to collect pictures (such as the second picture) through the camera.
  • the electronic device 100 recognizes that the second picture does not include a human face image, it will automatically black out the screen.
  • the electronic device 100 recognizes that the second picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image. If the yaw degree of the human face is not within the first preset angle range, the electronic device 100 may automatically black out the screen. If the yaw of the human face is within the first preset angle range, the electronic device 100 may continue to brighten the screen.
  • the second picture does not include a human face image, it means that no user is paying attention (looking or gazing at) the display screen. If the second picture includes a face image, but the yaw of the user's face corresponding to the face image is not within the first preset angle range, it means that the user's face orientation is relative to the line between the camera and the user's head The rotation angle of is larger, and the possibility that the user is paying attention (looking or staring) the display screen is lower.
  • the electronic device 100 may have a black screen (that is, enter a sleep mode or a power saving mode). In this way, the waste of energy consumption of the electronic device 100 can be reduced.
  • the above method can also be applied to the process in which the electronic device 100 automatically lights up the display screen when the display screen of the electronic device 100 is in the screen saver state.
  • the display screen being in the screen saver state means that the electronic device 100 executes the screen saver program and displays the screen saver picture on the display screen.
  • the screen brightness of the display screen is relatively dark, which can reduce the energy consumption of the electronic device.
  • the display screen is in the screen saver state.
  • the display screen is also in sleep mode or power saving mode.
  • Voice assistants are an important application of electronic devices (such as the aforementioned electronic device 100).
  • the voice assistant can conduct intelligent interactions with users in intelligent dialogue and instant question and answer.
  • the voice assistant can also recognize the user's voice command and make the mobile phone execute the event corresponding to the voice command.
  • the display screen 101 of the electronic device 100 is black; or, as shown in FIG. 6(b), the electronic device 100 displays a photo.
  • the voice assistant of the electronic device 100 is in a sleep state.
  • the electronic device 100 can monitor voice data. When the voice data (such as the wake-up word "small E, small E”) is monitored, it can be judged whether the voice data matches the wake-up word.
  • the electronic device 100 can start the voice assistant, and the display screen 101 displays the voice recognition interface shown in (c) in FIG. 6.
  • the electronic device 100 may receive a voice command input by the user (such as "play music"), and then execute an event corresponding to the voice command (such as increasing the volume of the electronic device 100).
  • a voice command input by the user such as "play music”
  • an event corresponding to the voice command such as increasing the volume of the electronic device 100.
  • the user needs to send voice data (including voice data matching the wake-up word and voice commands) at least twice before controlling the electronic device 100 to execute the corresponding voice control event.
  • the electronic device 100 cannot directly execute the voice control event corresponding to the voice command according to the voice command.
  • the electronic device 100 when the display screen is black or the display screen is on, the electronic device 100 does not need to receive and match the wake-up word, and can directly execute the corresponding voice command according to the voice command. event. It should be noted that the voice assistant can also be in a sleep state when the display is on.
  • the electronic device 100 may collect the first picture through a camera.
  • the electronic device 100 recognizes that the first picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image.
  • the electronic device 100 obtains the position yaw degree of the user corresponding to the face image.
  • the electronic device 100 collects voice data, and obtains the yaw degree of the sound source of the voice data.
  • the electronic device 100 executes the aforementioned voice data (That is, voice command) corresponding to the voice control event.
  • the aforementioned voice data is not a wake-up word preset in the electronic device 100; rather, it is used to control the electronic device 100 to execute a voice command corresponding to a voice control event.
  • the preset wake-up word in the electronic device 100 is "small E, small E”
  • the above voice data may be voice commands such as "play music” or "turn up the volume”.
  • the voice command "play music” is used to control the electronic device 100 to play music (that is, a voice control event).
  • the voice command "turn up the volume” is used to control the electronic device 100 to turn up the volume (that is, a voice control event).
  • the position yaw of the user mentioned above is the angle between the "line between the camera and the user's head" and the first straight line.
  • the yaw degree of the sound source of the voice data is the angle between the "line of the camera and the sound source of the voice data" and the first straight line.
  • the first line e.g., in FIG. 7 (a) and FIG. 7 (b shown) and the O P O Q Y-axis direction in FIGS. 9A-9C
  • the first line is perpendicular to the display screen, and the first Pass the camera in a straight line.
  • the first straight line as shown in FIG. 7 is a O P O Q.
  • O P O Q perpendicular to the display screen, and O P O Q through the point where the camera O P.
  • the connection with the camera user's head in FIG. 7 for the location of the yaw O P O A, user A is O P O A O P O Q and angle ⁇ a.
  • the connection between the camera and the user’s head is O P O S
  • the yaw of the sound source S of the voice data is the angle between O P O S and O P O Q ⁇ '.
  • the method for the electronic device 100 to obtain the yaw degree of the sound source of the voice data may refer to the method for obtaining the yaw degree of the sound source of the voice data in the conventional technology, which will not be repeated here in the embodiment of the application.
  • the third preset angle range may be an angle range that takes a value around 0°.
  • the third preset angle range may be [-p°, p°].
  • the value range of p can be (0, 5) or (0, 3).
  • the voice data is highly likely to be the voice uttered by the user.
  • the yaw of the human face is within the first preset angle range, it means that the user is more likely to be paying attention (looking or staring) to the display screen.
  • the electronic device 100 can determine the voice data It is sent by a user who is paying attention (looking or staring). At this time, the electronic device 100 can directly execute the event corresponding to the aforementioned voice data (ie, voice command). For example, the electronic device 100 may start the voice assistant and start the voice assistant when the yaw of the human face is within the first preset angle range, and the difference between the yaw of the position and the sound source yaw is within the third preset angle. Directly recognize the above voice data, and execute the voice control event corresponding to the voice data (ie, voice command).
  • the display screen 101 of the electronic device 100 is black; or, as shown in (b) of FIG. 8A, the display screen 101 of the electronic device 100 is bright.
  • the electronic device 100 (such as the DSP of the electronic device 100) can monitor voice data. Assume that the electronic device 100 detects any voice data, such as "playing music".
  • the electronic device 100 determines that a user is paying attention to the display screen (that is, the yaw degree of the human face is within the first preset angle range), the position yaw degree of the user who is paying attention to the display screen and the voice data (such as "playing music") If the difference in the yaw of the sound source is within the third preset angle range, the electronic device 100 can determine that the above-mentioned voice data is sent by the user who is paying attention (looking or gazing). The electronic device 100 can directly play music. After the electronic device detects the voice data, it can perform semantic analysis first.
  • the electronic device After determining the effective voice command, determine whether the human face yaw is within the first preset angle range, and the position yaw and the sound of the detected voice data. Whether the difference of the source yaw degree is within the third preset angle range, if all are within the preset range, the action corresponding to the voice data is directly executed; after the electronic device detects the voice data, it can also judge the yaw of the human face first Whether the degree is within the first preset angle range, and whether the difference between the position yaw degree and the sound source yaw degree of the detected voice data is within the third preset angle range, if both are within the preset range, Then perform semantic analysis and perform specific operations corresponding to the voice data.
  • the display screen 101 of the electronic device 100 is black. If the electronic device 100 determines that a user is paying attention to the display screen, the difference between the yaw degree of the position of the user paying attention to the display screen and the yaw degree of the sound source of the voice data (such as "playing music") is within the third preset angle range , The electronic device 100 can also light up the display screen.
  • the human face yaw degree, the first preset angle range, and the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the value range of the position yaw of the user relative to the camera (or the display screen) is [-FOV, FOV].
  • FOV field of view
  • the value range of the position yaw ⁇ of the user relative to the camera (or display screen) is (0°, FOV).
  • the user a with respect to the position of the camera (or display) a degree of yaw O P O a O P O Q and angle ⁇ a . ⁇ a is in the range of (0 °, FOV].
  • the user's yaw relative to the camera (or display screen) position is 0°.
  • the electronic device 100 can directly execute the event corresponding to the voice data.
  • the user may be outside the field of view (ie, FOV) of the camera of the electronic device 100.
  • FOV field of view
  • the aforementioned first image does not include the face image of the user.
  • the user wants to control the electronic device 100 to execute the corresponding event through voice data (ie, voice commands), he still needs to issue the above-mentioned wake-up words (such as "small E, small E") to wake up the voice assistant of the electronic device 100 , And then send a voice command to the electronic device 100 (such as "turn up the volume").
  • voice data ie, voice commands
  • FIG. 8B shows a logical block diagram of the interaction principle of each module in the electronic device 100 provided in an embodiment of the present application.
  • the “sound collection” module 801 of the electronic device 100 can collect voice data (such as voice data 1), and deliver the collected voice data 1 to the “wake-up engine” 802.
  • the "wake-up engine” 802 (such as AP) determines whether the voice data 1 matches the wake-up word (such as "small E, small E").
  • the "wake-up engine” 802 determines that the voice data 1 matches the wake-up word, the "wake-up engine” 802 will send the voice data (such as voice data 2) subsequently collected by the "sound collection” module 801 to the "voice recognition” module 803.
  • the "voice recognition” module 803 performs voice recognition (such as semantic analysis, etc.) on the voice data 2, and then the electronic device 100 executes the event corresponding to the voice data 2.
  • the "sound collection” module 801 can collect voice data (such as voice data 3).
  • the “sound collection” module 801 can send the collected voice data 3 to the “no wake-up engine” 807.
  • the “sound source localization” module 805 can also perform sound source localization on the voice data 3 to obtain the yaw degree of the sound source of the voice data 3.
  • the “sound source localization” module 805 can send the sound source yaw degree of the voice data 3 to the “wake-up-free engine” 807.
  • the "following display screen” module 804 of the electronic device 100 determines that a user is following the display screen
  • the “following person positioning” module 806 can locate the user following the display screen to obtain the position offset of the user following the display screen. Voyage. Then, the “person of interest positioning” module 806 may send the acquired position yaw to the “no wake-up engine” 807.
  • the “wake-up-free engine” 807 may send voice data 3 to the “voice recognition” module 803 when the difference between the position yaw degree and the sound source yaw degree is within the third preset angle range.
  • the “voice recognition” module 803 performs voice recognition (such as semantic analysis, etc.) on the voice data 3, and then the electronic device 100 executes an event corresponding to the voice data 3.
  • the electronic device 100 can recognize the voice collected by the electronic device 100 Data 3, and directly execute the event corresponding to the voice data 3. Through the method of the embodiment of the present application, the electronic device 100 can realize voice interaction with the user without wake-up words.
  • the aforementioned “sound collection” module 801 may be a sound sensor of the electronic device 100.
  • the sound sensor can collect voice data around the electronic device 100.
  • the aforementioned "focus on display screen” module 804 may include a camera. Part of the functions of the “focus on the display screen” module 804 may be integrated in the processor of the electronic device 100.
  • the above-mentioned “wake-up engine” 802, “wake-up-free engine” 807, “voice recognition” module 803, “sound source localization” module 805, “focused person localization” module 806, etc. may be integrated in the processor of the electronic device 100.
  • the functions of the aforementioned “wake-up engine” 802 and “wake-up-free engine” 807 can be implemented in the DSP of the electronic device 100.
  • Part of the functions of the “Focus on Display Screen” module 804 may be implemented in the NPU of the electronic device 100.
  • the method for the electronic device 100 to obtain the position yaw of the user is described here.
  • the camera of the electronic device 100 may include the sensor and lens shown in FIG. 9A.
  • the vertical distance between the sensor and the lens is d.
  • O X of the sensor Taking the center O X of the sensor as the origin of the coordinates, the horizontal line passing through O X is the x-axis, and the vertical line passing O X is the y-axis.
  • O P is the center point of the lens.
  • the point K A is an imaging point of the user A on the sensor of the camera (for example, the pixel point where the nose tip of the face image in the first picture a is located).
  • the first picture a is a picture taken by a camera.
  • the first picture a includes a face image of user A.
  • the coordinate point of K A in the above-mentioned coordinate system is (-x a , 0).
  • the length of O X K A is x a .
  • x a d ⁇ tan(f c ( ⁇ a )).
  • the unit of x a may be a pixel.
  • the length of the above O X K A is x a, which can be specifically: the distance between the point O X and the point K A by x a pixels.
  • ⁇ f c ( ⁇ ).
  • FIG. 9B the user B is located in the point O B (front camera).
  • the position yaw degree ⁇ b of the user B relative to the camera is 0°.
  • the user C is located at the O C point (front left of the camera).
  • the position yaw degree of the user C relative to the camera is ⁇ c .
  • x c d ⁇ tan( ⁇ c ).
  • the length of the above O X K C being x c can be specifically: the distance between the point O X and the point K C is x c pixels.
  • the corresponding x can be obtained by adjusting the position of the user relative to the camera to ⁇ and assigning different values.
  • FIG. 10 shows a correspondence table of x and ⁇ provided by an embodiment of the present application.
  • FIG. 11 an example of a correspondence table between x and ⁇ provided in this embodiment of the application.
  • the electronic device 100 may obtain the position parameter x of the face image in the first picture, and then find the position yaw degree ⁇ corresponding to x.
  • the electronic device 100 obtains x a shown in FIG. 9 as an example, and describes the method for the electronic device 100 to obtain the position parameter x of the user's face image in the first picture:
  • the electronic device 100 may obtain the face feature information of the face image in the first picture (such as the first picture a) by means of face detection.
  • the facial feature information can include the coordinates of the left eye center position (1235, 1745), the right eye center position coordinates (1752, 1700), the nose position coordinates (1487, 2055), and the left mouth corner position coordinates ( 1314, 2357) and the coordinates of the right corner of the mouth (1774, 2321) and so on.
  • the coordinates of each position in the face position information are in a coordinate system with the upper left corner of the first picture as the coordinate origin O.
  • x a can be the vertical line between the center line L of the first picture a in the x-axis direction and the nose position coordinates (1487, 2055).
  • the face position information may include: the height of the face image (for example, the above "height”: 1795, which means that the height of the face image is 1795 pixels); the width of the face image (for example, , The above "width”: 1496, which means that the height of the face image is 1496 pixels); the distance between the face image and the left boundary of the first picture (for example, the above "left”: 761, which means that the face image and the first picture The distance between the left border is 761 pixels); the distance between the face image and the upper border of the first picture (for example, the above "top”: 1033 means that the distance between the face image and the upper border of the first picture is 1033 pixels) .
  • the length of the first picture a in the horizontal direction is r
  • the electronic device 100 may collect the first picture through a camera.
  • the electronic device 100 recognizes that the first picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image.
  • the electronic device 100 obtains the position yaw degree of the user corresponding to the face image.
  • the electronic device 100 in response to determining that the human face yaw degree is within the first preset angle range and the position yaw degree is not within the second preset angle range, the electronic device 100 will not light up the screen .
  • the electronic device 100 may automatically black out the screen.
  • the foregoing third preset range may be [-m°, m°].
  • the value range of m can be [40, 60], or the value range of m can be [45, 65], etc.
  • the human face yaw degree, the first preset angle range, and the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the electronic device 100 can determine that a user is paying attention to the display screen of the electronic device 100. If a user is paying attention to the display screen, then the user may be the owner or a user who has the owner's consent to operate or view the electronic device 100; or, the user may also be a user who wants to operate or view the electronic device 100 without the owner's consent .
  • the owner of the electronic device 100 or a user approved by the owner will be located directly in front of the electronic device 100 or at a position closer to the front of the electronic device 100 when operating or viewing the electronic device 100.
  • the position yaw of such users is within the second preset angle range.
  • the position yaw degree is not within the second preset angle range, it means that the user who pays attention to the display screen is on both sides of the electronic device 100 and is in a relatively remote position from the front of the electronic device 100.
  • the user may not be the owner of the electronic device 100, or the user may be operating or viewing the electronic device 100 without the owner's consent.
  • the user may be triggering the electronic device 100 to light up the display screen through the method of the embodiment of the present application; or, the user may be stealing the content displayed on the display screen of the electronic device 100.
  • the electronic device 100 is currently black, the electronic device 100 will not light up the screen; if the electronic device 100 is currently on the screen, the electronic device 100 can automatically black out the screen. In this way, the data stored in the electronic device 100 can be protected from being stolen.
  • the electronic device 100 when the electronic device 100 has a black screen or a bright screen, in response to determining that the human face yaw degree is within the first preset angle range and the position yaw degree is not within the second preset angle range, the electronic device 100 You can also send out an alarm. , And the display status does not change, and the screen remains black or bright.
  • the electronic device 100 may issue a voice alarm prompt.
  • the electronic device 100 may emit a "dididi" prompt; or, the electronic device 100 may emit a voice prompt of "safety alert, safety alert!.
  • the electronic device 100 may issue a vibration alarm prompt.
  • the embodiment of the application does not limit this.
  • the electronic device 100 may click Turn on the screen.
  • the electronic device 100 may continue to brighten the screen in response to determining that the human face yaw degree is within the first preset angle range and the position yaw degree is within the second preset angle range.
  • the electronic device 100 may collect the first picture through a camera.
  • the electronic device 100 recognizes that the first picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image.
  • the electronic device 100 can perform face recognition on the user.
  • the electronic device 100 has a black screen, in response to determining that the yaw degree of the human face is within the first preset angle range and the face recognition fails, the electronic device 100 will not light up the screen.
  • the screen of the electronic device 100 is on, in response to determining that the yaw degree of the human face is within the first preset angle range and the face recognition fails, the electronic device 100 may automatically black out the screen.
  • the human face yaw degree, the first preset angle range, and the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the electronic device 100 obtains the user's face yaw degree corresponding to the face image to determine that the human face yaw degree is within the first preset angle range
  • the electronic device 100 can determine that a user is paying attention to the display screen of the electronic device 100. If a user is following the display screen and the face recognition fails, it means that the user following the display screen is not an authorized user. At this time, if the electronic device 100 is currently black, the electronic device 100 will not light up the screen; if the electronic device 100 is currently on the screen, the electronic device 100 can automatically black out the screen. In this way, the data stored in the electronic device 100 can be protected from being stolen.
  • the electronic device 100 may also issue an alarm prompt.
  • the specific method for the electronic device 100 to issue an alarm prompt can refer to the description in the foregoing embodiment, which is not repeated in the embodiment of the present application.
  • the electronic device 100 may collect the first picture through a camera, and collect voice data through one or more (for example, a microphone array) microphone, where the one or more microphones may be equipped on the electronic device, It can also be independent of the electronic device but connected to the electronic device.
  • the electronic device 100 recognizes that the first picture includes a face image.
  • the electronic device 100 obtains the yaw degree of the user's face corresponding to the face image.
  • the electronic device 100 obtains the position yaw degree of the user corresponding to the face image.
  • the electronic device 100 In response to determining that the human face yaw degree is within the first preset angle range, when the electronic device 100 collects voice data through a microphone, it performs enhancement processing on the voice data emitted by the sound source corresponding to the position yaw degree. Further, when the electronic device 100 collects voice data through a microphone, it may also perform attenuation processing on the voice data emitted by other sound sources.
  • the other orientation may be an orientation whose deviation from the yaw degree of the position is outside the preset angle range (such as the first preset angle range or the third preset angle range).
  • the electronic device 100 can determine that a user is paying attention to the display screen of the electronic device 100. If a user is paying attention to the display screen, the electronic device 100 may perform enhancement processing on the voice data sent by the user paying attention to the display screen (that is, the sound source whose position yaw corresponds to the azimuth). In this way, the electronic device 100 can specifically collect the voice data sent by the user who pays attention to the display screen.
  • the method of the embodiment of the present application may be applied to a process in which the electronic device 100 plays audio data.
  • the electronic device 100 may not be able to accurately collect the voice commands (ie, voice data) issued by the user because the volume of the audio data played by the electronic device 100 is high.
  • the method of the embodiment of the present application may include: the electronic device 100 collects the first picture through a camera. The electronic device 100 recognizes that the first picture includes a face image. The electronic device 100 obtains the yaw degree of the user's face corresponding to the face image. In response to determining that the human face yaw degree is within the first preset angle range, the electronic device 100 lowers the playback volume of the electronic device 100.
  • the electronic device 100 can determine that a user is paying attention to the display screen of the electronic device 100.
  • the electronic device 100 is playing audio data, if a user is paying attention to the display screen, the user is more likely to control the electronic device 100 through voice commands (ie, voice data).
  • voice commands ie, voice data
  • the electronic device 100 when the electronic device 100 is playing audio data, if a user is paying attention to the display screen, the electronic device 100 can not only lower the playback volume of the electronic device 100, but also prepare for the collection of voice commands to improve the collection of the electronic device 100. The accuracy of voice data.
  • the electronic device 100 may also perform enhancement processing on the voice data emitted by the sound source corresponding to the azimuth of the user's position yaw when collecting voice data through a microphone. In this way, the electronic device 100 can specifically collect the voice data of the user who pays attention to the display screen.
  • Another embodiment of the present application further provides an electronic device, which may include a processor, a memory, a display screen, a microphone, and a camera.
  • the memory, display screen, camera, and microphone are coupled with the processor.
  • the memory is used to store computer program codes.
  • the computer program codes include computer instructions.
  • the processor executes the computer instructions, the electronic device can execute various functions or steps performed by the electronic device 100 in the above method embodiments.
  • the structure of the electronic device can refer to the structure of the electronic device 100 shown in FIG. 3.
  • the aforementioned camera is used to collect pictures.
  • the camera can capture the first picture when the display is black.
  • the foregoing processor is configured to recognize that the first picture includes a face image, and obtain the face yaw of the first user; in response to determining that the face yaw of the first user is within the first preset angle range, then The display automatically lights up.
  • the first user is a user corresponding to the face image in the first picture.
  • the yaw of the face of the first user is the left-right rotation angle of the first user's face orientation relative to the first connection, and the first connection is the connection between the camera and the first user's head.
  • the above-mentioned processor is further configured to, in response to determining that the yaw degree of the first user's face is within the first preset angle range, and the first user's eyes are opened, automatically light the Description display screen.
  • the above-mentioned processor is further configured to respond to determining that the yaw of the first user's face is within the first preset angle range and the first user's eyes are looking at the display screen, then The display is automatically lit.
  • the aforementioned processor is further configured to respond to determining that the face yaw of the first user is within the first preset angle range, and the face yaw of the first user is within the first If the duration within a preset angle range exceeds the preset time threshold, the display screen is automatically turned on.
  • the above-mentioned processor is further configured to obtain the position yaw degree of the first user before automatically illuminating the display screen, and the position yaw degree of the first user is the difference between the camera and the first user.
  • the aforementioned microphone is used to collect voice data.
  • the above-mentioned processor is also used to obtain the sound source yaw degree of the voice data.
  • the sound source yaw degree is the angle between the connection line between the camera and the sound source of the voice data and the first straight line; in response to the human face of the first user If the yaw degree is within the first preset angle range, and the difference between the position yaw degree of the first user and the sound source yaw degree is within the third preset angle range, the voice control event corresponding to the voice data is executed.
  • the aforementioned processor is further configured to respond to determining that the yaw of the first user's face is not within the first preset angle range, or the difference between the yaw of the position of the first user and the yaw of the sound source is not Within the third preset angle range, the voice data is recognized; in response to determining that the voice data is the preset wake-up word, the voice control function of the electronic device is activated.
  • the processor is also used to start the voice control function and execute the corresponding voice control event in response to the voice data collected by the microphone.
  • the processor is further configured to, in response to determining that the yaw of the human face is within the first preset angle range, when the voice data is collected by the microphone, the voice data emitted by the sound source corresponding to the yaw of the position is enhanced deal with.
  • the electronic device also includes a multimedia playing module (such as a speaker).
  • a multimedia playing module (such as a speaker).
  • the above-mentioned processor is also used for when the multimedia playing module plays multimedia data, the multimedia data includes audio data, and in response to determining that the first user's face yaw is within the first preset angle range, the multimedia playing module is lowered Play volume.
  • the functions of the processor, memory, display screen, microphone, and camera of the electronic device include but are not limited to the above functions.
  • functions of the processor, memory, display screen, microphone, and camera of the electronic device reference may be made to the various functions or steps performed by the electronic device 100 in the foregoing method embodiment, which will not be repeated in the embodiment of the present application.
  • the computer storage medium includes computer instructions.
  • the computer instructions run on an electronic device, the electronic device executes the functions performed by the electronic device 100 in the foregoing method embodiment. Or steps.
  • Another embodiment of the present application provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute various functions or steps performed by the electronic device 100 in the foregoing method embodiments.
  • the disclosed system, device, and method can be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be other division methods for example, multiple units or components may be Combined or can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • the aforementioned storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.

Abstract

本申请实施例提供一种屏幕控制和语音控制方法及电子设备，涉及电子技术领域，可以在电子设备的显示屏被使用或者被查看的可能性较高时，才自动点亮电子设备的显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费。具体方案包括：显示屏黑屏时，电子设备通过摄像头采集第一图片；电子设备识别到第一图片中包括人脸图像，获取第一用户的人面偏航度；响应于确定第一用户的人面偏航度在第一预设角度范围内，则电子设备自动点亮显示屏。该第一用户是第一图片中的人脸图像对应的用户；第一用户的人面偏航度是第一用户的面部朝向相对于第一连线的左右旋转角度，第一连线是摄像头与第一用户的头部的连线。

Description

一种屏幕控制和语音控制方法及电子设备
本申请要求在2019年1月25日提交中国国家知识产权局、申请号为201910075866.1、发明名称为“一种屏幕控制和语音控制方法及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及电子技术领域，尤其涉及一种屏幕控制和语音控制方法及电子设备。
背景技术
随着显示屏技术的发展，越来越多的电子设备上配置显示屏，以显示电子设备的各项参数或者音视频资讯。上述显示屏可以为触摸屏。例如，无论是冰箱、洗衣机、空调等大型家居设备，还是音箱、空气净化器、厨卫用品等小型家居设备都可以配置显示屏。该显示屏可以显示对应家居设备的运行参数、家庭监控、时钟日历、数码相册和新闻资讯等一项或多项内容。
目前，显示屏一般为常亮或者响应于用户对物理按键或者显示屏(如触摸屏)的操作点亮。但是，显示屏常亮会增加家居设备的能耗，造成不必要的能源损耗。并且，显示屏常亮还会加快对显示屏的损耗，缩短显示屏的使用寿命。而响应于用户的操作点亮显示屏会增加家居设备所需要的时间，影响用户体验。
在一些方案中，家居设备上可以配置一个传感器。当该传感器检测到有用户与家居设备的距离小于预设距离阈值时，点亮家居设备的显示屏。但是，即使用户与家居设备的距离小于预设距离阈值，该用户也不一定想要使用显示屏或者查看显示屏显示的内容。如此，则可能会导致显示屏的误点亮。
发明内容
本申请实施例提供一种屏幕控制和语音控制方法及电子设备，可以在电子设备的显示屏被使用或者被查看的可能性较高时，才自动点亮电子设备的显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费。
本申请采用如下技术方案：
第一方面，本申请实施例提供一种屏幕控制方法，该屏幕控制方法可以应用于电子设备。该电子设备包括显示屏和摄像头。该屏幕控制方法可以包括：显示屏黑屏时，电子设备通过摄像头采集第一图片；电子设备识别到第一图片中包括人脸图像，获取第一用户的人面偏航度；响应于确定第一用户的人面偏航度在第一预设角度范围内，则电子设备自动点亮显示屏。其中，该第一用户是第一图片中的人脸图像对应的用户；第一用户的人面偏航度是第一用户的面部朝向相对于第一连线的左右旋转角度，第一连线是摄像头与第一用户的头部的连线。
可以理解，如果第一用户的人面偏航度在第一预设角度范围内，则表示第一用户的面部朝向相对于第一连线的旋转角度较小。此时，第一用户在关注(在看或者凝视)显示屏的可能性较高，电子设备可以自动点亮显示屏。换言之，电子设备可以在显示屏被使用或者查看的可能性较高时，自动点亮电子设备的显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费。
结合第一方面，在一种可能的设计方式中，上述响应于确定第一用户的人面偏航度在第一预设角度范围内，电子设备自动点亮显示屏，包括：响应于确定第一用户的人面偏航度在 第一预设角度范围内，且第一用户的眼睛睁开，电子设备自动点亮显示屏。
其中，如果第一用户的人面偏航度在第一预设角度范围内，且第一用户的至少一个眼睛睁开，则表示第一用户在关注显示屏。此时，电子设备可以自动点亮显示屏。当然，即使第一用户的人面偏航度在第一预设角度范围内，但是如果第一用户两个眼睛均未睁开(即用户闭合双眼)，则表示第一用户并没有在关注显示屏。此时，电子设备则不点亮显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费，同时提高交互的智能性。
结合第一方面，在另一种可能的设计方式中，上述响应于确定第一用户的人面偏航度在第一预设角度范围内，电子设备自动点亮显示屏，包括：响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的眼睛看向显示屏，电子设备自动点亮显示屏。
可以理解，如果第一用户的人面偏航度在第一预设角度范围内，且第一用户的眼睛看向显示屏，则表示用户在关注显示屏。此时，电子设备可以自动点亮显示屏。当然，即使上述人面偏航度在第一预设角度范围内，但是如果用户两个眼睛没有看向显示屏，则表示用户并没有在关注显示屏。此时，电子设备则不点亮显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费，同时提高交互的智能性。
结合第一方面，在另一种可能的设计方式中，上述响应于确定第一用户的人面偏航度在第一预设角度范围内，电子设备自动点亮显示屏，包括：响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的人面偏航度在第一预设角度范围内的持续时间超过预设时间阈值，电子设备自动点亮显示屏。
其中，如果人面偏航度在第一预设角度范围内的持续时间未超过预设时间阈值，则表示用户没有关注显示屏，可能只是在转身或者转头时用户正好面向显示屏，使用户的人面偏航度在第一预设角度范围内。在这种情况下，电子设备不会点亮显示屏。如果人面偏航度在第一预设角度范围内的持续时间超过预设时间阈值，则表示用户在关注显示屏，电子设备可以自动点亮显示屏。因此可以提高判断的准确度，提高交互的智能性。
结合第一方面，在另一种可能的设计方式中，在电子设备自动点亮显示屏之前，电子设备还可获取第一用户的位置偏航度，第一用户的位置偏航度是摄像头与第一用户的头部的连线和第一直线的夹角，第一直线垂直于显示屏，且第一直线经过摄像头。上述响应于确定第一用户的人面偏航度在第一预设角度范围内，电子设备自动点亮显示屏，包括：响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的位置偏航度在第二预设角度范围内，电子设备自动点亮显示屏。
其中，如果位置偏航度不在第二预设角度范围内，则表示关注显示屏的用户在电子设备的两侧，距离电子设备的正前方较为偏远的方位。在这种情况下，该用户可能不是电子设备的机主，或者该用户没有经过机主同意在操作或者查看电子设备。例如，该用户可能是在通过本申请实施例的方法触发电子设备点亮显示屏；或者，该用户可能是在窃取电子设备的显示屏所显示的内容。在这种情况下，如果电子设备当前黑屏，电子设备则不会点亮屏幕；如果电子设备当前亮屏，电子设备则可以自动黑屏。这样，可以保护电子设备中保存的数据不被窃取。
结合第一方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：响应于确定第一用户的位置偏航度不在第二预设角度范围内，电子设备发出报警指示。该报警指示可以提醒机主有其他用户正在关注显示屏。
结合第一方面，在另一种可能的设计方式中，在电子设备自动点亮显示屏之前，本申请 实施例的方法还可以包括：电子设备对第一用户进行人脸识别。上述响应于确定第一用户的人面偏航度在第一预设角度范围内，电子设备自动点亮显示屏，包括：响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的人脸识别通过，电子设备自动点亮显示屏。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备可以确定有用户在关注电子设备的显示屏。如果有用户在关注显示屏，且人脸识别未通过，则表示关注显示屏的用户不是经过授权的用户。此时，如果电子设备当前黑屏，电子设备则不会点亮屏幕；如果电子设备当前亮屏，电子设备则可以自动黑屏。这样，可以保护电子设备中保存的数据不被窃取。
结合第一方面，在另一种可能的设计方式中，在电子设备自动点亮显示屏之后，本申请实施例的方法还可以包括：电子设备通过摄像头采集第二图片；电子设备识别第二图片中是否包括人脸图像；响应于确定第二图片中不包括人脸图像，电子设备自动黑屏。这样，可以减少对电子设备的能耗的浪费。
结合第一方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：响应于确定第二图片中包括人脸图像，电子设备获取第二用户的人面偏航度，第二用户是第二图片中的人脸图像对应的用户；第二用户的人面偏航度是第二用户的面部朝向相对于第二连线的左右旋转角度，第二连线的摄像头与第二用户的头部的连线；响应于确定第二用户的人面偏航度不在第一预设角度范围内，电子设备自动黑屏。这样，可以减少对电子设备的能耗的浪费。
结合第一方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：电子设备通过麦克风采集语音数据；电子设备获取语音数据的声源偏航度，声源偏航度是摄像头与语音数据的声源的连线和第一直线的夹角；响应于第一用户的人面偏航度在第一预设角度范围内，且第一用户的位置偏航度和声源偏航度的差值在第三预设角度范围内，电子设备执行语音数据对应的语音控制事件。
可以理解，如果用户的位置偏航度和语音数据的声源偏航度的差值在第三预设角度范围内，则表示该语音数据是该用户发出的语音的可能性很高。如果上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内，那么电子设备可以确定上述语音数据是正在关注(在看或者凝视)的用户发出的。此时，电子设备可以直接执行上述语音数据(即语音命令)对应的事件。例如，电子设备可以在上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内时，启动语音助手并直接识别上述语音数据，并执行该语音数据(即语音命令)对应的语音控制事件。
第二方面，本申请实施例提供一种语音控制方法，该语音控制方法可以应用于电子设备。该电子设备包括麦克风、显示屏和摄像头。该语音控制方法可以包括：电子设备通过摄像头采集第一图片，通过麦克风采集语音数据；电子设备识别到第一图片中包括人脸图像，获取人脸图像对应的用户的人面偏航度，并获取用户的位置偏航度；电子设备获取语音数据的声源偏航度，声源偏航度是摄像头与语音数据的声源的连线和第一直线的夹角；响应于确定人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内，电子设备执行语音数据对应的语音控制事件。
其中，第二方面中所述的人面偏航度、第一连线、位置偏航度和第一直线的详细描述可以参考第一方面及其可能的设计方式中的描述，本申请实施例这里不予赘述。
可以理解，如果上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的 差值在第三预设角度范围内，那么电子设备可以确定上述语音数据是正在关注(在看或者凝视)的用户发出的。此时，电子设备可以直接执行上述语音数据(即语音命令)对应的事件；而不需要在识别到唤醒词之后，再启动语音助手识别语音数据执行语音数据对应的语音控制事件。
结合第一方面或第二方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：响应于确定人面偏航度不在第一预设角度范围内，或者位置偏航度和声源偏航度的差值不在第三预设角度范围内，电子设备识别语音数据；响应于确定语音数据是预设唤醒词，电子设备启动电子设备的语音控制功能。其中，语音控制功能启动后，电子设备响应于麦克风采集的语音数据执行对应的语音控制事件。
结合第一方面或第二方面，在另一种可能的设计方式中，电子设备中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；位置参数用于表征人脸图像在对应图片中的位置。上述电子设备获取第一用户的位置偏航度，包括：电子设备获取人脸图像在第一图片中的位置参数；电子设备查找与获取到的位置参数对应的位置偏航度；并将查找到的位置偏航度作为位置偏航度。
结合第一方面或第二方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：响应于确定人面偏航度在第一预设角度范围内，电子设备通过麦克风采集语音数据时，对位置偏航度对应方位的声源发出的语音数据进行增强处理。例如，通过将多个麦克风按照一定规律组成一个麦克风阵列,当语音和环境信息被多个麦克风收集时,麦克风阵列可以通过调节每个通道的滤波系数,在期望方向上(该位置偏航度对应方向)有效地形成一个指向目标声源的波束,对这个波束内的信号进行增强,波束外的信号进行抑制,从而达到同时提取声源和抑制噪声的目的。
进一步的，电子设备通过麦克风采集语音数据时，还可以对其他方位的声源发出的语音数据进行衰减处理。上述其他方位可以为与上述位置偏航度的偏差在预设角度范围(如上述第一预设角度范围或者第三预设角度范围)之外的方位。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备可以确定有用户在关注电子设备的显示屏。如果有用户在关注显示屏，电子设备则可以对关注显示屏的用户(即上述位置偏航度对应方位的声源)发出的语音数据进行增强处理。这样，电子设备便可以针对性采集关注显示屏的用户发出的语音数据。
结合第一方面或第二方面，在另一种可能的设计方式中，本申请实施例的方法还可以包括：电子设备播放多媒体数据时，其中所述多媒体数据包含音频数据，响应于确定人面偏航度在第一预设角度范围内，电子设备则调低电子设备的播放音量。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备可以确定有用户在关注电子设备的显示屏。在电子设备播放音频数据的过程中，如果有用户在关注显示屏，那么该用户通过语音命令(即语音数据)控制电子设备的可能性较高。此时，电子设备可以调低电子设备的播放音量，做好采集语音命令的准备。
第三方面，本申请实施例提供一种电子设备，该电子设备包括处理器、存储器、显示屏和摄像头；存储器、显示屏和摄像头与处理器耦合，存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当处理器执行计算机指令时，如果显示屏黑屏，摄像头，用于采集第一图片；处理器，用于识别到第一图片中包括人脸图像，获取第一用户的人面偏航度，第一用户是第一图片中的人脸图像对应的用户；第一用户的人面偏航度是第一用户的面部朝 向相对于第一连线的左右旋转角度，第一连线是摄像头与第一用户的头部的连线；响应于确定第一用户的人面偏航度在第一预设角度范围内，则自动点亮显示屏。
结合第三方面，在一种可能的设计方式中，上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，则自动点亮显示屏，包括：上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的眼睛睁开，则自动点亮显示屏。
结合第三方面，在另一种可能的设计方式中，上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，则自动点亮显示屏，包括：上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的眼睛看向显示屏，则自动点亮显示屏。
结合第三方面，在另一种可能的设计方式中，上述处理器，具体用于响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的人面偏航度在第一预设角度范围内的持续时间超过预设时间阈值，则自动点亮显示屏。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于在自动点亮显示屏之前，获取第一用户的位置偏航度，第一用户的位置偏航度是摄像头与第一用户的头部的连线和第一直线的夹角，第一直线垂直于显示屏，且第一直线经过摄像头。处理器，具体用于响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的位置偏航度在第二预设角度范围内，则自动点亮显示屏。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于响应于确定第一用户的位置偏航度不在第二预设角度范围内，则发出报警指示。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于在自动点亮显示屏之前，对第一用户进行人脸识别。上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，则自动点亮显示屏，包括：上述处理器，用于响应于确定第一用户的人面偏航度在第一预设角度范围内，且第一用户的人脸识别通过，则自动点亮显示屏。
结合第三方面，在另一种可能的设计方式中，上述摄像头，还用于在处理器自动点亮显示屏之后，采集第二图片。处理器，还用于识别第二图片中是否包括人脸图像；响应于确定第二图片中不包括人脸图像，则自动黑屏。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于响应于确定第二图片中包括人脸图像，获取第二用户的人面偏航度，第二用户是第二图片中的人脸图像对应的用户；第二用户的人面偏航度是第二用户的面部朝向相对于第二连线的左右旋转角度，第二连线的摄像头与第二用户的头部的连线；响应于确定第二用户的人面偏航度不在第一预设角度范围内，则自动黑屏。
结合第三方面，在另一种可能的设计方式中，上述电子设备还包括麦克风。麦克风，用于采集语音数据。处理器，还用于获取语音数据的声源偏航度，声源偏航度是摄像头与语音数据的声源的连线和第一直线的夹角；响应于第一用户的人面偏航度在第一预设角度范围内，且第一用户的位置偏航度和声源偏航度的差值在第三预设角度范围内，则执行语音数据对应的语音控制事件。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于响应于确定第一用户的人面偏航度不在第一预设角度范围内，或者第一用户的位置偏航度和声源偏航度的差值不在第三预设角度范围内，则识别语音数据；响应于确定语音数据是预设唤醒词，则启动电子设备的语音控制功能。其中，处理器，还用于启动语音控制功能启动后，响应于麦克风采集 的语音数据执行对应的语音控制事件。
结合第三方面，在另一种可能的设计方式中，上述存储器中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；位置参数用于表征人脸图像在对应图片中的位置。上述处理器，用于获取第一用户的位置偏航度，包括：处理器，用于获取第一用户的人脸图像在第一图片中的位置参数；查找与获取到的位置参数对应的位置偏航度；并将查找到的位置偏航度作为第一用户的位置偏航度。
结合第三方面，在另一种可能的设计方式中，上述处理器，还用于响应于确定第一用户的人面偏航度在第一预设角度范围内，则通过麦克风采集语音数据时，对第一用户的位置偏航度对应方位的声源发出的语音数据进行增强处理。
结合第三方面，在另一种可能的设计方式中，上述电子设备还可以包括多媒体播放模块。上述处理器，还用于多媒体播放模块播放多媒体数据时，该多媒体数据需要包含音频数据，响应于确定第一用户的人面偏航度在第一预设角度范围内，则调低多媒体播放模块的播放音量。
第四方面，本申请实施例提供一种电子设备，该电子设备包括处理器、存储器、显示屏、摄像头和麦克风；存储器、显示屏和摄像头与处理器耦合，存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当处理器执行计算机指令时，摄像头用于采集第一图片；麦克风用于采集语音数据；处理器，用于识别到第一图片中包括人脸图像，获取人脸图像对应的用户的人面偏航度，并获取用户的位置偏航度；人面偏航度是用户的面部朝向相对于第一连线的左右旋转角度，第一连线是摄像头与用户的头部的连线；位置偏航度是摄像头与用户的头部的连线和第一直线的夹角，第一直线垂直于显示屏，且第一直线经过摄像头；获取语音数据的声源偏航度，声源偏航度是摄像头与语音数据的声源的连线和第一直线的夹角；响应于确定人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内，则执行语音数据对应的语音控制事件。
结合第四方面，在一种可能的设计方式中，上述处理器，还用于响应于确定人面偏航度不在第一预设角度范围内，或者位置偏航度和声源偏航度的差值不在第三预设角度范围内，则识别语音数据；响应于确定语音数据是预设唤醒词，则启动电子设备的语音控制功能。其中，处理器，还用于启动语音控制功能启动后，响应于麦克风采集的语音数据执行对应的语音控制事件。
结合第四方面，在另一种可能的设计方式中，上述处理器中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；位置参数用于表征人脸图像在对应图片中的位置。处理器，用于获取用户的位置偏航度，包括：处理器，用于获取人脸图像在第一图片中的位置参数；查找与获取到的位置参数对应的位置偏航度，并将查找到的位置偏航度作为位置偏航度。
结合第四方面，在另一种可能的设计方式中，上述处理器，还用于响应于确定人面偏航度在第一预设角度范围内，则通过麦克风采集语音数据时，对位置偏航度对应方位的声源发出的语音数据进行增强处理。
结合第四方面，在另一种可能的设计方式中，上述电子设备还包括多媒体播放模块。处理器，还用于多媒体播放模块播放多媒体数据时，该多媒体数据包含音频数据，响应于确定第一用户的人面偏航度在第一预设角度范围内，则调低多媒体播放模块的播放音量。
第五方面，本申请实施例提供一种计算机存储介质，该计算机存储介质包括计算机指令， 当所述计算机指令在电子设备上运行时，使得所述电子设备执行如第一方面或第二方面及其任一种可能的设计方式所述的方法。
第六方面，本申请实施例提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如第一方面或第二方面及其任一种可能的设计方式所述的方法。
可以理解地，上述提供的第三方面和第四方面及其任一种可能的设计方式所述的电子设备，第五方面所述的计算机存储介质，以及第六方面所述的计算机程序产品均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
附图说明
图1为本申请实施例提供的一种屏幕控制方法所应用的场景实例示意图；
图2为本申请实施例提供的一种显示屏和摄像头的实例示意图；
图3为本申请实施例提供的一种电子设备的硬件结构示意图；
图4为本申请实施例提供的一种摄像头成像原理示意图；
图5为本申请实施例提供的另一种摄像头成像原理示意图；
图6为本申请实施例提供的一种语音控制场景示意图；
图7为本申请实施例提供的一种位置偏航度和声源偏航度的示意图；
图8A为本申请实施例提供的另一种语音控制场景示意图；
图8B为本申请实施例提供的一种电子设备中各个模块的交互原理逻辑框图；
图9A为本申请实施例提供的一种夹角β与位置参数x的关系原理示意图；
图9B为本申请实施例提供的另一种夹角β与位置参数x的关系原理示意图；
图9C为本申请实施例提供的另一种夹角β与位置参数x的关系原理示意图；
图10为本申请实施例提供的一种夹角β与位置参数x的关系实例示意图；
图11为本申请实施例提供的一种夹角β与位置参数x的关系实例示意图；
图12为本申请实施例提供的一种计算位置参数x的方法原理示意图；
图13为本申请实施例提供的另一种计算位置参数x的方法原理示意图。
具体实施方式
本申请实施例提供一种屏幕控制方法，可以应用于电子设备自动点亮显示屏的过程中。具体的，该电子设备包括显示屏和摄像头，电子设备可以通过摄像头检测是否有用户在关注显示屏(如用户在看或者凝视显示屏)。如果有用户在关注显示屏，电子设备则可以自动点亮该显示屏。例如，如图1中的(a)所示，有用户关注显示屏的情况下，电子设备的显示屏被点亮。其中，有用户关注显示屏时，该显示屏被使用或者查看的可能性较高。此时，自动点亮电子设备的显示屏，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费，提高交互的智能性。
在电子设备的显示屏被点亮后，如果预设时间内，没有用户关注电子设备的显示屏，该电子设备则可以自动黑屏。例如，如图1中的(b)所示，没有用户关注显示屏的情况下，电子设备的显示屏黑屏。
需要注意的是，为了使得电子设备可以通过摄像头准确检测到是否有用户在关注显示屏，该摄像头设置在显示屏上方。例如，如图2所示，摄像头201可以设置在显示屏200的上边框上。或者，摄像头可以设置在电子设备的其他位置，只要电子设备通过摄像头可以准确检测到是否有用户在关注显示屏即可。
示例性的，本申请实施例中的电子设备可以为包括显示屏和摄像头模组的智能音箱、智能电视机、冰箱、洗衣机、空调、空气净化器、厨卫用品等家居设备。进一步的，本申请实施例中的电子设备还可以为包括显示屏和摄像头模组的便携式计算机(如手机)、平板电脑、桌面型、膝上型、手持计算机、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本，以及蜂窝电话、个人数字助理(personal digital assistant，PDA)、增强现实(augmented reality，AR)\虚拟现实(virtual reality，VR)设备、媒体播放器等设备，本申请实施例对该电子设备的具体形态不作特殊限制。
请参考图3，其示出本申请实施例提供的一种电子设备100的结构示意图。该电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。
其中，传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M，声音传感器等多个传感器。
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDL)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处 理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。所述I2S接口和所述PCM接口都可以用于音频通信。
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，外部存储器，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电 源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如Wi-Fi网络)，蓝牙(blue tooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，NFC，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处 理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP主要用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行电子设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机 接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。电子设备100可以设置至少一个麦克风170C。在另一些实施例中，电子设备100可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，电子设备100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。
本申请实施例提供的屏幕控制方法可以在上述电子设备100中实现。该电子设备100包括显示屏和摄像头。摄像头用于采集图像。其中，该摄像头采集的图像用于电子设备100检测是否有用户在关注显示屏。该显示屏用于显示电子设备100的处理器生成的图像或者来自其他设备的图像等。
本申请实施例提供一种屏幕控制方法。该屏幕控制方法可以应用于电子设备100的显示屏黑屏时，电子设备100自动点亮显示屏的过程中。其中，电子设备100黑屏时，显示屏处于休眠模式或者省电模式；本发明各实施例中电子设备黑屏是在显示屏上电且开关打开的情况下的黑屏，即显示屏可以显示但没有显示内容。
在该屏幕控制方法中，电子设备100可以通过摄像头采集第一图片。电子设备100识别到第一图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。响应于确定上述人面偏航度在第一预设角度范围内，电子设备100则可以自动点亮显示屏。
其中，人面偏航度是用户的面部朝向和“摄像头与用户头部的连线”(即第一连线)的偏离角度。人面偏航度也可以是用户的面部朝向相对于第一连线的左右旋转角度。例如，摄像头与用户头部的连线可以为摄像头与用户头部的任一器官(如鼻子或者嘴巴等)的连线。
例如，如图4所示，以用户A为例。O PO A是摄像头与用户A头部的连线，X AO A表示用户A的面部朝向。L AO A与用户A的面部朝向所在直线X AO A垂直，η A＝90°。用户A的人面偏航度α A是X AO A与O PO A的夹角。以用户B为例。O PO B是摄像头与用户B头部的连线，X BO B表示用户B的面部朝向。L BO B与用户B的面部朝向所在直线X BO B垂直，η B＝90°。用户B的人面偏航度α B是X BO B与O PO B的夹角。以用户C为例。O PO C是摄像头与用户C头部的连线，X CO C表示用户C的面部朝向。L CO C与用户C的面部朝向所在直线X CO C垂直，η C＝90°。用户C的人面偏航度α C是X CO C与O PO C的夹角。
又例如，如图5所示，以用户D为例。O PO D是摄像头与用户D头部的连线，X DO D表示用户D的面部朝向。L DO D与用户D的面部朝向所在直线X DO D垂直，η D＝90°。用户D的人面偏航度α D是X DO D与O PO D的夹角。以用户E为例。O PO E是摄像头与用户E头部的连线，X EO E表示用户E的面部朝向。L EO E与用户E的面部朝向所在直线X EO E垂直，η E＝90°。用户E的人面偏航度α E是X EO E与O PO E的夹角。以用户F为例。O PO F是摄像头与用户F头部的连线，X FO F表示用户F的面部朝向。L FO F与用户F的面部朝向所在直线X FO F垂直，η F＝90°。用户F的人面偏航度α F是X FO F与O PO F的夹角。
一般而言，人面偏航度的取值范围为[-90°，90°]。其中，如果用户的面部朝向相对于摄像头与用户头部之间的连线向左旋转(即向左偏离)，人面偏航度的取值范围为[-90°，0°)。例如，如图4所示，用户A的面部朝向相对于摄像头与用户头部的连线向左旋转，且向左旋转的角度为α A，α A∈[-90°，0°)。又例如，如图5所示，用户D的面部朝向相对于摄像头与用户头部的连线向左旋转，且向左旋转的角度为α D，α D∈[-90°，0°)。
如果用户的面部朝向相对于摄像头与用户头部之间的连线向右旋转(即向右偏离)，人面偏航度的取值范围为(0°，90°]。例如，如图4所示，用户B的面部朝向相对于摄像头与用户B头部的连线向右旋转，且向右边旋转的角度为α B，α B∈(0°，90°]。又例如，如图5所示，用户E的面部朝向相对于摄像头与用户E头部的连线向右旋转，且向右旋转的角度为α E，α E∈(0°，90°]。又例如，如图5所示，用户F的面部朝向相对于摄像头与用户F头部的连线向右旋转，且向右旋转的角度为α F，α F∈(0°，90°]。
参考图4和图5可知：人面偏航度越接近于0°，用户关注显示屏的可能性越高。例如，如图4所示，用户C的人面偏航度α C＝0°，用户A的人面偏航度α A和用户B的人面偏航度α B的人面偏航度接近于0°。因此，图4所示的用户A、用户B和用户C关注显示屏的可能性很高。
参考图4和图5可知：人面偏航度的绝对值越大，用户关注显示屏的可能性越低。例如，用户D的人面偏航度α D的绝对值、用户E的人面偏航度α E的绝对值，以及用户F的人面偏航度α F的绝对值均较大。因此，图4所示的用户D、用户E和用户F关注显示屏的可能性较低。
由上述描述可知：上述第一预设角度范围可以为在0°左右取值的角度范围。示例性的， 第一预设角度范围可以为[-n°，n°]。例如，n的取值范围可以为(0，10)或者(0，5)等。例如，n＝2，或者n＝1，或者n＝3等。
可以理解，如果人面偏航度在第一预设角度范围内，则表示用户的面部朝向相对于摄像头与用户头部之间的连线的旋转角度较小。此时，用户在关注(在看或者凝视)显示屏的可能性较高，电子设备100可以自动点亮显示屏。换言之，电子设备100可以在显示屏被使用或者查看的可能性较高时，自动点亮电子设备100的显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费。
需要说明的是，电子设备100识别第一图片中是否包括人脸图像的方法可以参考常规技术中，识别人脸图像的具体方法，本申请实例这里不予赘述。
示例性的，电子设备100可以通过人脸检测的方式获取第一图片中的人脸图像的人脸特征。该人脸特征可以包括上述人面偏航度。具体的，该人脸特征还可以包括人脸位置信息(faceRect)、人脸特征点信息(landmarks)和人脸姿态信息。该人脸姿态信息可以包括人面俯仰角度(pitch)、平面内旋转角度(roll)和人面偏航度(即左右旋转角度，yaw)。
其中，电子设备100可以提供一个接口(如Face Detector接口)，该接口可以接收摄像头拍摄的第一图片。然后，电子设备100的处理器(如NPU)可以对第一图片进行人脸检测，得到上述人脸特征。最后，电子设备100可以返回检测结果(JSON Object)，即上述人脸特征。
例如，以下为本申请实施例中，电子设备100返回的检测结果(JSON)示例。
Figure PCTCN2020072610-appb-000001
其中，上述代码中，“"id":0”表示上述人脸特征对应的人脸ID为0。其中，一张图片(如第一图片)中可以包括一个或多个人脸图像。电子设备100可以分配该一个或多个人脸图像不同的ID，以标识人脸图像。
“"height":1795”表示人脸图像(即人脸图像在第一图片中所在的人脸区域)的高度为1795个像素点。“"left":761”表示人脸图像与第一图片左边界的距离为761个像素点。 “"top":1033”表示人脸图像与第一图片上边界的距离为1033个像素点。“"width":1496”表示人脸图像的宽度为1496个像素点。“"pitch":-2.9191732”表示人脸ID为0的人脸图像的人面俯仰角度为-2.9191732°。“"roll":2.732926”表示人脸ID为0的人脸图像的平面内旋转角度为2.732926°。
“"yaw":0.44898167”表示人脸ID为0的人脸图像的人面偏航度(即左右旋转角度)α＝0.44898167°。由α＝0.44898167°，0.44898167°>0°可知，用户的面部朝向相对于摄像头与该用户头部的连线向右旋转0.44898167°。假设上述n＝2，即上述第一预设角度范围为[-2°，2°]。由于α＝0.44898167°，且0.44898167°∈[-2°，2°]；因此，电子设备100可以确定用户在关注(在看或者凝视)显示屏的可能性较高，电子设备100可以自动点亮显示屏。
在另一实施例中，电子设备100还可以判断用户的眼睛是否睁开。例如，电子设备100可以判断用户的至少一个眼睛是否睁开。响应于确定上述人面偏航度在第一预设角度范围内，且用户的至少一个眼睛睁开，电子设备100则可以自动点亮显示屏。可以理解，如果上述人面偏航度在第一预设角度范围内，且用户的至少一个眼睛睁开，则表示用户在关注显示屏。此时，电子设备100可以自动点亮显示屏。当然，即使上述人面偏航度在第一预设角度范围内，但是如果用户两个眼睛均未睁开(即用户闭合双眼)，则表示用户并没有在关注显示屏。此时，电子设备100则不点亮显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费，同时提高交互的智能性。
示例性的，电子设备100可以通过以下方法判断用户的眼睛是否睁开：电子设备100在对上述用户进行人脸检测时，判断摄像头是否采集到用户的虹膜信息；如果摄像头采集到虹膜信息，电子设备100则确定用户的眼睛睁开；如果摄像头没有采集到虹膜信息，电子设备100则确定用户的眼睛没有睁开。当然，还可以采用其他已有技术进行眼睛是否睁开的检测。
在另一实施例中，电子设备100还可以判断用户的眼睛是否看向显示屏。响应于确定上述人面偏航度在第一预设角度范围内，且用户的眼睛看向显示屏，电子设备100则可以自动点亮显示屏。可以理解，如果上述人面偏航度在第一预设角度范围内，且用户的眼睛看向显示屏，则表示用户在关注显示屏。此时，电子设备100可以自动点亮显示屏。当然，即使上述人面偏航度在第一预设角度范围内，但是如果用户两个眼睛没有看向显示屏，则表示用户并没有在关注显示屏。此时，电子设备100则不点亮显示屏。这样，可以减少显示屏被误点亮的可能性，减少对电子设备的能耗的浪费，同时提高交互的智能性。
需要说明的是，电子设备100判断用户的眼睛是否看向显示屏的方法可以参考常规技术中，例如通过判断用户瞳孔与显示屏的位置关系来实现；或者使用眼动仪来实现等。判断用户的眼睛是否看向显示屏的方法，本申请实施例这里不予赘述。
进一步的，响应于确定上述人面偏航度在第一预设角度范围内，电子设备100还可以判断人面偏航度在第一预设角度范围内的持续时间是否超过预设时间阈值。如果人面偏航度在第一预设角度范围内的持续时间未超过预设时间阈值，则表示用户没有关注显示屏，可能只是在转身或者转头时用户正好面向显示屏，使用户的人面偏航度在第一预设角度范围内。在这种情况下，电子设备100不会点亮显示屏。如果人面偏航度在第一预设角度范围内的持续时间超过预设时间阈值，则表示用户在关注显示屏，电子设备100可以自动点亮显示屏。因此可以提高判断的准确度，提高交互的智能性。
在另一实施例中，电子设备100点亮显示屏之后，电子设备100可以继续通过摄像头采集图片(如第二图片)。
在一种情况下，电子设备100识别到第二图片中不包括人脸图像，则自动黑屏。
在另一种情况下，电子设备100识别到第二图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。如果该人面偏航度不在第一预设角度范围内，电子设备100则可以自动黑屏。如果该人面偏航度在第一预设角度范围内，电子设备100可以继续亮屏。
可以理解，如果第二图片中不包括人脸图像，则表示没有用户关注(在看或者凝视)显示屏。如果第二图片中包括人脸图像，但人脸图像对应的用户的人面偏航度不在第一预设角度范围内，则表示用户的面部朝向相对于摄像头与用户头部之间的连线的旋转角度较大，用户在关注(在看或者凝视)显示屏的可能性较低。此时，电子设备100可以黑屏(即进入休眠模式或者省电模式)。这样，可以减少对电子设备100的能耗的浪费。
在一些实施例中，上述方法还可以应用于电子设备100的显示屏处于屏保状态时，电子设备100自动点亮显示屏的过程中。其中，显示屏处于屏保状态是指：电子设备100执行屏幕保护程序，在显示屏显示屏幕保护画面。其中，显示屏显示屏幕保护画面时，显示屏的屏幕亮度比较暗，可以降低电子设备的能耗。显示屏处于屏保状态显示屏也处于休眠模式或者省电模式。
语音助手是电子设备(如上述电子设备100)的一项重要应用。语音助手可以与用户进行智能对话和即时问答的智能交互。并且，语音助手还可以识别用户的语音命令，并使手机执行该语音命令对应的事件。例如，如图6中的(a)所示，电子设备100的显示屏101黑屏；或者，如图6中的(b)所示，电子设备100显示照片。此时，电子设备100的语音助手处于休眠状态。电子设备100可以监测语音数据。当监测到语音数据(如唤醒词“小E，小E”)时，可以判断该语音数据与唤醒词是否匹配。如果该语音数据与唤醒词匹配，电子设备100可以开启语音助手，显示屏101显示图6中的(c)所示的语音识别界面。此时，电子设备100可以接收用户输入的语音命令(如“播放音乐”)，然后执行语音命令对应的事件(如调高电子设备100的音量)。在上述语音控制的过程中，用户需要发出至少两次语音数据(包括：与唤醒词匹配的语音数据，以及语音命令)，才可以控制电子设备100执行对应的语音控制事件。电子设备100不能根据语音命令直接执行该语音命令对应的语音控制事件。
基于此，本申请实施例提供的方法中，在显示屏黑屏或者显示屏亮屏的情况下，电子设备100不需要接收并匹配唤醒词，只根据语音命令，便可以直接执行该语音命令对应的事件。需要注意的是，显示屏亮屏的情况下，语音助手也可以处于休眠状态。
具体的，电子设备100可以通过摄像头采集第一图片。电子设备100识别到第一图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。电子设备100获取人脸图像对应的用户的位置偏航度。电子设备100采集语音数据，并获取语音数据的声源偏航度。响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内，电子设备100则执行上述语音数据(即语音命令)对应的语音控制事件。
需要注意的是，上述语音数据不是电子设备100中预设的唤醒词；而是用于控制电子设备100执行对应语音控制事件的语音命令。例如，假设电子设备100中预设的唤醒词是“小E，小E”，那么上述语音数据则可以为“播放音乐”或者“调高音量”等语音命令。语音命令“播放音乐”用于控制电子设备100播放音乐(即语音控制事件)。语音命令“调高音量”用于控制电子设备100调高音量(即语音控制事件)。
其中，上述用户的位置偏航度是“摄像头与用户头部的连线”与第一直线的夹角。上述 语音数据的声源偏航度是“摄像头与语音数据的声源的连线”与第一直线的夹角。上述第一直线(例如，图7中的(a)和图7中的(b)所示的O PO Q以及图9A-9C中的Y轴方向)垂直于显示屏，且该第一直线经过摄像头。
例如，如图7中的(a)或者图7中的(b)所示，第一直线是O PO Q。O PO Q垂直于显示屏，且O PO Q经过摄像头所在点O P。如图7中的(a)所示，摄像头与用户头部的连线为O PO A，用户A的位置偏航度是O PO A与O PO Q的夹角β a。如图7中的(b)所示，摄像头与用户头部的连线为O PO S，语音数据的声源S的声源偏航度是O PO S与O PO Q的夹角β＇。
需要说明的是，电子设备100获取用户的位置偏航度的方法可以参考本申请实施例后续相关描述。电子设备100获取语音数据的声源偏航度的方法可以参考常规技术中，获取语音数据的声源偏航度的方法，本申请实施例这里不予赘述。
参考上述图7中的(a)和图7中的(b)可知：位置偏航度β a与声源偏航度β＇的差值越接近于0°，上述语音数据是用户A发出的语音的可能性越高。位置偏航度β a与声源偏航度β＇的差值的绝对值越大，上述语音数据是用户A发出的语音的可能性越低。
由上述描述可知：上述第三预设角度范围可以为在0°左右取值的角度范围。示例性的，第三预设角度范围可以为[-p°，p°]。例如，p的取值范围可以为(0，5)或者(0，3)等。例如，p＝2，或者p＝4，或者p＝3等。
可以理解，如果用户的位置偏航度和语音数据的声源偏航度的差值在第三预设角度范围内，则表示该语音数据是该用户发出的语音的可能性很高。并且，由上述实施例可知：如果人面偏航度在第一预设角度范围内，则表示用户在关注(在看或者凝视)显示屏的可能性较高。因此，如果上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内，那么电子设备100可以确定上述语音数据是正在关注(在看或者凝视)的用户发出的。此时，电子设备100可以直接执行上述语音数据(即语音命令)对应的事件。例如，电子设备100可以在上述人面偏航度在第一预设角度范围内，且位置偏航度和声源偏航度的差值在第三预设角度范围内时，启动语音助手并直接识别上述语音数据，并执行该语音数据(即语音命令)对应的语音控制事件。
例如，如图8A中的(a)所示，电子设备100的显示屏101黑屏；或者，如图8A中的(b)所示，电子设备100的显示屏101亮屏。电子设备100(如电子设备100的DSP)可以监测语音数据。假设电子设备100监测到任一语音数据，例如“播放音乐”。并且，电子设备100确定有用户在关注显示屏(即上述人面偏航度在第一预设角度范围内)，关注显示屏的用户的位置偏航度和上述语音数据(如“播放音乐”)的声源偏航度的差值在第三预设角度范围内，电子设备100则可以确定上述语音数据是正在关注(在看或者凝视)的用户发出的。电子设备100则可以直接播放音乐。电子设备检测到语音数据后，可以先进行语义分析，确定有效语音指令后，确定人面偏航度是否在第一预设角度范围内，以及位置偏航度和该检测到的语音数据的声源偏航度的差值是否在第三预设角度范围内，如果都在预设范围内，则直接执行语音数据对应的动作；电子设备检测到语音数据后，也可以先判断人面偏航度是否在第一预设角度范围内，以及位置偏航度和该检测到的语音数据的声源偏航度的差值是否在第三预设角度范围内，如果都在预设范围内，则进行语义分析，并执行该语音数据对应的具体操作。
可选的，如图8A中的(a)所示，电子设备100的显示屏101黑屏。如果电子设备100确定有用户在关注显示屏，关注显示屏的用户的位置偏航度和上述语音数据(如“播放音乐”) 的声源偏航度的差值在第三预设角度范围内，电子设备100还可以点亮显示屏。
需要说明的是，人面偏航度、第一预设角度范围，以及电子设备100获取人脸图像对应的用户的人面偏航度，确定人面偏航度在第一预设角度范围内的详细描述，可以参考上述实例中的相关描述，本申请实施例这里不予赘述。
一般而言，用户相对于摄像头(或者显示屏)的位置偏航度的取值范围为[-FOV，FOV]。其中，摄像头的视场角(field of view，FOV)的大小决定摄像头的视野范围。
可以理解，如果用户相对于摄像头(或者显示屏)在上述第一直线的右侧，用户相对于摄像头(或者显示屏)的位置偏航度β的取值范围为(0°，FOV]。例如，如图9A所示，用户A相对于摄像头(或者显示屏)的位置偏航度为O PO A与O PO Q的夹角β a。β a的取值范围为(0°，FOV]。如果用户在摄像头(或者显示屏)的正前方(即用户在上述第一直线上)，用户相对于摄像头(或者显示屏)的位置偏航度为0°。例如，如图9B所示，用户B相对于摄像头(或者显示屏)的位置偏航度为O PO B与O PO Q的夹角β b，β b＝0°。如果用户相对于摄像头(或者显示屏)在上述第一直线的左侧，用户相对于摄像头(或者显示屏)的位置偏航度β的取值范围为[-FOV，0°)。例如，如图9C所示，用户C相对于摄像头(或者显示屏)的位置偏航度为O PO C与O PO Q的夹角β c。β c的取值范围为[-FOV，0°)。
如果第一图像中包括人脸图像，则表示该用户的位置偏航度β∈[-FOV，FOV]。并且，电子设备确定该用户正在关注(在看或者凝视)显示屏、且上述语音数据是该用户发出的，那么电子设备100则可以直接执行该语音数据对应的事件。
需要注意的是，用户可能会在电子设备100的摄像头的视野范围(即视场角FOV)外。在这种情况下，上述第一图像中不包括该用户的人脸图像。此时，如果用户想要通过语音数据(即语音命令)控制电子设备100执行对应的事件，还是需要先发出上述唤醒词(如“小E，小E”)，以唤醒电子设备100的语音助手，然后再向电子设备100发出语音命令(如“调高音量”)。
示例性的，请参考图8B，其示出本申请实施例提供的电子设备100中各个模块的交互原理逻辑框图。一般而言，如图8B所示，电子设备100的“声音采集”模块801可以采集语音数据(如语音数据1)，并将采集到的语音数据1交给“唤醒引擎”802。由“唤醒引擎”802(如AP)判断该语音数据1是否与唤醒词(如“小E，小E”)匹配。如果“唤醒引擎”802确定语音数据1与唤醒词匹配，“唤醒引擎”802才会向“语音识别”模块803发送“声音采集”模块801后续采集的语音数据(如语音数据2)。由“语音识别”模块803对语音数据2进行语音识别(如语义分析等)，然后电子设备100执行语音数据2对应的事件。
而本申请实施例中，“声音采集”模块801可以采集语音数据(如语音数据3)。“声音采集”模块801可以向“免唤醒引擎”807发送采集到的语音数据3。“声源定位”模块805还可以对该语音数据3进行声源定位，获取语音数据3的声源偏航度。“声源定位”模块805可以向“免唤醒引擎”807发送语音数据3的声源偏航度。并且，电子设备100的“关注显示屏”模块804确定有用户在关注显示屏后，可以由“关注人定位”模块806对关注显示屏的用户进行位置定位，获取关注显示屏的用户的位置偏航度。然后，“关注人定位”模块806可以向“免唤醒引擎”807发送获取到的位置偏航度。“免唤醒引擎”807可以在位置偏航度与声源偏航度的差值在第三预设角度范围内时，向“语音识别”模块803发送语音数据3。由“语音识别”模块803对语音数据3进行语音识别(如语义分析等)，然后电子设备100执行语音数据3对应的事件。
综上所述，本申请实施例中，如果用户关注电子设备100的显示屏时，向电子设备100发出语音命令(如上述语音数据3)，电子设备100便可以识别电子设备100采集到的语音数据3，并直接执行该语音数据3对应的事件。通过本申请实施例的方法，电子设备100可以实现与用户之间免唤醒词的语音交互。
示例性的，上述“声音采集”模块801可以为电子设备100的声音传感器。该声音传感器可以采集电子设备100周围的语音数据。上述“关注显示屏”模块804可以包括摄像头。该“关注显示屏”模块804的部分功能可以集成在电子设备100的处理器中。上述“唤醒引擎”802、“免唤醒引擎”807、“语音识别”模块803、“声源定位”模块805、“关注人定位”模块806等可以集成在电子设备100的处理器中。例如，上述“唤醒引擎”802和“免唤醒引擎”807的功能可以在电子设备100的DSP中实现。“关注显示屏”模块804的部分功能可以在电子设备100的NPU中实现。
本申请实施例这里对电子设备100获取用户的位置偏航度的方法进行说明。
其中，用户相对于摄像头的位置不同，则用户的位置偏航度β不同；用户的位置偏航度β不同，则摄像头拍摄得到的第一图片中人脸图像的位置不同。本申请实施例中，位置参数x用于表征用户的人脸图像在第一图片中的位置。具体的，x＝d×tan(f c(β))。
本申请实施例这里以图9A为例，对x＝d×tan(f c(β))进行说明。电子设备100的摄像头可以包括图9A所示的传感器和透镜。传感器和透镜之间的垂直距离为d。以传感器的中心O X为坐标原点，经过O X的水平线为x轴，经过O X的垂直线为y轴。O P为透镜的中心点。
如图9A所示，用户A位于O A点(摄像头的右前方)。用户A相对于摄像头的位置偏航度为β a。光线O AO P经过透镜折射为O PK A，折射角为θ a。即O XO P与O PK A的夹角为θ a。其中，θ a＝f c(β a)。需要说明的是，θ＝f c(β)与摄像头的硬件(如透镜)相关。θ与β之间的函数关系θ＝f c(β)可以通过实验多次测试得到。
其中，点K A是用户A在摄像头的传感器上的一个成像点(例如，第一图片a中的人脸图像的鼻尖所在的像素点)。第一图片a为摄像头拍摄的图片。该第一图片a中包括用户A的人脸图像。K A在上述坐标系中的坐标点为(-x a，0)。O XK A的长度为x a。x a可以表征用户A的人脸图像在第一图片a中的位置。根据三角函数可知：x a＝d×tan(θ a)。由θ a＝f c(β a)和x a＝d×tan(θ a)可以得出：x a＝d×tan(f c(β a))。需要说明的是，本申请实施例中，x a的单位可以为像素点。上述O XK A的长度为x a具体可以为：点O X与点K A的之间相距x a个像素点。
综上所述，用户的人脸图像在第一图片中的位置参数x与该用户的位置偏航度β存在如下函数关系：x＝d×tan(f c(β))。其中，θ＝f c(β)。
例如，如图9B所示，用户B位于O B点(摄像头的正前方)。用户B相对于摄像头的位置偏航度β b＝0°。光线O BO P经过透镜折射为O PK B，折射角θ b＝f c(β b)＝0°。x b＝d×tan(θ b)＝0。又例如，如图9C所示，用户C位于O C点(摄像头的左前方)。用户C相对于摄像头的位置偏航度为β c。光线O BO P经过透镜折射为O PK B，折射角θ c＝f c(β c)。x c＝d×tan(θ c)。上述O XK C的长度为x c具体可以为：点O X与点K C的之间相距x c个像素点。
需要说明的是，θ＝f c(β)和d均与摄像头的硬件相关。本申请实施例中，可以通过调整用户相对于摄像头的位置为β赋不同的值，得到对应的x。示例性的，请参考图10，其示出本申请实施例提供的一种x与β的对应关系表。如图10所示，β＝-50°时，x＝x 5；β＝-40°时，x＝x 4；β＝-30°时，x＝x 3；β＝-20°时，x＝x 2；β＝-10°时，x＝x 1；β＝0°时，x＝x 0；β＝10°时，x＝-x 1；β＝20°时，x＝-x 2；β＝30°时，x＝-x 3；β＝40°时，x＝-x 4；β＝50°时，x＝-x 5等。其中，x的单位为 像素点。x 0等于0个像素点。
例如，如图11所示，为本申请实施例提供的一种x与β的对应关系表实例。如图11所示，β＝0°时，x等于0个像素点；β＝10°时，x等于500个像素点；β＝20°时，x等于1040个像素点；β＝25°时，x等于1358个像素点等。
本申请实施例中，电子设备100可以获取人脸图像在第一图片中的位置参数x，然后查找与x对应的位置偏航度β。
本申请实施例以电子设备100获取图9所示的x a为例，对电子设备100获取用户的人脸图像在第一图片中的位置参数x的方法进行说明：
在一种实现方式中，电子设备100可以通过人脸检测的方式获取第一图片(如第一图片a)中人脸图像的人脸特征信息。例如，人脸特征信息可以包括上述代码中所示的左眼中心位置坐标(1235，1745)、右眼中心位置坐标(1752，1700)、鼻子位置坐标(1487，2055)、左嘴角位置坐标(1314，2357)和右嘴角位置坐标(1774，2321)等。需要注意的是，如图12所示，该人脸位置信息中各个位置的坐标在以第一图片的左上角为坐标原点O的坐标系中。如图12所示，x a可以为第一图片a在x轴方向的中线L与鼻子位置坐标(1487，2055)的垂
Figure PCTCN2020072610-appb-000002
片a)中人脸图像的人脸位置信息(faceRect)。例如，如图13所示，人脸位置信息可以包括：人脸图像的高度(例如，上述"height":1795，表示人脸图像的高度为1795个像素点)；人脸图像的宽度(例如，上述"width":1496，表示人脸图像的高度为1496个像素点)；人脸图像与第一图片左边界的距离(例如，上述"left":761，表示人脸图像与第一图片左边界的距离为761个像素点)；人脸图像与第一图片上边界的距离(例如，上述"top":1033，表示人脸图像与第一图片上边界的距离为1033个像素点)。如图13所示，第一图片a在水平方向的长度为r个
Figure PCTCN2020072610-appb-000003
在另一实施例中，电子设备100可以通过摄像头采集第一图片。电子设备100识别到第一图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。电子设备100获取人脸图像对应的用户的位置偏航度。在电子设备100黑屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度不在第二预设角度范围内，电子设备100则不会点亮屏幕。在电子设备100亮屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度不在第二预设角度范围内，电子设备100则可以自动黑屏。
示例性的，上述第三预设范围可以为[-m°，m°]。例如，m的取值范围可以为[40，60]，或者，m的取值范围可以为[45，65]等。例如，m＝50，或者m＝45。
需要说明的是，人面偏航度、第一预设角度范围，以及电子设备100获取人脸图像对应的用户的人面偏航度，确定人面偏航度在第一预设角度范围内的详细描述，可以参考上述实例中的相关描述，本申请实施例这里不予赘述。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备100可以确定有用户在关注电子设备100的显示屏。如果有用户在关注显示屏，那么该用户可能是机主或者经过机主同意操作或者查看电子设备100的用户；或者，该用户也可能是没有经过机主同意欲操作或者查看电子设备100的用户。
一般而言，电子设备100的机主或者经过机主同意的用户在操作或者查看电子设备100 时，都会位于电子设备100的正前方，或者距离电子设备100的正前方较近的方位。这类用户的位置偏航度在第二预设角度范围内。
如果位置偏航度不在第二预设角度范围内，则表示关注显示屏的用户在电子设备100的两侧，距离电子设备100的正前方较为偏远的方位。在这种情况下，该用户可能不是电子设备100的机主，或者该用户没有经过机主同意在操作或者查看电子设备100。例如，该用户可能是在通过本申请实施例的方法触发电子设备100点亮显示屏；或者，该用户可能是在窃取电子设备100的显示屏所显示的内容。在这种情况下，如果电子设备100当前黑屏，电子设备100则不会点亮屏幕；如果电子设备100当前亮屏，电子设备100则可以自动黑屏。这样，可以保护电子设备100中保存的数据不被窃取。
进一步的，在电子设备100黑屏或者亮屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度不在第二预设角度范围内，电子设备100还可以发出报警提示。，且显示屏状态不改变，仍保持黑屏或者亮屏。
示例性的，电子设备100可以发出语音报警提示。例如，电子设备100可以发出“嘀嘀嘀”的提示音；或者，电子设备100可以发出“安全警报，安全警报！”的语音提示。或者，电子设备100可以发出振动报警提示。本申请实施例对此不作限制。
进一步的，在电子设备100黑屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度在第二预设角度范围内，电子设备100则可以点亮屏幕。在电子设备100亮屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且位置偏航度在第二预设角度范围内，电子设备100则可以继续亮屏。
在另一实施例中，电子设备100可以通过摄像头采集第一图片。电子设备100识别到第一图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。电子设备100可以对该用户进行人脸识别。在电子设备100黑屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且人脸识别未通过，电子设备100则不会点亮屏幕。在电子设备100亮屏的情况下，响应于确定上述人面偏航度在第一预设角度范围内，且人脸识别未通过，电子设备100则可以自动黑屏。
需要说明的是，人面偏航度、第一预设角度范围，以及电子设备100获取人脸图像对应的用户的人面偏航度，确定人面偏航度在第一预设角度范围内的详细描述，可以参考上述实例中的相关描述，本申请实施例这里不予赘述。电子设备100对上述用户进行人脸识别的方法，可以参考常规技术中进行人脸识别的具体方法，本申请实施例这里不予赘述。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备100可以确定有用户在关注电子设备100的显示屏。如果有用户在关注显示屏，且人脸识别未通过，则表示关注显示屏的用户不是经过授权的用户。此时，如果电子设备100当前黑屏，电子设备100则不会点亮屏幕；如果电子设备100当前亮屏，电子设备100则可以自动黑屏。这样，可以保护电子设备100中保存的数据不被窃取。
进一步的，在电子设备100黑屏或者亮屏的情况下，如果上述人面偏航度在第一预设角度范围内，且人脸识别未通过，电子设备100还可以发出报警提示。电子设备100发出报警提示的具体方法可以参考上述实施例中的描述，本申请实施例这里不予赘述。
在另一实施例中，电子设备100可以通过摄像头采集第一图片，通过一个或多个(例如麦克风阵列)麦克风采集语音数据，其中这一个或多个麦克风可以是装备在该电子设备上的，也可以是独立于电子设备但与该电子设备相连的。电子设备100识别到第一图片中包括人脸 图像。电子设备100获取人脸图像对应的用户的人面偏航度。电子设备100获取人脸图像对应的用户的位置偏航度。响应于确定上述人面偏航度在第一预设角度范围内，电子设备100通过麦克风采集语音数据时，对上述位置偏航度对应方位的声源发出的语音数据进行增强处理。进一步的，电子设备100通过麦克风采集语音数据时，还可以对其他方位的声源发出的语音数据进行衰减处理。上述其他方位可以为与上述位置偏航度的偏差在预设角度范围(如上述第一预设角度范围或者第三预设角度范围)之外的方位。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备100可以确定有用户在关注电子设备100的显示屏。如果有用户在关注显示屏，电子设备100则可以对关注显示屏的用户(即上述位置偏航度对应方位的声源)发出的语音数据进行增强处理。这样，电子设备100便可以针对性采集关注显示屏的用户发出的语音数据。
在另一实施例中，本申请实施例的方法可以应用于电子设备100播放音频数据的过程中。电子设备100播放音频数据时，可能会因为电子设备100播放音频数据的音量较高，使得电子设备100不能准确采集到用户发出的语音命令(即语音数据)。为了提高电子设备100采集语音数据的准确性，本申请实施例的方法可以包括：电子设备100通过摄像头采集第一图片。电子设备100识别到第一图片中包括人脸图像。电子设备100获取人脸图像对应的用户的人面偏航度。响应于确定上述人面偏航度在第一预设角度范围内，电子设备100调低该电子设备100的播放音量。
可以理解，如果人面偏航度在第一预设角度范围内，电子设备100可以确定有用户在关注电子设备100的显示屏。在电子设备100播放音频数据的过程中，如果有用户在关注显示屏，那么该用户通过语音命令(即语音数据)控制电子设备100的可能性较高。此时，电子设备100可以调低电子设备100的播放音量，做好采集语音命令的准备。
进一步的，在电子设备100播放音频数据的过程中，如果有用户在关注显示屏，电子设备100不仅可以调低电子设备100的播放音量，做好采集语音命令的准备，以提高电子设备100采集语音数据的准确性。电子设备100还可以通过麦克风采集语音数据时，对上述用户的位置偏航度对应方位的声源发出的语音数据进行增强处理。这样，电子设备100便可以针对性采集关注显示屏的用户的语音数据。
本申请另一实施例还提供一种电子设备，该电子设备可以包括处理器、存储器、显示屏、麦克风和摄像头。其中，存储器、显示屏、摄像头和麦克风与处理器耦合。存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当处理器执行所述计算机指令时，电子设备可以执行上述方法实施例中电子设备100所执行的各个功能或者步骤。其中，该电子设备的结构可以参考图3所示的电子设备100的结构。
例如，上述摄像头，用于采集图片。摄像头可以在显示屏黑屏时采集第一图片。上述处理器，用于识别到第一图片中包括人脸图像，获取第一用户的人面偏航度；响应于确定第一用户的人面偏航度在第一预设角度范围内，则自动点亮显示屏。其中，第一用户是第一图片中的人脸图像对应的用户。第一用户的人面偏航度是第一用户的面部朝向相对于第一连线的左右旋转角度，第一连线是摄像头与第一用户的头部的连线。
进一步的，上述处理器还用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛睁开，则自动点亮所述显示屏。
进一步的，上述处理器还用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛看向所述显示屏，则自动点亮所述显示屏。
进一步的，上述处理器还用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的人面偏航度在所述第一预设角度范围内的持续时间超过预设时间阈值，则自动点亮所述显示屏。
进一步的，上述处理器还用于在自动点亮所述显示屏之前，获取所述第一用户的位置偏航度，所述第一用户的位置偏航度是所述摄像头与所述第一用户的头部的连线和第一直线的夹角，所述第一直线垂直于所述显示屏，且所述第一直线经过所述摄像头；响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的位置偏航度在第二预设角度范围内，则自动点亮所述显示屏。
上述麦克风，用于采集语音数据。上述处理器，还用于获取语音数据的声源偏航度，声源偏航度是摄像头与语音数据的声源的连线和第一直线的夹角；响应于第一用户的人面偏航度在第一预设角度范围内，且第一用户的位置偏航度和声源偏航度的差值在第三预设角度范围内，则执行语音数据对应的语音控制事件。
进一步的，上述处理器，还用于响应于确定第一用户的人面偏航度不在第一预设角度范围内，或者第一用户的位置偏航度和声源偏航度的差值不在第三预设角度范围内，则识别语音数据；响应于确定语音数据是预设唤醒词，则启动电子设备的语音控制功能。其中，处理器，还用于启动语音控制功能启动后，响应于麦克风采集的语音数据执行对应的语音控制事件。
进一步的，处理器，还用于响应于确定人面偏航度在第一预设角度范围内，则通过麦克风采集语音数据时，对位置偏航度对应方位的声源发出的语音数据进行增强处理。
进一步的，电子设备还包括多媒体播放模块(如扬声器)。上述处理器，还用于多媒体播放模块播放多媒体数据时，该多媒体数据包含音频数据，响应于确定第一用户的人面偏航度在第一预设角度范围内，则调低多媒体播放模块的播放音量。
需要说明的是，电子设备的处理器、存储器、显示屏、麦克风和摄像头等的功能包括但不限于上述功能。电子设备的处理器、存储器、显示屏、麦克风和摄像头的其他功能可以参考上述方法实施例中电子设备100所执行的各个功能或者步骤，本申请实施例这里不予赘述。
本申请另一实施例提供一种计算机存储介质，该计算机存储介质包括计算机指令，当所述计算机指令在电子设备上运行时，使得电子设备执行上述方法实施例中电子设备100所执行的各个功能或者步骤。
本申请另一实施例提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述方法实施例中电子设备100所执行的各个功能或者步骤。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本实施例所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接 耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本实施例各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器执行各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：快闪存储器、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本实施例的具体实施方式，但本实施例的保护范围并不局限于此，任何在本实施例揭露的技术范围内的变化或替换，都应涵盖在本实施例的保护范围之内。因此，本实施例的保护范围应以所述权利要求的保护范围为准。

Claims (40)

  1. 一种屏幕控制方法，其特征在于，应用于电子设备，所述电子设备包括显示屏和摄像头，所述方法包括：
    所述电子设备通过所述摄像头，采集显示屏黑屏时的第一图片；
    响应于确定识别到所述第一图片中包括人脸图像，所述电子设备获取第一用户的人面偏航度，所述第一用户是第一图片中的人脸图像对应的用户；所述第一用户的人面偏航度是所述第一用户的面部朝向相对于第一连线的左右旋转角度，所述第一连线是所述摄像头与所述第一用户的头部的连线；
    响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏。
  2. 根据权利要求1所述的方法，其特征在于，所述响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏，包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛睁开，所述电子设备自动点亮所述显示屏。
  3. 根据权利要求1或2所述的方法，其特征在于，所述响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏，包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛看向所述显示屏，所述电子设备自动点亮所述显示屏。
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，所述响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏，包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的人面偏航度在所述第一预设角度范围内的持续时间超过预设时间阈值，所述电子设备自动点亮所述显示屏。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，在所述电子设备自动点亮所述显示屏之前，所述方法还包括：
    所述电子设备获取所述第一用户的位置偏航度，所述第一用户的位置偏航度是所述摄像头与所述第一用户的头部的连线和第一直线的夹角，所述第一直线垂直于所述显示屏，且所述第一直线经过所述摄像头；
    所述响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏，包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的位置偏航度在第二预设角度范围内，所述电子设备自动点亮所述显示屏。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    响应于确定所述第一用户的位置偏航度不在所述第二预设角度范围内，则所述电子设备发出报警指示。
  7. 根据权利要求1-6中任一项所述的方法，其特征在于，在所述电子设备自动点亮所述显示屏之前，所述方法还包括：
    所述电子设备对所述第一用户进行人脸识别；
    所述响应于确定所述第一用户的人面偏航度在第一预设角度范围内，所述电子设备自动点亮所述显示屏，包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的人脸识别成功，所述电子设备自动点亮所述显示屏。
  8. 根据权利要求1-7中任一项所述的方法，其特征在于，在所述电子设备自动点亮所述显示屏之后，所述方法还包括：
    所述电子设备通过所述摄像头采集第二图片；
    所述电子设备识别所述第二图片中是否包括人脸图像；
    响应于确定所述第二图片中不包括人脸图像，所述电子设备自动黑屏。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    响应于确定所述第二图片中包括人脸图像，所述电子设备获取第二用户的人面偏航度，所述第二用户是第二图片中的人脸图像对应的用户；所述第二用户的人面偏航度是所述第二用户的面部朝向相对于第二连线的左右旋转角度，所述第二连线的所述摄像头与所述第二用户的头部的连线；
    响应于确定所述第二用户的人面偏航度不在所述第一预设角度范围内，所述电子设备自动黑屏。
  10. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    所述电子设备通过与之相连的麦克风采集语音数据；
    所述电子设备获取所述语音数据的声源偏航度，所述声源偏航度是所述摄像头与所述语音数据的声源的连线和所述第一直线的夹角；
    响应于所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的位置偏航度和所述声源偏航度的差值在第三预设角度范围内，所述电子设备执行所述语音数据对应的语音控制事件。
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：
    响应于确定所述第一用户的人面偏航度不在所述第一预设角度范围内，和/或所述第一用户的位置偏航度和所述声源偏航度的差值不在所述第三预设角度范围内，所述电子设备识别所述语音数据；
    响应于确定所述语音数据是预设唤醒词，所述电子设备启动所述电子设备的语音控制功能；
    其中，所述语音控制功能启动后，所述电子设备响应于所述麦克风采集的语音数据，执行对应的语音控制事件。
  12. 根据权利要求10或11所述的方法，其特征在于，所述电子设备中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；所述位置参数用于表征人脸图像在对应图片中的位置；
    所述电子设备获取所述第一用户的位置偏航度，包括：
    所述电子设备获取所述第一用户的人脸图像在所述第一图片中的位置参数；
    所述电子设备查找与获取到的位置参数对应的位置偏航度；并将查找到的位置偏航度作为所述第一用户的位置偏航度。
  13. 根据权利要求10-12中任一项所述的方法，其特征在于，所述方法还包括：
    响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，所述电子设备通过所述麦克风采集语音数据时，对所述第一用户的位置偏航度对应方位的声源发出的语音数据进行增强处理。
  14. 根据权利要求1-13中任一项所述的方法，其特征在于，所述方法还包括：
    响应于确定所述电子设备正在播放多媒体数据，且所述第一用户的人面偏航度在所述第一预设角度范围内，所述电子设备调低所述电子设备的播放音量。
  15. 一种语音控制方法，其特征在于，应用于电子设备，所述电子设备包括麦克风、显示屏和摄像头，所述方法包括：
    所述电子设备通过所述摄像头采集第一图片，通过所述麦克风采集语音数据；
    响应于确定识别到所述第一图片中包括人脸图像，所述电子设备获取所述人脸图像对应的用户的人面偏航度，并获取所述用户的位置偏航度；所述人面偏航度是所述用户的面部朝向相对于第一连线的左右旋转角度，所述第一连线是所述摄像头与所述用户的头部的连线；所述位置偏航度是所述摄像头与所述用户的头部的连线和第一直线的夹角，所述第一直线垂直于所述显示屏，且所述第一直线经过所述摄像头；
    所述电子设备获取所述语音数据的声源偏航度，所述声源偏航度是所述摄像头与所述语音数据的声源的连线和所述第一直线的夹角；
    响应于确定所述人面偏航度在第一预设角度范围内，且所述位置偏航度和所述声源偏航度的差值在第三预设角度范围内，所述电子设备执行所述语音数据对应的语音控制事件。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    响应于确定所述人面偏航度不在所述第一预设角度范围内，和/或所述位置偏航度和所述声源偏航度的差值不在所述第三预设角度范围内，所述电子设备识别所述语音数据；
    响应于确定所述语音数据是预设唤醒词，所述电子设备启动所述电子设备的语音控制功能；
    其中，所述语音控制功能启动后，所述电子设备响应于所述麦克风采集的语音数据执行对应的语音控制事件。
  17. 根据权利要求15或16所述的方法，其特征在于，所述电子设备中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；所述位置参数用于表征人脸图像在对应图片中的位置；
    所述获取所述用户的位置偏航度，包括：
    所述电子设备获取所述人脸图像在所述第一图片中的位置参数；
    所述电子设备查找与获取到的位置参数对应的位置偏航度，并将查找到的位置偏航度作为所述位置偏航度。
  18. 根据权利要求15-17中任一项所述的方法，其特征在于，所述方法还包括：
    响应于确定所述人面偏航度在所述第一预设角度范围内，所述电子设备通过所述麦克风采集语音数据时，对所述位置偏航度对应方位的声源发出的语音数据进行增强处理。
  19. 根据权利要求15-18中任一项所述的方法，其特征在于，所述方法还包括：
    所述电子设备播放多媒体数据时，响应于确定所述人面偏航度在所述第一预设角度范围内，所述电子设备调低所述电子设备的播放音量。
  20. 一种电子设备，其特征在于，所述电子设备包括一个或多个处理器、一个或多个存储器、显示屏和摄像头；所述一个或多个存储器、所述显示屏和所述摄像头与所述一个或多个处理器耦合，所述一个或多个存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述一个或多个处理器执行所述计算机指令时，
    所述摄像头，用于采集图片；
    所述一个或多个处理器，用于在所述显示屏黑屏时，识别所述摄像头采集的第一图片中是否包括人脸图像，如果包含人脸图像，则获取第一用户的人面偏航度，所述第一用户是所述第一图片中的人脸图像对应的用户；所述第一用户的人面偏航度是所述第一用户的面部朝向相对于第一连线的左右旋转角度，所述第一连线是所述摄像头与所述第一用户的头部的连线；响应于确定所述第一用户的人面偏航度在第一预设角度范围内，则指示所述显示屏亮屏。
  21. 根据权利要求20所述的电子设备，其特征在于，所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在第一预设角度范围内，则指示所述显示屏亮屏，包括：
    所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛睁开，则指示所述显示屏亮屏。
  22. 根据权利要求20或21所述的电子设备，其特征在于，所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在第一预设角度范围内，则指示所述显示屏亮屏，包括：
    所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的眼睛看向所述显示屏，则指示所述显示屏亮屏。
  23. 根据权利要求20-22中任一项所述的电子设备，其特征在于，所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在第一预设角度范围内，则指示所述显示屏亮屏，包括：
    所述一个或多个处理器，具体用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的人面偏航度在所述第一预设角度范围内的持续时间超过预设时间阈值，则指示所述显示屏亮屏。
  24. 根据权利要求20-23中任一项所述的电子设备，其特征在于，所述一个或多个处理器，还用于在指示点亮所述显示屏之前，获取所述第一用户的位置偏航度，所述第一用户的位置偏航度是所述摄像头与所述第一用户的头部的连线和第一直线的夹角，所述第一直线垂直于所述显示屏，且所述第一直线经过所述摄像头；
    所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在第一预设角度范围内，则指示所述显示屏亮屏，包括：
    所述一个或多个处理器，用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的位置偏航度在第二预设角度范围内，则指示所述显示屏亮屏。
  25. 根据权利要求24所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述第一用户的位置偏航度不在所述第二预设角度范围内，发出报警指示。
  26. 根据权利要求20-25中任一项所述的电子设备，其特征在于，所述一个或多个处理器，还用于在指示所述显示屏亮屏之前，对所述第一用户进行人脸识别；
    所述一个或多个处理器，具体用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的人脸识别通过，则指示所述显示屏亮屏。
  27. 根据权利要求20-26中任一项所述的电子设备，其特征在于，所述摄像头，还用于在所述处理器自动点亮所述显示屏之后，采集第二图片；
    所述一个或多个处理器，还用于识别所述第二图片中是否包括人脸图像；响应于确定所述第二图片中不包括人脸图像，则指示所述显示屏黑屏。
  28. 根据权利要求27所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述第二图片中包括人脸图像，获取第二用户的人面偏航度，所述第二用户是第二 图片中的人脸图像对应的用户；所述第二用户的人面偏航度是所述第二用户的面部朝向相对于第二连线的左右旋转角度，所述第二连线的所述摄像头与所述第二用户的头部的连线；响应于确定所述第二用户的人面偏航度不在所述第一预设角度范围内，则指示所述显示屏黑屏。
  29. 根据权利要求24所述的电子设备，其特征在于，所述电子设备还包括一个或多个麦克风；
    所述一个或多个麦克风，用于采集语音数据；
    所述一个或多个处理器，还用于获取所述语音数据的声源偏航度，所述声源偏航度是所述摄像头与所述语音数据的声源的连线和所述第一直线的夹角；响应于所述第一用户的人面偏航度在所述第一预设角度范围内，且所述第一用户的位置偏航度和所述声源偏航度的差值在第三预设角度范围内，则执行所述语音数据对应的语音控制事件。
  30. 根据权利要求29所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述第一用户的人面偏航度不在所述第一预设角度范围内，和/或所述第一用户的位置偏航度和所述声源偏航度的差值不在所述第三预设角度范围内，则识别所述语音数据；响应于确定所述语音数据是预设唤醒词，则启动所述电子设备的语音控制功能；
    其中，所述一个或多个处理器，还用于启动所述语音控制功能启动后，响应于所述一个或多个麦克风采集的语音数据执行对应的语音控制事件。
  31. 根据权利要求29或30所述的电子设备，其特征在于，所述一个或多个存储器中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；所述位置参数用于表征人脸图像在对应图片中的位置；
    所述一个或多个处理器，用于获取所述第一用户的位置偏航度，包括：
    所述一个或多个处理器，用于获取所述第一用户的人脸图像在所述第一图片中的位置参数；查找与获取到的位置参数对应的位置偏航度；并将查找到的位置偏航度作为所述第一用户的位置偏航度。
  32. 根据权利要求29-31中任一项所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，则通过所述一个或多个麦克风采集语音数据时，对所述第一用户的位置偏航度对应方位的声源发出的语音数据进行增强处理。
  33. 根据权利要求20-32中任一项所述的电子设备，其特征在于，
    所述一个或多个处理器，还用于在播放多媒体数据时，响应于确定所述第一用户的人面偏航度在所述第一预设角度范围内，则调低播放音量。
  34. 一种电子设备，其特征在于，所述电子设备包括一个或多个处理器、一个或多个存储器、显示屏、摄像头和一个或多个麦克风；所述存储器、所述显示屏和所述摄像头与所述一个或多个处理器耦合；所述摄像头用于采集第一图片；所述麦克风用于采集语音数据；
    所述一个或多个存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述一个或多个处理器执行所述计算机指令时，
    所述一个或多个处理器，用于确定所述第一图片中包括人脸图像时，获取所述人脸图像对应的用户的人面偏航度，并获取所述用户的位置偏航度；所述人面偏航度是所述用户的面部朝向相对于第一连线的左右旋转角度，所述第一连线是所述摄像头与所述用户的头部的连线；所述位置偏航度是所述摄像头与所述用户的头部的连线和第一直线的夹角，所述第一直线垂直于所述显示屏，且所述第一直线经过所述摄像头；获取所述语音数据的声源偏航度， 所述声源偏航度是所述摄像头与所述语音数据的声源的连线和所述第一直线的夹角；响应于确定所述人面偏航度在第一预设角度范围内，且所述位置偏航度和所述声源偏航度的差值在第三预设角度范围内，则执行所述语音数据对应的语音控制事件。
  35. 根据权利要求34所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述人面偏航度不在所述第一预设角度范围内，和/或所述位置偏航度和所述声源偏航度的差值不在所述第三预设角度范围内，则识别所述语音数据；响应于确定所述语音数据是预设唤醒词，则启动所述电子设备的语音控制功能；
    其中，所述处理器，还用于在启动所述语音控制功能启动后，响应于所述一个或多个麦克风采集的语音数据，执行所述语音数据对应的语音控制事件。
  36. 根据权利要求34或35所述的电子设备，其特征在于，所述一个或多个处理器中预先保存有多个位置参数，以及每个位置参数对应的位置偏航度；所述位置参数用于表征人脸图像在对应图片中的位置；
    所述获取所述用户的位置偏航度，包括：
    获取所述人脸图像在所述第一图片中的位置参数；查找与获取到的位置参数对应的位置偏航度，并将查找到的位置偏航度作为所述位置偏航度。
  37. 根据权利要求34-36中任一项所述的电子设备，其特征在于，所述一个或多个处理器，还用于响应于确定所述人面偏航度在所述第一预设角度范围内，则通过所述麦克风采集语音数据时，对所述位置偏航度对应方位的声源发出的语音数据进行增强处理。
  38. 根据权利要求34-37中任一项所述的电子设备，其特征在于，所述一个或多个处理器，还用于在播放多媒体数据时，响应于确定所述人面偏航度在所述第一预设角度范围内，则调低播放音量。
  39. 一种计算机存储介质，其特征在于，包括计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行如权利要求1-19中任一项所述的方法。
  40. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1-19中任一项所述的方法。
PCT/CN2020/072610 2019-01-25 2020-01-17 一种屏幕控制和语音控制方法及电子设备 WO2020151580A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910075866.1 2019-01-25
CN201910075866.1A CN109710080B (zh) 2019-01-25 2019-01-25 一种屏幕控制和语音控制方法及电子设备

Publications (1)

Publication Number Publication Date
WO2020151580A1 true WO2020151580A1 (zh) 2020-07-30

Family

ID=66263015

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/072610 WO2020151580A1 (zh) 2019-01-25 2020-01-17 一种屏幕控制和语音控制方法及电子设备

Country Status (2)

Country Link
CN (1) CN109710080B (zh)
WO (1) WO2020151580A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112188289A (zh) * 2020-09-04 2021-01-05 青岛海尔科技有限公司 用于控制电视的方法及装置、设备
CN113627290A (zh) * 2021-07-27 2021-11-09 歌尔科技有限公司 一种音箱控制方法、装置、音箱及可读存储介质
CN113965641A (zh) * 2021-09-16 2022-01-21 Oppo广东移动通信有限公司 音量调节方法及装置、终端及计算机可读存储介质
WO2022042274A1 (zh) * 2020-08-31 2022-03-03 华为技术有限公司 一种语音交互方法及电子设备
CN114422686A (zh) * 2020-10-13 2022-04-29 Oppo广东移动通信有限公司 参数调整方法及相关装置
CN114779916A (zh) * 2022-03-29 2022-07-22 杭州海康威视数字技术股份有限公司 一种电子设备屏幕唤醒方法、门禁管理方法及装置

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109710080B (zh) * 2019-01-25 2021-12-03 华为技术有限公司 一种屏幕控制和语音控制方法及电子设备
CN110164443B (zh) * 2019-06-28 2021-09-14 联想(北京)有限公司 用于电子设备的语音处理方法、装置以及电子设备
CN110456938B (zh) 2019-06-28 2021-01-29 华为技术有限公司 一种曲面屏的防误触方法及电子设备
CN110415695A (zh) * 2019-07-25 2019-11-05 华为技术有限公司 一种语音唤醒方法及电子设备
CN110364159B (zh) * 2019-08-19 2022-04-29 北京安云世纪科技有限公司 一种语音指令的执行方法、装置及电子设备
CN110718225A (zh) * 2019-11-25 2020-01-21 深圳康佳电子科技有限公司 一种语音控制方法、终端以及存储介质
CN111276140B (zh) * 2020-01-19 2023-05-12 珠海格力电器股份有限公司 语音命令识别方法、装置、系统及存储介质
CN111256404B (zh) * 2020-02-17 2021-08-27 海信（山东）冰箱有限公司 一种储藏装置和储藏装置的控制方法
CN113741681B (zh) * 2020-05-29 2024-04-26 华为技术有限公司 一种图像校正方法与电子设备
CN111736725A (zh) * 2020-06-10 2020-10-02 京东方科技集团股份有限公司 智能镜子及智能镜子唤醒方法
CN112188341B (zh) * 2020-09-24 2024-03-12 江苏紫米电子技术有限公司 一种耳机唤醒方法、装置、耳机及介质
CN116391212A (zh) * 2020-11-06 2023-07-04 华为技术有限公司 一种防止手势误识别的方法及电子设备
CN112489578A (zh) * 2020-11-19 2021-03-12 北京沃东天骏信息技术有限公司 商品呈现方法和装置
CN112687295A (zh) * 2020-12-22 2021-04-20 联想(北京)有限公司 一种输入控制方法及电子设备
CN112667084B (zh) * 2020-12-31 2023-04-07 上海商汤临港智能科技有限公司 车载显示屏的控制方法及装置、电子设备和存储介质
CN117616486A (zh) * 2021-07-15 2024-02-27 海信视像科技股份有限公司 控制方法及控制设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103747346A (zh) * 2014-01-23 2014-04-23 中国联合网络通信集团有限公司 一种多媒体视频播放的控制方法及多媒体视频播放器
CN105912903A (zh) * 2016-04-06 2016-08-31 上海斐讯数据通信技术有限公司 一种移动终端的解锁方法及移动终端
CN106155621A (zh) * 2015-04-20 2016-11-23 钰太芯微电子科技（上海）有限公司 可识别声源位置的关键词语音唤醒系统及方法及移动终端
CN107765858A (zh) * 2017-11-06 2018-03-06 广东欧珀移动通信有限公司 确定人脸角度的方法、装置、终端及存储介质
CN109710080A (zh) * 2019-01-25 2019-05-03 华为技术有限公司 一种屏幕控制和语音控制方法及电子设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20020033829A (ko) * 2000-08-01 2002-05-07 요트.게.아. 롤페즈 디바이스를 소리 소스에 겨냥하기
CN103902963B (zh) * 2012-12-28 2017-06-20 联想(北京)有限公司 一种识别方位及身份的方法和电子设备
KR102163850B1 (ko) * 2014-01-29 2020-10-12 삼성전자 주식회사 디스플레이장치 및 그 제어방법
CN104238948B (zh) * 2014-09-29 2018-01-16 广东欧珀移动通信有限公司 一种智能手表点亮屏幕的方法及智能手表
KR101761631B1 (ko) * 2015-12-29 2017-07-26 엘지전자 주식회사 이동단말기 및 그 제어방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103747346A (zh) * 2014-01-23 2014-04-23 中国联合网络通信集团有限公司 一种多媒体视频播放的控制方法及多媒体视频播放器
CN106155621A (zh) * 2015-04-20 2016-11-23 钰太芯微电子科技（上海）有限公司 可识别声源位置的关键词语音唤醒系统及方法及移动终端
CN105912903A (zh) * 2016-04-06 2016-08-31 上海斐讯数据通信技术有限公司 一种移动终端的解锁方法及移动终端
CN107765858A (zh) * 2017-11-06 2018-03-06 广东欧珀移动通信有限公司 确定人脸角度的方法、装置、终端及存储介质
CN109710080A (zh) * 2019-01-25 2019-05-03 华为技术有限公司 一种屏幕控制和语音控制方法及电子设备

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022042274A1 (zh) * 2020-08-31 2022-03-03 华为技术有限公司 一种语音交互方法及电子设备
CN112188289A (zh) * 2020-09-04 2021-01-05 青岛海尔科技有限公司 用于控制电视的方法及装置、设备
CN114422686A (zh) * 2020-10-13 2022-04-29 Oppo广东移动通信有限公司 参数调整方法及相关装置
CN113627290A (zh) * 2021-07-27 2021-11-09 歌尔科技有限公司 一种音箱控制方法、装置、音箱及可读存储介质
CN113965641A (zh) * 2021-09-16 2022-01-21 Oppo广东移动通信有限公司 音量调节方法及装置、终端及计算机可读存储介质
CN113965641B (zh) * 2021-09-16 2023-03-28 Oppo广东移动通信有限公司 音量调节方法及装置、终端及计算机可读存储介质
CN114779916A (zh) * 2022-03-29 2022-07-22 杭州海康威视数字技术股份有限公司 一种电子设备屏幕唤醒方法、门禁管理方法及装置

Also Published As

Publication number Publication date
CN109710080B (zh) 2021-12-03
CN109710080A (zh) 2019-05-03

Similar Documents

Publication Publication Date Title
WO2020151580A1 (zh) 一种屏幕控制和语音控制方法及电子设备
WO2020156269A1 (zh) 一种具有柔性屏幕的电子设备的显示方法及电子设备
WO2021000876A1 (zh) 一种语音控制方法、电子设备及系统
WO2021052214A1 (zh) 一种手势交互方法、装置及终端设备
EP3901753A1 (en) Screen control method, electronic device, and storage medium
WO2020207328A1 (zh) 图像识别方法和电子设备
WO2021063311A1 (zh) 具有折叠屏的电子设备的显示控制方法及电子设备
CN116070684B (zh) 一种集成芯片以及处理传感器数据的方法
WO2022193989A1 (zh) 电子设备的操作方法、装置和电子设备
CN111543049B (zh) 一种拍照方法及电子设备
WO2020019355A1 (zh) 一种可穿戴设备的触控方法、可穿戴设备及系统
CN113641488A (zh) 一种基于用户使用场景进行资源优化的方法和装置
WO2022116930A1 (zh) 内容共享方法、电子设备及存储介质
WO2022089000A1 (zh) 文件系统检测方法、电子设备及计算机可读存储介质
WO2020237617A1 (zh) 控屏方法、装置、设备及存储介质
CN114090102B (zh) 启动应用程序的方法、装置、电子设备和介质
CN114880251B (zh) 存储单元的访问方法、访问装置和终端设备
CN113676339B (zh) 组播方法、装置、终端设备及计算机可读存储介质
WO2020221062A1 (zh) 一种导航操作方法及电子设备
EP4199488A1 (en) Voice interaction method and electronic device
WO2020078267A1 (zh) 在线翻译过程中的语音数据处理方法及装置
CN113496477A (zh) 屏幕检测方法及电子设备
WO2023029916A1 (zh) 批注展示方法、装置、终端设备及可读存储介质
CN113572798B (zh) 设备控制方法、系统、设备和存储介质
CN115206308A (zh) 一种人机交互的方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20745661

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20745661

Country of ref document: EP

Kind code of ref document: A1