WO2023019999A1 - 工作模式的切换控制方法、电子设备及可读存储介质 - Google Patents

工作模式的切换控制方法、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2023019999A1
WO2023019999A1 PCT/CN2022/089195 CN2022089195W WO2023019999A1 WO 2023019999 A1 WO2023019999 A1 WO 2023019999A1 CN 2022089195 W CN2022089195 W CN 2022089195W WO 2023019999 A1 WO2023019999 A1 WO 2023019999A1
Authority
WO
WIPO (PCT)
Prior art keywords
image sensor
working mode
address
image
mode
Prior art date
Application number
PCT/CN2022/089195
Other languages
English (en)
French (fr)
Inventor
叶凌
陈国乔
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US18/547,641 priority Critical patent/US20240137649A1/en
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Priority to EP22857317.6A priority patent/EP4287605A1/en
Publication of WO2023019999A1 publication Critical patent/WO2023019999A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/65—Control of camera operation in relation to power supply
    • H04N23/651—Control of camera operation in relation to power supply for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation

Definitions

  • the present application relates to the technical field of data processing, and in particular to a working mode switching control method, electronic equipment and a readable storage medium.
  • An electronic device such as a mobile phone, has a camera including an image sensor, and the image sensor includes two working modes to adapt to different power consumption requirements.
  • the image sensor includes two working modes to adapt to different power consumption requirements.
  • the mutual switching between the two working modes of the image sensor requires the processor to generate a high level or a low level to act on the switching pin of the image sensor, and the control logic of the processor is relatively complicated.
  • the present application provides a working mode switching control method, an electronic device, a program product and a storage medium, aiming to solve the problem of complex control logic when switching the working mode of an image sensor.
  • the present application provides a method for switching control of working modes, which is applied to electronic equipment.
  • the electronic equipment includes a processor and a camera.
  • the image sensor in the camera includes at least two working modes.
  • the switching control method of working modes includes: The processor sends a signal to the image sensor through the I2C interface, wherein the signal is used to control the image sensor to operate in a working mode corresponding to the signal.
  • the processor sends signals to the image sensor through the I2C interface, and controls the image sensor to operate in the corresponding working mode of the signal, realizing the switching of the working mode of the image sensor controlled by the processor through the signal sent through the I2C interface.
  • the processor needs to generate high and low levels to switch the working mode of the image sensor, so as to solve the problem of complex control logic caused by the processor using high and low levels to switch the working mode of the image sensor.
  • the signal is an I2C address.
  • the image sensor includes an I2C control module
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor sends the I2C address to the image sensor through the I2C interface, triggering the operation of the I2C control module corresponding to the I2C address, To control the image sensor to run in the working mode corresponding to the I2C address.
  • one I2C control module corresponds to two I2C addresses, and one I2C address corresponds to at least one working mode of the image sensor.
  • the image sensor includes the same number of I2C control modules as the number of working modes of the image sensor, and each I2C control module corresponds to an I2C address, and one I2C address corresponds to a working mode of the image sensor.
  • one I2C address corresponds to at least two working modes of the image sensor
  • the switching control method of the working mode further includes: the image sensor responds to the switching signal, and controls the image sensor to correspond to the working mode of the same I2C address.
  • One mode of operation operates and the switching signal is sent by the processor to the image sensor or generated by the image sensor.
  • the working modes of the image sensor include: a first working mode and a second working mode, and the second working mode has lower power consumption than the first working mode; the first working mode may also be called a normal working mode or In normal mode, the resolution of the image generated in the first working mode is the first image resolution; the second working mode can also be called a lower power consumption mode, and the resolution of the image generated in the second working mode is the first Second image resolution; first image resolution > second image resolution.
  • the first mode of operation of the image sensor corresponds to the first I2C address
  • the second mode of operation of the image sensor corresponds to the second I2C address
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, and sends a signal to the image sensor The sensor sends the first I2C address to control the image sensor to operate in the first working mode; the processor determines that the camera is turned off, sends the second I2C address to the image sensor, and controls the image sensor to operate in the second working mode; the processor determines the display of the electronic device The screen is turned off, the second I2C address is sent to the image sensor, and the image sensor is controlled to operate in the second working mode.
  • the working modes of the image sensor include: a first working mode, a second working mode and a third working mode, the second working mode has lower power consumption than the first working mode, and the third working mode has lower power consumption than the second working mode
  • the power consumption of the working mode is low, the first working mode of the image sensor corresponds to the first I2C address, the second working mode of the image sensor corresponds to the second I2C address, and the third working mode of the image sensor corresponds to the third I2C address;
  • the first working mode also It can be called the normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution; the second working mode can also be called a lower power consumption mode, and the generated The resolution of the image is the second image resolution; the third working mode can also be called the ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third image resolution; the first image resolution > Second image resolution > Third image resolution.
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, sends the first I2C address to the image sensor, and controls the image sensor to operate in the first working mode; The sensor sends the second I2C address to control the image sensor to run in the second working mode, or sends the third I2C address to the image sensor to control the image sensor to run in the third working mode; the processor determines that the display screen of the electronic device is off, and sends The image sensor sends the second I2C address to control the image sensor to operate in the second operating mode, or sends the third I2C address to the image sensor to control the image sensor to operate in the third operating mode; the processor uses the detection result of the light and shadow change to determine the camera detection When the light and shadow changes, send the second I2C address to the image sensor to control the image sensor to operate in the second working mode.
  • the detection result of the light and shadow change is performed by the image sensor in the third working mode using the image data of the ultra-low resolution image.
  • the detection of light and shadow changes is obtained; the processor uses the image data of the lower-resolution image to determine that the electronic device meets the exit conditions of the second working mode, sends the third I2C address to the image sensor, and controls the image sensor to run in the third working mode.
  • a low resolution image is generated by the image sensor in the second mode of operation.
  • the image sensor since the image sensor is set with an ultra-low power consumption mode, the power consumption of the ultra-low power consumption mode is lower than that of the low power consumption mode.
  • the image sensor uses the ultra-low power consumption Mode operation can reduce power consumption.
  • the electronic device is not in use may be understood as: the electronic device is in standby, the display screen is off, or the electronic device is in a locked screen state.
  • the working modes of the image sensor include: a first working mode, a second working mode and a third working mode, the second working mode has lower power consumption than the first working mode, and the third working mode has lower power consumption than the second working mode
  • the working mode has low power consumption
  • the first working mode of the image sensor corresponds to the first I2C address
  • the second working mode and the third working mode of the image sensor correspond to the second I2C address
  • the first working mode can also be called normal working mode or normal mode, the resolution of the image generated in the first working mode is the first image resolution
  • the second working mode can also be called a lower power consumption mode, and the resolution of the image generated in the second working mode is the second Image resolution
  • the third working mode can also be called an ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third image resolution
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, sends the first I2C address to the image sensor, and controls the image sensor to operate in the first working mode; The sensor sends the second I2C address to control the image sensor to operate in the second working mode or the third working mode; the processor determines that the display screen of the electronic device is off, sends the second I2C address to the image sensor, and controls the image sensor to work in the second working mode mode or the third working mode.
  • the image sensor since the image sensor is set with an ultra-low power consumption mode, the power consumption of the ultra-low power consumption mode is lower than that of the low power consumption mode.
  • the image sensor uses the ultra-low power consumption Mode operation can reduce power consumption.
  • the electronic device is not in use may be understood as: the electronic device is in standby, the display screen is off, or the electronic device is in a locked screen state.
  • the second working mode and the third working mode of the image sensor correspond to the second I2C address, and time-division multiplexing of the second I2C address can be realized.
  • the image sensor responds to the switching signal, and controls the image sensor to operate in one of the working modes corresponding to the same I2C address, including: the image sensor responds to the first switching signal, and controls the image sensor to operate in the second Working mode operation, the first switching signal is generated by the processor or the image sensor using the detection result of the light and shadow change to determine when the camera detects the light and shadow change, and the detection result of the light and shadow change is generated by the image sensor in the third working mode using ultra-low resolution
  • the image data of the high-resolution image is obtained by detecting the change of light and shadow; the image sensor responds to the second switching signal to control the image sensor to operate in the third working mode, and the second switching signal is used by the processor or the image sensor to use the image of the lower resolution image
  • the data is generated by determining that the electronic device satisfies the exit condition of the second working mode, and the lower-resolution image is generated by the image sensor in the second working mode.
  • the image sensor responds to the switching signal, and controls the image sensor to operate in one of the working modes corresponding to the same I2C address, including: the image sensor receives the information transmitted by the I2C control module corresponding to the second I2C address
  • the control signal is used to control the image sensor to operate in the working mode specified by the control signal, and the control signal is used to specify the third working mode or the second working mode.
  • the working modes of the image sensor include: a first working mode, a second working mode, a third working mode and a fourth working mode, the second working mode has lower power consumption than the first working mode, and the fourth working mode The working mode has lower power consumption than the second working mode, the third working mode has lower power consumption than the fourth working mode, the first working mode corresponds to the first I2C address, the second working mode corresponds to the second I2C address, and the fourth working mode corresponds to the first I2C address.
  • the third working mode corresponds to the fourth I2C address;
  • the first working mode can also be called normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution;
  • the second The working mode can also be called a lower power consumption mode, and the resolution of the image generated in the second working mode is the second image resolution;
  • the third working mode can also be called an ultra-low power consumption mode, and in the third working mode
  • the resolution of the image generated under is the third image resolution;
  • the fourth working mode can be referred to as a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution;
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, sends the first I2C address to the image sensor, and controls The image sensor operates in the first working mode; the processor determines that the camera is turned off, sends a
  • the image sensor additionally sets a low power mode and an ultra-low power mode, the power consumption of the ultra-low power mode is lower than that of the low power mode, and the power consumption of the low power mode is lower than that of the low power mode.
  • the image sensor can ensure that the electronic equipment controls the image sensor to operate in four working modes, and further ensures that the electronic equipment controls the image sensor to select a suitable power consumption mode to operate, so as to reduce the power consumption of the electronic equipment.
  • the working modes of the image sensor include: a first working mode, a second working mode, a third working mode and a fourth working mode, the second working mode has lower power consumption than the first working mode, and the fourth working mode The working mode has lower power consumption than the second working mode, and the third working mode has lower power consumption than the fourth working mode.
  • the first working mode and the second working mode correspond to the first I2C address
  • the fourth working mode and the third working mode correspond to the first I2C address.
  • the first working mode can also be called normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution
  • the second working mode can also be called lower power consumption mode, the resolution of the image generated in the second working mode is the second image resolution
  • the third working mode can also be called the ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third Image resolution
  • the fourth working mode can be called a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, sends the first I2C address to the image sensor, and controls the image sensor to run in the first working mode; processing The processor determines that the camera is turned off, sends the second I2C address to the image sensor, and controls the image sensor to operate in the third working mode; the processor determines
  • the image sensor responds to the switching signal, and controlling the image sensor to operate in one of the working modes corresponding to the same I2C address includes: the image sensor responds to the first switching signal, and controls the image sensor to operate in the fourth Mode operation, the first switching signal is generated by the processor or image sensor using the detection result of the light and shadow change to determine when the camera detects the light and shadow change, and the detection result of the light and shadow change is generated by the image sensor in the third working mode using ultra-low resolution
  • the image data of the image is obtained by detecting changes in light and shadow;
  • the image sensor responds to the second switching signal to control the image sensor to operate in the second working mode, and the second switching signal is determined by the processor according to the face and/or gesture detection results Generated when there is a face and/or gesture in the captured image, the face and/or gesture detection result is obtained by the image sensor in the fourth working mode using the image data of the low-resolution image to detect the face and/or gesture .
  • an I2C address corresponds to at least two working modes of the image sensor
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor sends the I2C address and a switching signal to the image sensor, and controls the image sensor to The working mode corresponding to the I2C address and switching signal runs.
  • the working modes of the image sensor include: a first working mode, a second working mode, a third working mode and a fourth working mode, the second working mode has lower power consumption than the first working mode, and the fourth working mode The working mode has lower power consumption than the second working mode, and the third working mode has lower power consumption than the fourth working mode.
  • the first working mode and the second working mode correspond to the first I2C address
  • the fourth working mode and the third working mode correspond to the first I2C address.
  • the first working mode can also be called normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution;
  • the second working mode can also be called lower power consumption mode, the resolution of the image generated in the second working mode is the second image resolution;
  • the third working mode can also be called the ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third Image resolution;
  • the fourth working mode can be called a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution; first image resolution>second image resolution>fourth image resolution rate>the third image resolution;
  • the processor sends the I2C address and switching signal to the image sensor, and controls the image sensor to operate in the corresponding working mode of the I2C address and the switching signal, including: the processor determines that the camera is started, and sends a signal to the image sensor The first I2C address and the first switching signal control the image sensor to operate in the first working mode; the processor determines that the camera is turned off, send
  • Face and/or gesture appear in the captured image, send the first I2C address and the second switching signal to the image sensor, control the image sensor to run in the second working mode, and the face and/or gesture detection result is determined by being in the fourth working mode
  • the image sensor uses the image data of the low-resolution image to detect the face and/or gesture; the processor determines that no face and/or hand appear in the image captured by the camera according to the face and/or gesture detection result potential, sending the second I2C address and the fourth switching signal to the image sensor to control the image sensor to operate in the third working mode; the processor uses the image data of the lower resolution image to determine that the electronic device meets the exit conditions of the second working mode, Send the second I2C address and the fourth switching signal to the image sensor to control the image sensor to operate in the third working mode, and the lower resolution image is generated by the image sensor in the second working mode.
  • the working modes of the image sensor include: a first working mode, a second working mode, a third working mode and a fourth working mode, the second working mode has lower power consumption than the first working mode, and the fourth working mode The working mode has lower power consumption than the second working mode, the third working mode has lower power consumption than the fourth working mode, the first working mode corresponds to the first I2C address, the second working mode, the fourth working mode and the third working mode correspond to the first I2C address.
  • the first working mode can also be called normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution
  • the second working mode can also be called lower power consumption mode, the resolution of the image generated in the second working mode is the second image resolution
  • the third working mode can also be called the ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third Image resolution
  • the fourth working mode can be called a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution
  • the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started, sends the first I2C address to the image sensor, and controls the image sensor to operate in the first working mode.
  • the image sensor responds to the switching signal, and controls the image sensor to operate in one of the working modes corresponding to the same I2C address, including: the image sensor receives the information transmitted by the I2C control module corresponding to the second I2C address
  • the control signal is used to control the image sensor to operate in the working mode specified by the control signal, and the control signal is used to specify the third working mode, the second working mode or the fourth working mode.
  • the control signal transmitted by the I2C control module corresponding to the second I2C address includes: a first control signal, a second control signal and a third control signal; wherein: the working mode specified by the first control signal is the second
  • the three working modes are generated when the processor determines that the camera is turned off, or when the processor determines that the display screen of the electronic device is turned off, or the processor determines the image captured by the camera based on the face and/or gesture detection results.
  • the working mode specified by the second control signal is the first Four working modes, generated when the processor determines that the camera is turned off, or generated when the processor determines that the display screen of the electronic device is off, or generated when the processor determines that the camera detects a light and shadow change using the detection result of the light and shadow change,
  • the detection result of the light and shadow change is obtained by the image sensor in the third working mode, using the image data of the ultra-low resolution image, to detect the light and shadow change
  • the working mode specified by the third control signal is the second working mode, which is determined by the processor Generated when it is determined that the camera is turned off, or generated when the processor determines that a human face and/or gesture appears in an image captured by the camera according to a face and/or gesture detection result.
  • the working modes of the image sensor include: a first working mode, a third working mode and a fourth working mode, the fourth working mode has lower power consumption than the first working mode, and the third working mode has lower power consumption than the fourth working mode.
  • the power consumption of the working mode is low, the first working mode of the image sensor corresponds to the first I2C address, the fourth working mode of the image sensor corresponds to the second I2C address, and the third working mode of the image sensor corresponds to the third I2C address;
  • the first working mode also It can be called the normal working mode or normal mode, and the resolution of the image generated in the first working mode is the first image resolution;
  • the third working mode can also be called the ultra-low power consumption mode, and the generated image in the third working mode
  • the resolution of the image is the third image resolution; the fourth working mode can be referred to as a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution; the first image resolution>the fourth Image resolution>third image resolution; wherein: the processor
  • the detection result of the light and shadow change is determined by the image sensor in the third working mode.
  • the image data of the ultra-low resolution image is obtained by detecting light and shadow changes; the processor uses the face and/or gesture detection results to determine that no face and/or gesture appear in the image captured by the camera, and sends a third I2C to the image sensor
  • the address controls the image sensor to operate in the third working mode, and the face and/or gesture detection result is obtained by the image sensor in the fourth working mode using the image data of the low-resolution image to detect the face and/or gesture.
  • the working modes of the image sensor include: a first working mode, a third working mode and a fourth working mode, the fourth working mode has lower power consumption than the first working mode, and the third working mode has lower power consumption than the fourth working mode.
  • the working mode has low power consumption, the first working mode of the image sensor corresponds to the first I2C address, the fourth working mode and the third working mode of the image sensor correspond to the second I2C address; the first working mode can also be called normal working mode or normal mode, the resolution of the image generated in the first working mode is the first image resolution; the third working mode can also be called ultra-low power consumption mode, and the resolution of the image generated in the third working mode is the third Image resolution; the fourth working mode can be called a low power consumption mode, and the resolution of the image generated in the fourth working mode is the fourth image resolution; the first image resolution>the fourth image resolution>the third image resolution rate; wherein: the processor sends a signal to the image sensor through the I2C interface, including: the processor determines that the camera is started
  • the image sensor responds to the switching signal, and controlling the image sensor to operate in one of the working modes corresponding to the same I2C address includes: the image sensor responds to the first switching signal, and controls the image sensor to operate in the fourth Mode operation, the first switching signal is generated by the processor or image sensor using the detection result of the light and shadow change to determine when the camera detects the light and shadow change, and the detection result of the light and shadow change is generated by the image sensor in the third working mode using ultra-low resolution
  • the image data of the image is obtained by detecting light and shadow changes;
  • the image sensor responds to the second switching signal to control the image sensor to operate in the third working mode, and the second switching signal is used by the processor or the image sensor to use the face and/or gesture detection results , it is determined that the image captured by the camera does not have a human face and/or gesture, and the detection result of the human face and/or gesture is performed by the image sensor in the fourth working mode using the image data of the low-resolution image to perform human face and/or gesture
  • the image sensor responds to the switching signal, and controls the image sensor to operate in one of the working modes corresponding to the same I2C address, including: the image sensor receives the information transmitted by the I2C control module corresponding to the second I2C address
  • the control signal is used to control the image sensor to operate in the working mode specified by the control signal, and the control signal is used to specify the third working mode or the fourth working mode.
  • the object that sends the signal includes: the camera driver of the processor, or the simplest image front-end or image front-end in the image signal processor of the processor, or the application processor of the processor, or the processor smart sensor hub.
  • the switching signal is sent to the image sensor by the simplest image front-end or image front-end in the image signal processor of the processor, or the application processor of the processor, or the intelligent sensor hub.
  • the switching signal generated and sent by the smart sensor hub since the smart sensor hub has low power consumption, the switching signal generated and sent by the smart sensor hub has the advantage of low power consumption.
  • the present application provides an electronic device, including: a display screen; a camera, the camera includes an image sensor, and the image sensor includes at least two working modes; one or more processors; a memory, on which a program is stored; When the program is executed by one or more processors, the electronic device executes the switching control method of the working mode according to any one of the first aspect.
  • the present application provides a computer-readable storage medium on which a computer program is stored, wherein, when the computer program is executed by a processor, the method for controlling switching of working modes according to any one of the first aspect is implemented.
  • the present application provides a computer program product.
  • the computer program product When the computer program product is run on a computer, the computer is made to execute the switching control method of the working mode according to any one of the first aspect.
  • Figures 1a and 1b are diagrams of an application scenario of the electronic device provided by the present application.
  • FIG. 2 is a schematic diagram of connection between an image sensor and a processor of an electronic device provided by the present application;
  • Figure 3a is a schematic structural diagram of the electronic device provided by the present application.
  • Figure 3b is a diagram showing the operation process of the logic unit in the electronic device provided by the present application.
  • FIG. 4a is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 4b is an example diagram of a software structure of an electronic device provided in an embodiment of the present application.
  • FIG. 5a and FIG. 5b are schematic diagrams of the connection between the image sensor and the processor of the electronic device provided in Embodiment 1 of the present application;
  • Fig. 5c is a timing diagram of the working mode switching control method provided by Embodiment 1 of the present application.
  • FIG. 6 is a display diagram of the working state of the electronic device provided in Embodiment 1 of the present application.
  • FIG. 7a is a schematic diagram of the connection between the image sensor and the processor of the electronic device provided in Embodiment 2 of the present application;
  • FIG. 7b is a schematic diagram of switching the working mode of the image sensor provided in Embodiment 2 of the present application.
  • FIG. 8a is a schematic diagram of the connection between the image sensor and the processor of the electronic device provided in Embodiment 3 of the present application;
  • Figure 8b is a schematic diagram of the connection between the image sensor and the processor of the electronic device provided in Embodiment 3 of the present application
  • FIG. 9a and FIG. 9b are schematic diagrams of the connection between the image sensor and the processor of the electronic device provided in Embodiment 4 of the present application;
  • FIG. 9c and FIG. 9d are schematic diagrams of switching the working modes of the image sensor provided in Embodiment 4 of the present application.
  • FIG. 10a is a schematic diagram of connection between an image sensor and a processor of an electronic device provided in Embodiment 4 of the present application;
  • FIG. 10b is a schematic diagram of switching the working mode of the image sensor provided in Embodiment 4 of the present application.
  • Fig. 11a, Fig. 11b, Fig. 11c and Fig. 11d are schematic diagrams of the connection between the image sensor and the processor of the electronic device provided in Embodiment 4 of the present application;
  • Fig. 12a is a schematic diagram of the connection between the image sensor and the processor of the electronic device provided in Embodiment 5 of the present application;
  • FIG. 12b and FIG. 12c are schematic diagrams of switching the working modes of the image sensor provided in Embodiment 5 of the present application.
  • Fig. 12d is a sequence diagram of the working mode switching control method provided by Embodiment 5 of the present application.
  • FIG. 13a and FIG. 13b are schematic diagrams of the connection between the image sensor and the processor of the electronic device provided in Embodiment 5 of the present application;
  • FIG. 14 is a schematic diagram of connection between an image sensor and a processor of an electronic device provided in Embodiment 6 of the present application.
  • words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • Users can take pictures or take selfies through electronic devices, and can also check web pages, news, articles, etc. through electronic devices, and can also play games and watch videos through electronic devices.
  • users check web pages, news, articles, play games or watch videos through electronic devices users will stare at the display screen of electronic devices for a long time. After using the image captured by the front camera to detect that the user is staring at the display screen for a long time, various corresponding events can be executed, such as the display screen not turning off, the ringtone volume decreasing, etc.
  • Figure 1a shows a scene where a user takes a selfie through a front camera of an electronic device.
  • Figure 1b shows a scenario where a user browses a web page through an electronic device.
  • the following two scenarios are used as examples to introduce solutions.
  • the front camera of the electronic device is involved in the two scenarios, the front camera of the electronic device is taken as an example for illustration. But none of these constitutes a limitation on the application scenarios of the solution.
  • the image sensor in the front camera of an electronic device generally has two working modes, a normal mode and a lower power consumption mode, and the lower power consumption mode consumes less power than the normal mode.
  • the normal mode may also be referred to as a normal working mode, which belongs to a common imaging mode.
  • the electronic device uses a front-facing camera to take pictures, and the image sensor operates in normal mode.
  • the front camera (the image sensor is used to replace the display in the figure, and the image sensor will be used to replace the front camera will be introduced below) is also powered on. After the front camera is powered on, it can enter the lower level under the control of the camera driver. power mode.
  • the image sensor in the lower power consumption mode will generate a lower resolution image at intervals, usually 640*480, and pass the image data of the lower resolution image through the mobile industry processor interface (mobile industry processor interface, MIPI) to the simplest image front end (Image Front End lite, IFE lite) unit.
  • the image data of the lower resolution image output by IFE lite can be used in a safe buffer stored in memory.
  • the simplest image front end (Image Front End lite, IFE lite) unit is an integrated unit in the image signal processor (image signal processor, ISP), and the image of the low-resolution image output by each camera of the electronic device The data can reach the IFE lite, which stores the image output by the camera in the safe buffer of the memory.
  • image signal processor image signal processor
  • the processor's controller can perform events such as the detection of human eye gaze on the display (also known as AO scheme).
  • the implementation of the AO solution by the controller involves three logic units: the automatic exposure module, the AO (always on) module, and the camera driver.
  • the three logic units are introduced below.
  • the automatic exposure module belongs to a logic unit of the controller, and is obtained by running an automatic exposure (automatic exposure, AE) algorithm by the controller.
  • the AO (always on) module is also a logical unit of the controller, which is obtained by running the AO (always on) scheme on the controller.
  • the AO solution refers to the intelligent perception solution based on the AO camera (always on camera), which usually includes functions such as human eye gaze recognition, machine owner recognition, and gesture recognition.
  • the typical feature is long-term low-power operation.
  • the camera driver is also a logical unit of the controller, which is used to configure parameters, working modes, etc. of the camera, and is also used to turn the camera on or off.
  • the controller of the processor uses the image data of the lower resolution image to execute the AO scheme in the following manner:
  • the display screen of the electronic device displays the webpage, and the user looks at the display screen of the electronic device to view the webpage.
  • the electronic device sends an instruction, and the front camera of the electronic device operates after responding to the instruction, and executes step S1 to capture the face image of the user.
  • the simplest image front-end unit performs step S2 to read the face image, and stores the face image in the security buffer of the internal memory based on the security mechanism.
  • the AO module executes step S3-1, acquires the image data of the face image stored in the security buffer of the internal memory, and determines whether the user's eyes are watching the display screen by analyzing the image data.
  • step S4 is executed to control the display screen of the electronic device not to turn off the screen.
  • the image quality of the face image captured by the camera restricts the accuracy of the AO module in determining whether the user's eyes are watching the display. Especially when the image brightness of the face image captured by the camera is high or low, the error of the AO module in determining whether the user's eyes are watching the display screen is relatively large. For this reason, the automatic exposure module executes step S3-2 to obtain the image data of the face image stored in the internal memory; use the image data to calculate the image brightness of the face image, compare the calculated image brightness with the standard brightness, and obtain the comparison Match the results; adjust the exposure parameters of the camera according to the comparison results, generally the exposure time and gain, and obtain the exposure time adjustment value and gain adjustment value.
  • the automatic exposure module also executes step S5, and transmits the calculated exposure duration adjustment value and gain adjustment value to the AO module, and the AO module then sends the exposure duration adjustment value and gain adjustment value to the camera as shown in step S6 in Figure 3a Drive, the camera is driven as shown in step S7 in Figure 3a, configure the camera to run with the exposure time adjustment value and the gain adjustment value.
  • the electronic device can send an instruction again, and the camera responds to the instruction of the electronic device, and operates to capture images with the adjustment value of the exposure time and the adjustment value of the gain.
  • the following describes how the AO module analyzes the image data, determines whether the user's eyes are watching the display screen, and the automatic exposure module adjusts the exposure parameters of the camera in conjunction with FIG. 3b.
  • the image sequence includes multiple frames of images captured by the camera, such as image frames 1, 2, 3, 4...n, where the camera starts to operate with a common exposure time and gain.
  • the common exposure time and gain can be pre-set.
  • the automatic exposure module sequentially acquires the image data of each frame of the image sequence according to the storage order of the images.
  • the automatic exposure module uses the image data of image frame 1 to calculate the image brightness of image frame 1, compares the image brightness of image frame 1 with the standard brightness, and obtains the comparison result .
  • the automatic exposure module will not perform operations, and the camera will still run with the original exposure time and gain.
  • the duration and gain refer to the aforementioned general exposure duration and gain. If the comparison result reflects that the difference between the image brightness of image frame 1 and the standard brightness is not less than the preset value, the automatic exposure module adjusts the exposure duration and gain of the camera according to the comparison result, and obtains the adjustment value of exposure duration 1 and the adjustment value of gain 1 .
  • the automatic exposure module transmits the adjustment value of exposure time 1 and gain 1 to the camera driver through the AO module.
  • the camera driver configures the camera to capture images by operating with an adjusted value of exposure time 1 and an adjusted value of gain 1.
  • the automatic exposure module follows the above processing method, Using the image data of image frame 2 and image frame 3, calculate the exposure duration 1 adjustment value and gain 1 adjustment value; the camera driver configures the camera to operate and capture images with the exposure duration 1 adjustment value and gain 1 adjustment value.
  • the image frame 4 is captured by the camera configured with an adjustment value of exposure time 1 and an adjustment value of gain 1 .
  • the automatic exposure module also samples the above-mentioned processing method, uses the image data of the image frame 4, and calculates the adjusted value of the exposure time length 2 and the adjusted value of the gain 2; the camera driver configures the camera to run and shoot images with the adjusted value of the exposure time length 2 and the adjusted value of the gain 2. This is repeated until the difference between the image brightness of the image frame compared by the automatic exposure module and the standard brightness is less than a preset value, such as ⁇ 10%, and stop.
  • a preset value such as ⁇ 10%
  • the AO module also sequentially acquires the image data of each frame of the image sequence according to the storage order of the images. For the image data of each frame of image acquired by the AO module, the AO module executes the following process to obtain the judgment result of human eyes watching the display screen in each frame of image. The following takes the image data of the image frame 1 processed by the AO module as an example.
  • the AO module compares the image data of image frame 1 with the sample feature library, and configures a confidence level for image frame 1 according to the comparison result between the image data of image frame 1 and the sample feature library, which is used to characterize the image frame 1.
  • the probability that the human eye looks at the display The AO module judges whether the confidence degree of image frame 1 is less than the threshold value, and the confidence degree of image frame 1 is not less than the threshold value, then it is determined that the human eyes in image frame 1 are watching the display screen, and the confidence degree of image frame 1 is less than the threshold value , it is determined that the human eye in image frame 1 is not looking at the display screen.
  • the sample feature database includes feature data of human eyes gazing at images on the display screen.
  • the method of determining the characteristic data is: obtain a large number of sample images, the sample images include the sample images of the human eyes watching the display screen and the sample images of the human eyes not watching the display screen, and use the image data of each sample image to learn, and obtain the representative human Characteristic data of the eye-gaze display image.
  • the sample images where the human eyes look at the display screen and the sample images where the human eyes do not watch the display screen both refer to the face images captured by the front camera of the electronic device.
  • the AO module determines that there is a frame of human eyes watching the display screen in the image within a certain period of time, and then executes corresponding events such as controlling the display screen of the electronic device to not turn off the screen, and the volume of the ringtone to decrease.
  • the AO module determines that there is a frame of images in a series of multiple frames of images, and the human eyes in this frame of images are watching the display screen, then the display screen of the electronic device is controlled to not turn off the screen, and the display screen of the electronic device is set Combined with the scheme of controlling the display screen to be off by a predetermined screen off time, the combination method is as follows.
  • the electronic device is set to turn off the screen for 15 seconds. Start the timing when the display screen of the electronic device is started to display data. After the specified time, such as 7 seconds, the front camera of the electronic device captures images, and the AO module acquires images in sequence, and executes Do the following:
  • the image data of the image is compared with the sample feature library, and the confidence level is configured for the image according to the comparison result of the image data of the image and the sample feature library. Judging whether the configured confidence of the image is less than the screened confidence threshold, if the configured confidence of the image is not less than the confidence threshold, it is determined that the human eye in the image is watching the display screen; if the configured confidence of the image is If the degree is less than the confidence threshold, it is determined that the human eyes in the image are not looking at the display screen.
  • the AO module determines that there is a frame of image in which the human eye is watching the display screen, and then controls the display screen not to turn off. If the AO module continues to determine that none of the human eyes in the image is looking at the display screen, it will control the display screen to turn off when the timer reaches 15 seconds.
  • the controller of the electronic device executes the aforementioned AO solution, if the user activates the front camera of the electronic device and enters the scene of taking a selfie with the front camera shown in Figure 1a, the electronic device will respond to the user's operation, The front camera takes pictures to obtain images, and the display screen displays the captured images.
  • the application processor (application processor, AP) sends a low-level power-on switching signal to the switching pin of the image sensor, and the working mode of the image sensor is switched from a lower power consumption mode to normal mode.
  • the image sensor in normal mode generates a standard resolution image at intervals, and sends the image data of the standard resolution image to the ISP's image front end (Image Front End, IFE) through the MIPI pin.
  • the image front-end IFE processes the image data of the standard resolution image, and displays the processed image data on the display screen.
  • the electronic device responds to the user's operation and executes a corresponding process. Specifically, the AP shown in Figure 2 will send a high-level power-on switching signal to the switching pin of the image sensor, and the image sensor will switch from the normal mode to the lower power consumption mode.
  • the image sensor is in a lower power consumption mode, the image sensor generates a lower resolution image, and the processor can execute various events such as an AO scheme according to the lower resolution image.
  • the processor can execute various events such as an AO scheme according to the lower resolution image.
  • the image sensor of the electronic device will also enter a lower power consumption mode, and the processor cooperates to execute various events such as the AO scheme, which will cause the processor to consume less power. big.
  • the switching between the lower power consumption mode and the normal mode of the image sensor requires the processor to generate a high level or a low level, which acts on the switching pin of the image sensor, and the control logic is more complicated.
  • the processor of the electronic device needs to control the two working modes of the image sensor to control the image
  • the sensor is virtualized into two devices, that is, two control logics of the virtual image sensor are configured. By controlling the switching operation of the two virtual image sensors, the power-on switching signal is sent to realize the switching of the working mode of the image sensor.
  • the logic is more complicated.
  • the processor of the electronic device needs to be equipped with the simplest image front end (Image Front End lite, IFE lite) unit, so that the use of the processor is limited, and the configuration
  • the processor with the simplest image front-end unit can cooperate with the image sensor to operate in a relatively low power consumption mode.
  • Fig. 4a is a composition example of an electronic device provided by the embodiment of the present application.
  • the electronic device 400 may include a processor 410, an external memory interface 420, an internal memory 421, a display screen 430, a camera 440, an antenna 1, an antenna 2, a mobile communication module 450, and a wireless communication module 460, etc.
  • the structure shown in this embodiment does not constitute a specific limitation on the electronic device.
  • the electronic device may include more or fewer components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 410 may include one or more processing units, for example: the processor 410 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • graphics processing unit graphics processing unit
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 400 .
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • Video codecs are used to compress or decompress digital video.
  • An electronic device may support one or more video codecs.
  • the electronic device can play or record video in multiple encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • the NPU is a neural-network (NN) computing processor.
  • NPU neural-network
  • Applications such as intelligent cognition of electronic devices can be realized through NPU, such as: image recognition, face recognition, speech recognition, text understanding, etc.
  • a memory may also be provided in the processor 410 for storing instructions and data.
  • the memory in processor 410 is a cache memory.
  • the memory may hold instructions or data that the processor 410 has just used or recycled. If the processor 410 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 410 is reduced, thus improving the efficiency of the system.
  • processor 410 may include one or more interfaces.
  • the interface may include an integrated circuit bus (Inter-Integrated Circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, And/or a universal serial bus (universal serial bus, USB) interface, etc.
  • I2C Inter-Integrated Circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • mobile industry processor interface mobile industry processor interface
  • MIPI mobile industry processor interface
  • general-purpose input and output general-purpose input/output
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL).
  • processor 410 may include multiple sets of I2C buses. The processor 410 may be respectively coupled to the camera 440 and the like through different I2C bus interfaces.
  • the I2S interface can be used for audio communication.
  • the PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • the MIPI interface can be used to connect the processor 410 with peripheral devices such as the display screen 430 and the camera 440 .
  • MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 410 communicates with the camera 440 through the CSI interface to realize the shooting function of the electronic device 400 .
  • the processor 410 communicates with the display screen 430 through the DSI interface to realize the display function of the electronic device 400 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 410 with the camera 440 , the display screen 430 , the wireless communication module 460 and so on.
  • the interface connection relationship between the modules shown in this embodiment is only for schematic illustration, and does not constitute a structural limitation of the electronic device 400 .
  • the electronic device 400 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the external memory interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device.
  • the external memory card communicates with the processor 410 through the external memory interface 420 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 421 may be used to store computer-executable program codes including instructions.
  • the processor 410 executes various functional applications and data processing of the electronic device 400 by executing instructions stored in the internal memory 421 .
  • the internal memory 421 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like.
  • the storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device.
  • the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 410 executes various functional applications and data processing of the electronic device by executing instructions stored in the internal memory 421 and/or instructions stored in a memory provided in the processor.
  • the electronic device realizes the display function through the GPU, the display screen 430 , and the application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 430 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 410 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the display screen 430 is used to display images, videos and the like.
  • the display screen 430 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oled, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the electronic device may include 1 or N display screens 430, where N is a positive integer greater than 1.
  • a series of graphical user interface can be displayed on the display screen 430 of the electronic device, and these GUIs are the main screen of the electronic device.
  • GUI graphical user interface
  • the size of the display screen 430 of the electronic device is fixed, and only limited controls can be displayed on the display screen 430 of the electronic device.
  • a control is a GUI element, which is a software component contained in an application that controls all data processed by the application and the interaction of these data. Users can interact with the control through direct manipulation. , so as to read or edit the relevant information of the application.
  • controls may include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and Widgets.
  • the electronic device can realize the shooting function through ISP, camera 440 , video codec, GPU, display screen 430 and application processor.
  • the ISP is used to process data fed back by the camera 440 .
  • the light is transmitted to the camera image sensor through the lens, the light signal is converted into an electrical signal, and the camera image sensor transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be located in the camera 440 .
  • Camera 440 includes a lens and an image sensor. Camera 440 is used to capture still images or video. The object generates an optical image through the lens and projects it to the image sensor.
  • the image sensor can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the image sensor converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the electronic device may include 1 or N cameras 440, where N is a positive integer greater than 1.
  • DSP is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy, etc.
  • the wireless communication function of the electronic device can be realized by the antenna 1, the antenna 2, the mobile communication module 450, the wireless communication module 460, the modem processor and the baseband processor.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 450 can provide wireless communication solutions including 2G/3G/4G/5G applied to electronic devices.
  • the mobile communication module 450 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
  • the mobile communication module 450 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
  • the mobile communication module 450 can also amplify the signal modulated by the modem processor, convert it into electromagnetic wave and radiate it through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 450 may be set in the processor 410 .
  • at least part of the functional modules of the mobile communication module 450 and at least part of the modules of the processor 410 may be set in the same device.
  • the wireless communication module 460 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 460 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 460 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 410 .
  • the wireless communication module 460 can also receive the signal to be transmitted from the processor 410 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • an operating system runs on top of the above components.
  • An application program can be installed and run on the operating system.
  • Fig. 4b is a block diagram of the software structure of the electronic device according to the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the Android system is divided into four layers, which are respectively the application program layer, the application program framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer from top to bottom.
  • the application layer can consist of a series of application packages. As shown in FIG. 4b, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, display, music, ringtone, and short message.
  • applications such as camera, gallery, calendar, call, map, navigation, WLAN, display, music, ringtone, and short message.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions. As shown in Figure 4b, the application framework layer may include a window manager, a content provider, a phone manager, a resource manager, a notification manager, a view system, etc., and in some embodiments of the present application, the application framework layer also includes Awareness services can be included.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make it accessible to applications.
  • Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.
  • the phone manager is used to provide communication functions of electronic devices. For example, the management of call status (including connected, hung up, etc.).
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify the download completion, message reminder, etc.
  • the notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window.
  • prompting text information in the status bar issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
  • the view system can be used to build applications.
  • a display interface can consist of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the perception service is used to implement the AO scheme proposed above. During the execution of the AO solution for the perception service, if it is detected that the human eye is watching the display in a frame of image, the display of the control application layer will be displayed in a way that does not turn off the screen. output.
  • the Android Runtime includes core library and virtual machine.
  • the Android runtime is responsible for the scheduling and management of the Android system.
  • the cold start of the application will run in the Android runtime, and the Android runtime obtains the optimized file status parameters of the application, and then the Android runtime can judge whether the optimized file is outdated due to system upgrades through the optimized file status parameters , and return the judgment result to the application control module.
  • the core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application program layer and the application program framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • a system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to realize 3D graphics drawing, image rendering, compositing and layer processing, etc.
  • the 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.
  • the camera driver is used to configure the parameters of the camera and turn the camera on or off.
  • an embodiment of the present application provides an electronic device, which includes a processor and an image sensor capable of interacting with the processor, and the image sensor is generally disposed on a front camera of the electronic device.
  • the hardware structure and software architecture of the electronic device may be as described above, and will not be described here.
  • the processor includes a controller, an image signal processor ISP and an application processor AP, and the ISP includes at least two integrated units of the image front end IFE and the simplest image front end IFE lite.
  • the controller can include three logic units: AO module, automatic exposure module and camera driver. The functions of the AO module, the automatic exposure module and the camera driver can be as described above.
  • the set working modes include a first working mode, a second working mode, and a third working mode.
  • the first working mode may also be referred to as a normal working mode or a normal mode, and the resolution of an image generated in the first working mode is the first image resolution, for example, may be 4208*3120.
  • the second working mode can also be called a lower power consumption mode, the power consumption of the second working mode is lower than that of the first working mode, and the resolution of the image generated in the second working mode is the second image resolution, Usually it can be 320*240, 520*392, 640*480, 720*540, 800*600, 960*720, 1040*784.
  • the third working mode may also be called an ultra-low power consumption mode.
  • the third working mode has the lowest power consumption, and the resolution of the image generated in the third working mode is the third image resolution, usually 16*12. Therefore, it can be seen that: first image resolution>second image resolution>third image resolution.
  • the image sensor and the processor are respectively provided with MIPI interfaces, pin 1 and pin 2.
  • the MIPI interface of the image sensor which can transmit the image data of the image generated by the image sensor.
  • the image data can be transmitted to the processor's ISP via the processor's MIPI interface.
  • the image front-end IFE in the ISP acquires the image data received by the MIPI interface of the processor; in other embodiments, the simplest image front-end IFE lite in the ISP acquires the image data received by the MIPI interface of the processor.
  • Pin 1 of the image sensor is used to transmit a one-bit level signal.
  • pin 1 of the processor is also used to transmit a bit level signal.
  • the level signal that the image sensor needs to send externally such as the detection result indicated by high level or low level, can be transmitted externally through pin 1, and then transmitted to the processing through pin 1 of the processor.
  • device ISP In some embodiments, the simplest image front end IFE lite in the ISP obtains the level signal received by the pin 1 of the processor.
  • pin 1 of the image sensor may be referred to as a GPO pin, INT pin.
  • Pin 2 of the image sensor is also used to transmit a bit level signal.
  • pin 2 of the processor is also used to transmit a bit level signal.
  • the pin 2 of the image sensor is a switching pin for receiving a switching signal for switching the working mode of the image sensor.
  • the IFE lite or IFE in the ISP of the processor, or the AP of the processor generates a switching signal, and sends the switching signal through the pin 2 of the processor, and the switching signal is transmitted to the image through the pin 2 of the image sensor.
  • the sensor, the image sensor switches the working mode according to the switching signal.
  • IFE lite, IFE or AP all have the ability to generate switching signals, but if IFE lite, IFE or AP is used to generate switching signals, the power consumption will increase in turn, and the control complexity will decrease in turn.
  • pin 2 of the image sensor may be referred to as a PONV pin, Xshutdown2 pin.
  • the switching control method of the working mode provided by this embodiment is applied to the aforementioned electronic equipment and electronic equipment
  • the image sensor enters the ultra-low power mode by default, or usually enters the ultra-low power mode under the control of the camera driver.
  • the switching control method of the working mode provided by this embodiment includes the following steps:
  • the image sensor in the ultra-low power consumption mode generates an ultra-low-resolution image every other time interval.
  • the electronic device is turned on and powered on, the display screen lights up and outputs a lock screen interface as shown in (a) in Figure 6, the image sensor operates in an ultra-low power consumption mode, and generates Super low resolution image.
  • the image sensor when the electronic device is on standby and the display screen is off, the image sensor will also operate in an ultra-low power consumption mode to generate ultra-low-resolution images at a configured time interval.
  • the image sensor in the ultra-low power consumption mode uses the image data of the ultra-low resolution image to detect light and shadow changes, and obtain a light and shadow change detection result.
  • the image sensor utilizes the image data of the ultra-low resolution image to detect the change of light and shadow, and is used to determine whether there is a change of light and shadow in front of the front camera of the electronic device.
  • the image sensor sends the light and shadow change detection result to the minimal image front end IFE lite.
  • the image sensor can input the detection result of light and shadow changes to the simplest image front-end IFE lite through pin 1.
  • the minimal image front-end IFE lite determines that the front camera of the electronic device detects a light and shadow change by using the detection result of the light and shadow change, and then generates a switching signal.
  • the simplest image front-end IFE lite determines that the front camera of the electronic device detects changes in light and shadow, indicating that the user may be using the electronic device and needs to switch the working mode of the image sensor, so IFE lite generates a switching signal to control the image sensor from the ultra-low power consumption mode Switch to a lower power mode.
  • the switching signal includes high and low levels.
  • the high-level switching signal is used to control the image sensor to enter an ultra-low power consumption mode
  • the low-level switching signal is used to control the image sensor to enter a low power consumption mode.
  • a high-level switching signal may also control the image sensor to switch to a lower power consumption mode
  • a low-level switching signal may control the image sensor to switch to an ultra-low power consumption mode.
  • the switching signal may also be generated by the IFE or the AP.
  • the IFE or AP interacts with the minimal image front end IFE lite.
  • the minimal image front end IFE lite uses the detection results of light and shadow changes to determine that the front camera of the electronic device detects light and shadow changes, and the IFE or AP generates a switching signal.
  • the IFE or the AP receives the detection result of the light and shadow change, uses the detection result of the light and shadow change to determine that the front camera of the electronic device has detected the light and shadow change, and generates a switching signal.
  • the display screen of the electronic device outputs a webpage, and the user watches the display screen for a long time while reading the webpage. Based on this, when the image sensor uses the image data of ultra-low resolution images to detect light and shadow changes, it will detect light and shadow changes, and the simplest image front-end IFE lite needs to generate switching signals.
  • the minimal image front-end IFE lite can use the detection results of light and shadow changes to execute some low-power events.
  • the display automatically rotates according to the direction of the face, triggers functions such as screenshot/playback pause, and page turning.
  • the minimal image front-end IFE lite determines that the front camera of the electronic device does not detect light and shadow changes, and then controls the image sensor to continue in the ultra-low power consumption mode.
  • the simplest image front end IFE lite sends a switching signal to the image sensor.
  • the simplest image front-end IFE lite can send a switching signal to the image sensor through pin 2, and the image sensor responds to the switching signal, switching the working mode from the ultra-low power consumption mode to the lower power consumption mode.
  • the image sensor in the lower power consumption mode generates a lower-resolution image every other time interval.
  • the image sensor in a lower power consumption mode sends image data of a lower resolution image to the simplest image front end IFE lite.
  • the image sensor sends the image data of the lower resolution image to the simplest image front end IFE lite through the MIPI pin.
  • the minimal image front end IFE lite can store image data of lower resolution images in a secure buffer in memory.
  • the controller of the processor can use the image data of the low-resolution image to perform various events, such as the human eye gazing at the display Screen detection (also known as AO solution), control the automatic rotation of the display screen according to the direction of the face, trigger screen capture/play pause, page turning and other functions.
  • the controller uses the image data of the lower-resolution image to execute various events. If the controller uses the image data of the lower-resolution image to determine that the electronic device meets the exit conditions of the executed event, such as executing In the AO solution, as described above, the controller continues to determine that none of the human eyes in the image is looking at the display screen, so the electronic device may not continue to execute the executed event. Moreover, the image sensor can continue to operate in a lower power consumption mode, or control the image sensor to switch to an ultra-low power consumption mode.
  • the minimal image front end IFE lite generates a switching signal to control the image sensor to switch to an ultra-low power consumption mode.
  • the image front end IFE generates a low-level power-on switching signal.
  • the user activates the front camera of the electronic device, and the display screen of the electronic device is shown in (c) in FIG. 6 , showing the shooting mode of the front camera.
  • the electronic device will respond to the user's operation, and the front camera will take pictures to obtain images, and the captured images will be displayed on the display screen.
  • the front camera of the electronic device is activated by a control command, and the image sensor needs to enter a normal mode.
  • the image front-end IFE will generate a low-level power-on switching signal to control the image sensor to enter the normal mode.
  • the IFE lite or the AP may also generate a low-level power-on switching signal to control the image sensor to enter the normal mode.
  • the image front-end IFE generates a low-level power-on switching signal as an example for illustration.
  • Power-on switching signal including high and low levels.
  • the low-level power-on switching signal is used to control the image sensor to switch to the normal mode
  • the high-level power-on switching signal is used to control the image sensor to exit the normal mode.
  • the high-level power-on switching signal may also control the image sensor to switch to the normal mode
  • the low-level power-on switching signal may control the image sensor to exit the normal mode. In this embodiment, it is described by taking a low-level power-on switching signal to control the image sensor to switch to the normal mode, and a high-level power-on switching signal to control the image sensor to exit the normal mode as an example.
  • step S508 is not limited to that shown in FIG. 5c.
  • the image front-end IFE When the image sensor is running in ultra-low power consumption mode, if the front camera is triggered to start, the image front-end IFE will also generate a low-level power-on switching signal to switch the image sensor to run in normal mode.
  • the image front end IFE sends a low-level power-on switching signal to the image sensor.
  • the image front-end IFE sends a low-level power-on switching signal to the image sensor through pin 2, and the image sensor receives the low-level power-on switching signal and switches to normal mode.
  • the image sensor in the normal mode generates a standard resolution image.
  • the image sensor can also configure the time interval as a period to generate a standard resolution image.
  • the image sensor in the normal mode sends the image data of the standard resolution image to the image front-end IFE.
  • the image sensor in the normal mode can send the image data of the standard resolution image to the image front-end IFE through the MIPI pin.
  • the image front-end IFE processes the image data of the standard resolution image, and displays the processed image data on the display screen.
  • the image front end IFE generates a high-level power-on switching signal.
  • the electronic device responds to the user's operation and executes a corresponding process.
  • the front camera of the electronic device is triggered to be turned off by a control command, and the image sensor needs to exit the normal mode.
  • the image front-end IFE generates a high-level power-on switching signal to control the image sensor to exit the normal mode.
  • the image front-end IFE generates a high-level power-on switching signal, which can control the image sensor to exit from the normal mode and switch to an ultra-low power consumption mode or a lower power consumption mode.
  • the front camera of the electronic device is turned off by a control instruction, and a high-level power-on switching signal can also be generated by IFE lite or AP to control the image sensor to enter an ultra-low power consumption mode or a lower power consumption mode .
  • the image front-end IFE generates a high-level power-on switching signal as an example for illustration.
  • the image front end IFE sends a high-level power-on switching signal to the image sensor.
  • the image front-end IFE can send a high-level power-on switching signal to the image sensor through pin 2.
  • the image sensor can directly switch to an ultra-low power consumption mode upon receiving a high-level power-on switching signal.
  • the image sensor receives a high-level power-on switching signal, and can control itself to be in a low power consumption mode or an ultra-low power consumption mode according to the state of the display screen.
  • the front-facing camera of the electronic device is turned off by a control instruction, and the electronic device is in a screen-off state as shown in (d) in FIG. After electrically switching the signal, the image sensor switches to an ultra-low power mode.
  • the front-facing camera of the electronic device is turned off by a control command. As shown in (e) in FIG. The image sensor switches to a lower power consumption mode after the power-on switching signal.
  • the image sensor operates in an ultra-low power consumption mode, and the execution returns to step S501 , and the image sensor operates in a low power consumption mode, and the execution returns to step S506 .
  • the image front-end IFE generates and sends a high-level power-on switching signal to the image sensor, and AP, or IFE, or IFE lite can generate and send a switching signal to the image sensor according to the current state of the electronic device, such as the state of the display screen. Signal.
  • the image sensor receives the high-level power-on switching signal and the switching signal, and combines the high-level power-on switching signal and the switching signal to switch the normal mode to a lower power consumption mode or an ultra-low power consumption mode.
  • the image sensor receives a high-level power-on switching signal and the first switching signal (such as high level), and the image sensor switches the normal mode to a lower power consumption mode; the image sensor receives a high-level The power-on switching signal and the second switching signal (such as low level), the image sensor switches the normal mode to the ultra-low power consumption mode.
  • the pin of the image sensor for receiving the switching signal may also be the pin 2 shown in FIG. 5 b . Based on this, the pin 2 of the image sensor can receive the power-on switching signal and the switching signal in time-division, combined with the high-level power-on switching signal and the switching signal, switch to a lower power consumption mode or an ultra-low power consumption mode.
  • the image sensor may also use different pins to receive the switch signal and the power-on switch signal.
  • the image sensor switches to a lower power consumption mode or an ultra-low power consumption mode based on a switching signal received by two pins and a power-on switching signal.
  • the power consumption of the ultra-low power consumption mode is lower than that of the low power consumption mode.
  • the image sensor operates in an ultra-low power consumption mode. Power consumption can be reduced.
  • the electronic device is not in use may be understood as: the electronic device is in standby, the display screen is off, or the electronic device is in a locked screen state.
  • the automatic exposure control module of the image sensor or the automatic exposure module of the controller can adjust the exposure parameters according to the brightness and darkness of the image generated by the image sensor to make the image sensor Generate an image with the desired brightness.
  • the control logic of the virtual image sensor is configured, which can be understood as a virtual image sensor in the ultra-low power mode.
  • the switching signal generated by IFE lite, IFE or AP proposed in the foregoing content to control the switching operation of the image sensor between the ultra-low power consumption mode and the low power consumption mode belongs to the category that the image sensor operates between the ultra-low power consumption mode and the relatively low power consumption mode.
  • Passive switching of low-power modes Passive switching can be understood as: the switching of the working mode of the image sensor needs to be controlled by whether it receives the switching signal sent by IFE lite, IFE or AP. When receiving the switching signal, the image sensor switches the working mode according to the switching signal.
  • the image sensor can be switched between the ultra-low power consumption mode and the low power consumption mode, and a control method of active switching of the image sensor can also be adopted. Active switching can be understood as the image sensor actively switches between ultra-low power mode and low power mode, without being limited by the switching signal sent by IFE lite, IFE or AP, as follows:
  • the image sensor uses the image data of the ultra-low resolution image to detect the change of light and shadow, and obtain the detection result of the change of light and shadow.
  • the image sensor determines that the front camera of the electronic device has detected the light and shadow change by using the detection result of the light and shadow change, and then switches the image sensor to operate in a lower power consumption mode.
  • the image sensor determines that the front camera of the electronic device does not detect light and shadow changes, and then controls the image sensor to continue in the ultra-low power consumption mode.
  • the image sensor is in a lower power consumption mode and generates a lower resolution image.
  • the image sensor can also use the image data of the lower resolution image to determine the exit condition that meets the execution event of the processor, such as executing the AO scheme, the image sensor, If it is continuously determined that no human eye is looking at the display screen in the image, the image sensor can continue to operate in a low power consumption mode, or control the image sensor to switch to an ultra-low power consumption mode.
  • another embodiment of the present application also provides an electronic device.
  • the electronic device of this embodiment has a processor that includes a controller , image signal processor ISP, application processor AP and intelligent sensor hub (sensor hub).
  • the intelligent sensor hub provides a solution based on a combination of software and hardware based on a low-power MCU and a lightweight RTOS operating system. Its main function is to connect and process data from various sensor devices.
  • the smart sensor hub is a conventional component in the processor, and common processors are all provided with the smart sensor hub.
  • the sensor hub can connect pin 1 and pin 2, and receive the detection result sent by the image sensor through pin 1, as indicated by high level or low level, and can also generate a switching signal or a power-on switching signal, and pass the pin Pin 2 sends a switching signal or a power-on switching signal to the image sensor.
  • the sensor hub has low power consumption, therefore, the switching signal or power-on switching signal generated by the sensor hub has the advantage of low power consumption.
  • the image sensor when the electronic device is powered on, the image sensor enters the ultra-low power consumption mode by default, or usually enters the ultra-low power consumption mode under the control of the camera driver.
  • the electronic device is in standby, the display is off, and the image sensor is also operating in an ultra-low power mode.
  • Fig. 7a shows the components included in the processor of the electronic device, and three working modes of the image sensor.
  • the image sensor is driven by the signal generated by the components in the processor, and the specific way of switching among the three working modes is shown in Fig. 7b.
  • the image sensor in the ultra-low power consumption mode executes the aforementioned steps S501 and S502 to obtain the detection result of the light and shadow change, and transmits the detection result of the light and shadow change to the sensor hub.
  • the sensor hub After the sensor hub receives the detection result of the light and shadow change, it can use the detection result of the light and shadow change to determine whether the front camera of the electronic device detects the light and shadow change. If it is determined that the front camera of the electronic device detects a change in light and shadow, a switching signal is generated and sent to the image sensor. The image sensor switches to a lower power consumption mode in response to the switching signal.
  • the sensor hub determines that the front camera of the electronic device detects light and shadow changes, and the switching signal may also be generated by the AP, or IFE, or IFE lite.
  • the switching signal generated by the sensor hub is taken as an example for illustration.
  • the image sensor in the lower power consumption mode executes the aforementioned steps S506 and S507, and sends the image data of the lower resolution image to the simplest image front end IFE lite.
  • the sensor hub can generate a low-level power-on switching signal, and send the low-level power-on switching signal to the image sensor through pin 2, and control the working mode of the image sensor to switch to normal model.
  • the image sensor in the normal mode can execute the aforementioned step S510 and step S512.
  • the sensor hub can generate a high-level power-on switching signal, and send the high-level power-on switching signal to the image sensor through pin 2, and control the working mode of the image sensor to switch to super low power mode or lower power mode.
  • the image sensor can directly switch to an ultra-low power consumption mode upon receiving a high-level power-on switching signal.
  • the image sensor receives a high-level power-on switching signal, and can control itself to be in a low power consumption mode or an ultra-low power consumption mode according to the state of the display screen.
  • the front-facing camera of the electronic device is turned off by a control instruction, and the electronic device is in a screen-off state. After the image sensor receives a high-level power-on switching signal sent by the sensor hub, the image sensor switches to an ultra-low power consumption mode.
  • the front camera of the electronic device is turned off by a control instruction, the electronic device is in a bright screen state, and after the image sensor receives a high-level power-on switching signal sent by the sensor hub, the image sensor switches to a lower power consumption mode. model.
  • the sensor hub sends a high-level power-on switching signal to the image sensor.
  • the image sensor can switch from normal mode to other working modes, AP, or IFE, or IFE lite, or sensor hub can
  • the current state of the device such as the state of the display, generates a toggle signal.
  • the image sensor receives the high-level power-on switching signal and the switching signal, combines the high-level power-on switching signal and the switching signal, and switches to a lower power consumption mode or an ultra-low power consumption mode.
  • the aforementioned high-level or low-level power-on switching signal can also be generated by AP, or IFE, or IFE lite, and sent to the image sensor.
  • the sensor hub can also reset the image sensor, that is, control the image sensor to initialize to an ultra-low power consumption mode.
  • an embodiment of the present application provides an electronic device, the electronic device includes a processor and an image sensor that can interact with the processor, and the image sensor is generally arranged at the front of the electronic device camera.
  • the hardware structure and software architecture of the electronic device may be as described above, and will not be described here.
  • the processor includes a controller, an image signal processor ISP and an application processor AP, and the ISP includes at least two integrated units of the image front end IFE and the simplest image front end IFE lite.
  • the controller can include three logic units: AO module, automatic exposure module and camera driver. The functions of AO module, automatic exposure module and camera driver are as described above, and will not be explained here.
  • the image sensor has two working modes: normal mode and lower power consumption mode.
  • the normal mode belongs to the normal working mode of the image sensor.
  • the resolution of the image generated in this mode is the resolution configured by the image sensor, for example, it can be 4208* 3120
  • the lower power consumption mode has lower power consumption than the normal mode
  • the image sensor is in the lower power consumption mode, and can generate lower resolution images, usually 320*240, 520*392, 640*480, 720*540, 800*600, 960*720, 1040*784.
  • the normal mode can also be called the first working mode as described in the first embodiment, and the lower power consumption mode can also be called the second working mode.
  • the first working mode can also be called the first working mode as described in the first embodiment
  • the lower power consumption mode can also be called the second working mode.
  • the image sensor and the processor are respectively provided with a MIPI interface and an I2C interface.
  • the image sensor is provided with an I2C control module connected to the I2C interface, and the I2C control module can interact with the processor through the I2C interface.
  • the I2C control module can be understood as a logic module provided in the image sensor, and can also be understood as a hardware component of the image sensor.
  • the camera driver of the controller, or the IFE lite or IFE in the ISP of the processor, or the AP of the processor generates an I2C address, and sends the I2C address through the I2C interface of the processor, and the I2C address is passed through the image sensor.
  • the I2C interface is transmitted to the image sensor, and the image sensor switches the working mode according to the I2C address.
  • the AP generates an I2C address as an example for description.
  • the AP issues different I2C addresses to control the image sensor to enter different working modes.
  • the AP is connected to the image sensor through the I2C bus, the image sensor is equipped with two I2C control modules, and the address settings of the two I2C control modules are different, the AP sends the I2C address, and the address setting The I2C control module with the I2C address issued by the AP is triggered and can transmit the control signal to the image sensor.
  • the image sensor detects the I2C control module that transmits the control signal and enters the corresponding working mode.
  • the address of one I2C control module is set to 1, and the address of the other I2C control module is set to 2.
  • the AP issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters the normal mode.
  • the AP issues an I2C address of 2, and the I2C control module with the address set to 2 transmits a control signal to the image sensor, and the image sensor enters a lower power consumption mode.
  • the AP is connected to the image sensor through the I2C bus, the image sensor is provided with an I2C control module, and one I2C control module is provided with two addresses, and the two addresses are time-division multiplexed.
  • the AP issues an I2C address, the I2C control module is triggered, and can transmit control signals to the image sensor, and the image sensor detects the I2C control module that transmits the control signal, and enters the working mode corresponding to the address.
  • the address of the I2C control module is represented by a value 1 and a value 2 for the convenience of description.
  • the address of the I2C control module can be represented by one byte of binary data, and the first 7 digits of the one byte of binary data can be used to indicate the address of the I2C control module.
  • the MIPI interface of the image sensor which can transmit the image data of the image generated by the image sensor.
  • the image data can be transmitted to the processor's ISP via the processor's MIPI interface.
  • the image front-end IFE in the ISP acquires the image data received by the MIPI interface of the processor; in other embodiments, the simplest image front-end IFE lite in the ISP acquires the image data received by the MIPI interface of the processor.
  • the functions of the image sensor in the normal mode and the lower power consumption mode, as well as the interaction process with the processor can refer to the content of the first embodiment above, and will not be described here.
  • the processor uses a method of issuing different I2C addresses, instead of using a power-on switching signal to control the image sensor to switch working modes, and the control logic is simple.
  • the embodiment of the present application provides another electronic device, which includes a processor and an image sensor that can interact with the processor.
  • the image sensor is generally set For the front camera of the electronic device.
  • the hardware structure and software architecture of the electronic device may be as described above, and will not be described here.
  • the processor includes a controller, an image signal processor ISP and an application processor AP, and the ISP includes at least two integrated units of an image front end IFE and a simplified image front end IFE lite.
  • the controller can include three logic units: AO module, automatic exposure module and camera driver. The functions of AO module, automatic exposure module and camera driver are as described above, and will not be explained here.
  • the image sensor is provided with three working modes: normal mode, lower power consumption mode and ultra-low power consumption mode, which are specifically as in the first embodiment above, and will not be described here.
  • the ultra-low power consumption mode and the lower power consumption mode may be collectively referred to as a low power consumption awareness mode.
  • the image sensor needs to be switched among three working modes: normal mode, lower power consumption mode and ultra-low power consumption mode, which will be described in detail below.
  • the image sensor and the processor are respectively provided with a MIPI interface, pin 1, pin 2 and an I2C interface.
  • the function of the MIPI interface and the pin 1 set by the image sensor and the processor can be the same as that of the first embodiment above, and will not be described here.
  • the normal mode corresponds to one I2C address
  • the low-power sensing mode corresponds to another I2C address.
  • the I2C interface of the processor and the image sensor is used to transmit the I2C address for switching the image sensor to operate in normal mode, or the I2C address to operate in low-power sensing mode.
  • the image sensor is provided with an I2C control module connected to the I2C interface, and the I2C control module can interact with the processor through the I2C interface.
  • the I2C control module can be understood as a logic module provided in the image sensor, and can also be understood as a hardware component of the image sensor.
  • the camera driver of the controller, or the IFE lite or IFE in the ISP of the processor, or the AP of the processor generates an I2C address, and sends the I2C address through the I2C interface of the processor, and the I2C address is passed through the image sensor.
  • the I2C interface is transmitted to the image sensor, and the image sensor switches between the low-power sensing mode and the normal mode according to the I2C address.
  • the AP generates an I2C address as an example for description.
  • the AP issues different I2C addresses to control the image sensor to enter the low-power sensing mode or normal mode.
  • the AP is connected to the image sensor through the I2C bus, and the image sensor is provided with two I2C control modules.
  • the addresses of the two I2C control modules are set differently.
  • the I2C control module with the issued I2C address is triggered, and can transmit the control signal to the image sensor, and the image sensor detects the I2C control module that transmits the control signal, and enters the corresponding working mode.
  • the address of one I2C control module is set to 1, and the address of the other I2C control module is set to 2.
  • the AP issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters the normal mode.
  • the AP issues an I2C address of 2, and the I2C control module with the address set to 2 transmits a control signal to the image sensor, and the image sensor enters a low-power sensing mode.
  • the AP is connected to the image sensor through the I2C bus, the image sensor is provided with an I2C control module, and one I2C control module is provided with two addresses, and the two addresses are time-division multiplexed.
  • the AP issues an I2C address, the I2C control module is triggered, and can transmit control signals to the image sensor, and the image sensor detects the I2C control module that transmits the control signal, and enters the working mode corresponding to the address.
  • Pin 2 of the image sensor is used to transmit a one-bit level signal. Similarly, pin 2 of the processor is also used to transmit a bit level signal. In some embodiments, the pin 2 of the image sensor is a switching pin for receiving a switching signal for switching the image sensor to operate in a low power consumption mode or an ultra-low power consumption mode. In some examples, pin 2 of the image sensor may be referred to as a PONV pin, Xshutdown2 pin.
  • the IFE lite or IFE in the ISP of the processor, or the AP of the processor can generate a switching signal, send the switching signal through the pin 2 of the processor, and then transmit the switching signal to the pin 2 of the image sensor.
  • An image sensor the image sensor switches between a low power consumption mode and an ultra-low power consumption mode according to the switching signal.
  • the image sensor and the processor are respectively provided with a MIPI interface, pin 1 and an I2C interface.
  • the function of the MIPI interface and the pin 1 set by the image sensor and the processor can be the same as that of the first embodiment above, and will not be described here.
  • the function of the I2C interface may be the same as that in the foregoing implementation manner, and will not be further described here.
  • the ultra-low power consumption mode and the lower power consumption mode of the image sensor are switched actively, as follows:
  • the image sensor operates in an ultra-low power consumption mode, uses the image data of an ultra-low-resolution image to detect light and shadow changes, and obtains the detection results of light and shadow changes.
  • the image sensor determines that the front camera of the electronic device has detected the light and shadow change by using the detection result of the light and shadow change, and then switches the image sensor to operate in a lower power consumption mode.
  • the image sensor determines that the front camera of the electronic device does not detect light and shadow changes, and then controls the image sensor to continue in the ultra-low power consumption mode.
  • the image sensor is in a lower power consumption mode and generates a lower resolution image.
  • the image sensor can also use the image data of the lower resolution image to determine the exit condition that meets the execution event of the processor, such as executing the AO scheme, the image sensor, If it is continuously determined that no human eye is looking at the display screen in the image, the image sensor can continue to operate in a low power consumption mode, or control the image sensor to switch to an ultra-low power consumption mode.
  • the ultra-low power consumption mode and the lower power consumption mode of the image sensor are switched by the image sensor according to the detection result of light and shadow changes, and the normal mode and the low power consumption sensing mode of the image sensor are switched by the processor Different I2C addresses are used for switching, so that simple control logic can be used to complete the switching of the working mode of the image sensor.
  • this possible implementation manner is taken as an example for description.
  • Figure 9d shows how the image sensor is driven by signals generated by components in the processor, and switches between normal mode, ultra-low power mode and lower power mode.
  • Fig. 9d (for simplicity of drawing, Fig. 9d does not show the pin 1 of processor, pin 2, MIPI interface and I2C interface, image sensor also does not show I2C interface and pin 2, and the I2C control module is shown Outside the image sensor, but it does not constitute a limitation on the internal structure of the processor and the image sensor), the electronic device is powered on, and the image sensor enters the ultra-low power consumption mode by default, or enters the ultra-low power consumption mode under the control of the camera driver. Alternatively, the electronic device is in standby, the display is off, and the image sensor is operating in an ultra-low power mode.
  • the image sensor in the ultra-low power consumption mode will generate an ultra-low-resolution image every other time interval, and use the image data of the ultra-low-resolution image to detect changes in light and shadow, and pass the detection results of light and shadow changes through the index Pin 1 is input to IFE lite.
  • IFE lite can use the detection results of light and shadow changes to execute some low-power events. If the front camera detects light and shadow changes, the display will not turn off, the ringtone volume will be reduced, and the display will be automatically rotated according to the direction of the face. Trigger screen capture/play pause, page turning and other functions.
  • the image sensor uses the detection result of the light and shadow change to determine that the front camera of the electronic device detects the light and shadow change, then generates and sends a switching signal to the image sensor, and switches the mode of the image sensor from an ultra-low power consumption mode to a lower power consumption mode.
  • the image sensor in the lower power consumption mode will also generate a lower resolution image every other time interval, and send the image data of the lower resolution image to the IFE lite through the MIPI pin.
  • the image data of the lower resolution image output by IFE lite is stored in the safe buffer of the internal memory.
  • the processor uses the image data of the lower-resolution image to perform various events, such as the detection of human eyes looking at the display screen proposed by the controller (also called the AO scheme), or to control the display screen according to the direction of the face. Auto-rotate, trigger screen capture/pause playback, turn pages, etc.
  • the AP issues the I2C address 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters the normal mode.
  • the image sensor in normal mode generates a standard resolution image every other time interval, and sends the image data of the standard resolution image to the IFE through the MIPI pin.
  • the IFE processes the image data of the standard resolution image, and displays the processed image data on the display screen.
  • the AP sends the I2C address 2 to control the image sensor to switch from the normal mode to the low power sensing mode.
  • the image sensor receives the I2C address 2
  • the I2C control module whose address is set to 2 transmits a control signal to the image sensor, and controls the image sensor to enter a lower power consumption mode or an ultra-low power consumption mode, and the following three implementation modes are possible:
  • the I2C control module whose address is set to 2 transmits a control signal to the image sensor, and the image sensor directly switches to an ultra-low power consumption mode.
  • the image sensor controls itself to switch to a lower power consumption mode or an ultra-low power consumption mode in combination with the I2C address issued by the AP and the state of the display screen.
  • the image sensor can be switched to a lower power consumption mode, or it can be switched to an ultra-low power consumption mode.
  • the image sensor controls itself to switch to a lower power consumption mode or an ultra-low power consumption mode in combination with the I2C address issued by the AP and the switching signal transmitted by pin 2.
  • the switching signal transmitted by pin 2 can be generated by the IFE lite or IFE in the ISP of the processor, or the AP of the processor, according to the current state of the electronic device, such as the state of the display screen. If the display screen is off, the generated switching signal is used to switch the image sensor to an ultra-low power consumption mode.
  • the use of the value 1 and the value 2 to represent the address of the I2C control module is for the convenience of expression and does not constitute a limitation on the specific implementation of the address of the I2C control module.
  • the address of the I2C control module can be represented by one byte of binary data, and the first 7 digits of the one byte of binary data can be used to indicate the address of the I2C control module.
  • the address of the I2C control module proposed in the following content will also be described by taking a numerical value as an example. Similarly, the numerical value proposed in the following content does not constitute a limitation on the specific implementation of the address of the I2C control module.
  • the electronic device provided in this embodiment can also use the sensor hub in the processor to receive the image sensor in the ultra-low power consumption mode, as shown in Figure 10a.
  • FIG. 10b shows the switching implementation of the image sensor shown in FIG. 10a in the normal mode, the lower power consumption mode and the ultra-low power consumption mode.
  • Figure 10b for simplicity of drawing, Figure 10b does not show the pin 1, pin 2, MIPI interface and I2C interface of the processor, and the image sensor does not show the I2C interface and pin 2, and the I2C control
  • the module is shown outside the image sensor, but it does not constitute a limitation on the internal structure of the processor and image sensor), after the sensor hub receives the detection result of the light and shadow change, it can use the detection result of the light and shadow change to determine the front camera of the electronic device Whether to detect light and shadow changes. If it is determined that the front camera of the electronic device detects a change in light and shadow, a switching signal is generated and sent to the image sensor.
  • the image sensor switches from the ultra-low power mode to a lower power mode in response to the switching signal.
  • the sensor hub determines that the front camera of the electronic device detects light and shadow changes, and the switching signal may also be generated by the AP, or IFE, or IFE lite.
  • the sensor hub can also be used to send different I2C addresses to send power-on switching signals, and control the image sensor to enter low-power sensing mode or normal mode.
  • the front camera of the electronic device is activated, and the sensor hub can issue the I2C address 1, and the I2C control module with the address set to 1 transmits the control signal to the image sensor, and the image sensor enters the normal mode.
  • the front camera of the electronic device is turned off, and the sensor hub can issue I2C address 2 to control the image sensor to switch from normal mode to low-power sensing mode.
  • the image sensor receives the I2C address 2, the I2C control module whose address is set to 2 transmits the control signal to the image sensor, and the way of controlling the image sensor to enter the low power consumption mode or the ultra-low power consumption mode can be as mentioned above. An implementation manner, no further description will be given here.
  • the image sensor and the processor are respectively provided with a MIPI interface, pin 1, pin 2 and an I2C interface.
  • the functions of pin 1 and the MIPI interface are the same as those described above, and will not be described here.
  • the pin 2 can transmit a switching signal for switching between the low-power sensing mode and the normal mode of the image sensor, and the switching signal includes high and low levels.
  • the processor transmits a high level through its own pin 2 and the image sensor's pin 2, and the image sensor is running in a low-power sensing mode, and the processor transmits Low level, the image sensor operates in normal mode.
  • the different I2C addresses transmitted by the I2C interface are used to control the switching operation between the ultra-low power consumption mode and the lower power consumption mode of the image sensor, one I2C address corresponds to the ultra-low power consumption mode, and the other The I2C address corresponds to a lower power mode.
  • the image sensor can be provided with two I2C control modules or one I2C control module.
  • Control the image sensor to switch between low-power sensing mode and normal mode with high and low-level switching signals, which can ensure that all modules in the image sensor are only started in normal mode, and the image sensor runs in low-power sensing mode , which can be operated only by some low-power modules, which can ensure low power consumption of the image sensor.
  • switching the image sensor into the normal mode by using high and low level switching signals also has the advantage that all modules can start up quickly after the image sensor receives the level signal, ensuring the performance of the image sensor.
  • the I2C interface transmits an I2C address plus different control signals, which are used to control the switching operation between the ultra-low power consumption mode and the lower power consumption mode of the image sensor.
  • the image sensor can be provided with an I2C control module.
  • the I2C interface transmits an I2C address 1 plus a first control signal to control the image sensor to operate in a lower power consumption mode, and the I2C interface transmits an I2C address 1 plus a second control signal to control the image sensor to operate in an ultra-low power consumption mode .
  • the processor sends the I2C address plus the control signal, which can be understood as the processor sends information carrying the I2C address and the control signal to the image sensor through the I2C bus.
  • the control signal can be understood as a switching command signal, the first control signal is used to control the image sensor to operate in a low power consumption mode, and the second control signal is used to control the image sensor to operate in a super power consumption mode.
  • the I2C interface of the image sensor receives the I2C address 1 and the control signal, the I2C control module with the address set to 1 is triggered, and further transmits the control signal to the image sensor, the image sensor analyzes the control signal, determines the working mode specified by the control signal, and switches the image sensor Enter the working mode specified by the control signal.
  • the way the processor controls the image sensor to operate in normal mode can also be that the processor transmits the level value through pin 2, and transmits the I2C address through the I2C interface, and the I2C address can control the ultra-low power consumption mode The same I2C address is used to switch between lower power modes, but this I2C address and normal control signals can be sent by the processor to the image sensor.
  • a conventional control signal can be understood as a signal that does not have the function of controlling the image sensor to operate in a certain working mode.
  • the control logic is simple. Therefore, the switching signal for controlling the switching of the image sensor between the ultra-low power consumption mode and the lower power consumption mode can also be sent by using different I2C addresses.
  • the switching between the normal mode of the image sensor and the low-power sensing mode can be realized by using different I2C addresses, and the switching between the low-power sensing modes of the image sensor can be It is realized by adding different control signals to the same I2C address.
  • the controller can be driven by the camera of the controller, or the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor, and send the I2C address to the image sensor, or the I2C address plus different control signals.
  • the processor generates and sends an I2C address 1 to the image sensor to control the image sensor to operate in normal mode; the processor generates and sends an I2C address 2 plus a control signal to the image sensor to control the image sensor to operate in a low-power sensing mode ; Specifically, the processor generates and sends I2C address 2 and the first control signal to the image sensor to control the image sensor to operate in a lower power consumption mode, and the processor generates and sends I2C address 2 and the second control signal to the image sensor to control The image sensor operates in ultra-low power mode.
  • the first control signal and the second control signal are switching command signals, which can designate the image sensor to operate in a certain working mode.
  • the I2C interface of the image sensor receives the I2C address 1, and the I2C control module with the address set to 1 is triggered to control the image sensor to run in normal mode.
  • the I2C interface of the image sensor receives the I2C address 2 and the control signal, the I2C control module with the address set to 2 is triggered, and further transmits the control signal to the image sensor, the image sensor analyzes the control signal, determines the working mode specified by the control signal, and switches the image sensor Enter the working mode specified by the control signal.
  • the processor controls the image sensor to run in the normal mode, or it can use I2C address 1 to add a control signal
  • the control signal can be a switching command signal for specifying the image sensor to run in the normal mode, or it can be General control signal.
  • each of the three working modes of the image sensor corresponds to one I2C address.
  • the processor is connected to the image sensor through an I2C bus, and the image sensor is provided with an I2C control module.
  • the AP issues different I2C addresses, the I2C control module whose address is set to the I2C address issued by the AP is triggered, and the image sensor is controlled to enter the working mode corresponding to the I2C address issued by the AP.
  • I2C addresses can also be generated and issued by the camera driver of the controller or IFE lite or IFE in the ISP of the processor.
  • the following uses AP as an example for description.
  • two I2C control modules are set in the image sensor, the address of one I2C control module corresponds to the normal mode of the image sensor, one address of one I2C control module corresponds to the lower power consumption mode, and the other address Corresponds to ultra-low power consumption mode.
  • the address of one I2C control module is set to 1, the address of the other I2C control module is set to 2 and 3, and the two addresses are time-division multiplexed.
  • the AP issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters the normal mode.
  • the AP issues an I2C address of 2, and the I2C control modules whose addresses are set to 2 and 3 transmit control signals to the image sensor, and the image sensor enters a lower power consumption mode corresponding to address 2.
  • the AP issues an I2C address of 3, and the I2C control modules whose addresses are set to 2 and 3 transmit control signals to the image sensor, and the image sensor enters a lower power consumption mode corresponding to address 3.
  • three I2C control modules are set in the image sensor, the address of one I2C control module corresponds to the normal mode of the image sensor, the address of one I2C control module corresponds to the lower power consumption mode, and the address of one I2C control module The address corresponds to the ultra-low power mode.
  • the address of one I2C control module is set to 1, the address of another I2C control module is set to 2, and the address of another I2C control module is set to 3.
  • the AP issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters the normal mode.
  • the AP issues an I2C address of 2, and the I2C control module with the address set to 2 transmits a control signal to the image sensor, and the image sensor enters a lower power consumption mode.
  • the AP issues an I2C address of 3 and the I2C control module with the address set to 3 transmits a control signal to the image sensor, and the image sensor enters a lower power consumption mode.
  • the sensor hub of the processor When the sensor hub of the processor is connected to the I2C interface of the processor, the sensor hub can also be used to issue different I2C addresses to control the corresponding I2C control module to be triggered, and the image sensor is controlled to enter the work corresponding to the issued I2C address model.
  • two I2C control modules are set in the image sensor, the address of one I2C control module corresponds to the normal mode of the image sensor, one address of one I2C control module corresponds to the lower power consumption mode, and the other address corresponds to the ultra-low power mode. consumption mode.
  • the address of one I2C control module is set to 1, the addresses of the other I2C control module are set to 2 and 3, and the two addresses are time-division multiplexed.
  • the sensor hub issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters normal mode.
  • the sensor hub sends the I2C address to 2, and the I2C control module with the address set to 2 and 3 transmits the control signal to the image sensor, and the image sensor enters the lower power consumption mode corresponding to address 2.
  • the sensor hub sends the I2C address 3, and the I2C control module with the address set to 2 and 3 transmits the control signal to the image sensor, and the image sensor enters the lower power consumption mode corresponding to the address 3.
  • three I2C control modules are set in the image sensor, the address of one I2C control module corresponds to the normal mode of the image sensor, the address of one I2C control module corresponds to the lower power consumption mode, and one I2C control module The address corresponds to the ultra-low power mode.
  • the address of one I2C control module is set to 1, the address of another I2C control module is set to 2, and the address of another I2C control module is set to 3.
  • the sensor hub issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters normal mode.
  • the sensor hub issues an I2C address of 2, and the I2C control module with the address set to 2 transmits control signals to the image sensor, and the image sensor enters a lower power consumption mode.
  • the sensor hub issues an I2C address of 3, and the I2C control module with the address set to 3 transmits control signals to the image sensor, and the image sensor enters a lower power consumption mode.
  • Another embodiment of the present application also provides an electronic device, the electronic device includes a processor and an image sensor capable of interacting with the processor, and the image sensor is generally arranged on a front camera of the electronic device.
  • the hardware structure and software architecture of the electronic device may be as described above, and will not be described here.
  • the processor includes a controller, an image signal processor ISP, an intelligent sensor hub sensor hub and an application processor AP, and the ISP includes at least an image front end IFE and a minimal image front end IFE lite Two integrated units.
  • the controller can include three logic units: AO module, automatic exposure module and camera driver. The functions of the AO module, the automatic exposure module and the camera driver can be as described above.
  • the image sensor includes four working modes, namely: a first working mode, a second working mode, and a fourth working mode.
  • the first working mode may also be referred to as a normal working mode or a normal mode
  • the resolution of an image generated in the first working mode is the first image resolution, for example, may be 4208*3120.
  • the second working mode can also be called a lower power consumption mode, the power consumption of the second working mode is lower than that of the first working mode, and the resolution of the image generated in the second working mode is the second image resolution, Usually it can be 320*240, 520*392, 640*480, 720*540, 800*600, 960*720, 1040*784.
  • the third working mode may also be called an ultra-low power consumption mode.
  • the third working mode has the lowest power consumption, and the resolution of the image generated in the third working mode is the third image resolution, usually 16*12.
  • the fourth working mode can also be called a low power consumption mode.
  • the low power consumption mode has lower power consumption than the low power consumption mode.
  • the resolution of the image generated in the fourth working mode is the fourth image resolution, usually: 64* 48, 72*54, 80*60, 96*72, 120*90, 128*96. Therefore, it can be seen that: first image resolution>second image resolution>fourth image resolution>third image resolution.
  • the image sensor and the processor are respectively provided with MIPI interface, pin 1, pin 2 and I2C interface.
  • MIPI interface The function of the MIPI interface and the pin 1 set by the image sensor and the processor can be the same as that of the first embodiment above, and will not be described here.
  • the four working modes of the image sensor can be divided into two groups, the first group includes normal mode and lower power consumption mode, and the second group includes low power consumption mode and ultra-low power consumption mode.
  • the processor and the I2C interface of the image sensor are connected through the I2C bus, and are used to transmit the I2C address, and the I2C address is used to control the image sensor to operate in one working mode of the first group or one working mode of the second group.
  • the camera driver of the controller, or the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor can generate and send different I2C addresses through the I2C interface of the processor, and the I2C address can then be
  • the image sensor is transmitted to the image sensor through the I2C interface of the image sensor, and the image sensor switches to the working mode corresponding to the I2C address.
  • the generation of the I2C address by the sensor hub is taken as an example for illustration.
  • the image sensor is provided with an I2C control module connected to the I2C interface, and the I2C control module can interact with the processor through the I2C interface to control switching of the working mode of the image sensor.
  • the I2C control module can be understood as a logic module provided on the image sensor, and can also be understood as a hardware component provided on the image sensor.
  • I2C control module in the image sensor can have the following two implementations:
  • the image sensor is provided with two I2C control modules, and the address settings of the two I2C control modules are different, the sensor hub issues the I2C address, and the address is set to be issued by the sensor hub
  • the I2C control module with the I2C address is triggered, and can transmit the control signal to the image sensor, and the image sensor detects the I2C control module that transmits the control signal, and enters the corresponding working mode.
  • the address of one I2C control module is set to 1, and the address of the other I2C control module is set to 2.
  • the sensor hub issues an I2C address of 1, and the I2C control module with the address set to 1 transmits control signals to the image sensor, and the image sensor can enter normal mode or lower power consumption mode.
  • the sensor hub sends the I2C address to 2, and the I2C control module with the address set to 2 transmits the control signal to the image sensor, and the image sensor can enter the ultra-low power mode or low power mode. This embodiment is described with this example.
  • the image sensor is provided with an I2C control module, and the I2C control module is provided with two addresses, the two addresses are time-division multiplexed, one address corresponds to the normal mode and the lower power consumption mode, and the other The address corresponds to low power mode and ultra low power mode.
  • the sensor hub issues an I2C address of the I2C control module.
  • the I2C control module is triggered and can transmit control signals to the image sensor.
  • the image sensor detects the I2C control module that transmits the control signal and enters the corresponding working mode of the I2C address.
  • Pin 2 of the image sensor is used to transmit a one-bit level signal.
  • pin 2 of the processor is also used to transmit a bit level signal.
  • pin 2 of the image sensor may be referred to as a PONV pin, Xshutdown2 pin.
  • pin 2 of the image sensor is a switching pin, which is used to receive switching signal 1 and switching signal 2 for switching the working mode of the image sensor, and switching signal 1 is used to control the image sensor to operate in normal mode or with lower power consumption. mode operation, switching signal 2 is used to control the low power mode or ultra-low power mode operation of the image sensor.
  • Switching signal 1 includes high and low levels.
  • a high-level switching signal 1 is used to control the image sensor to enter a lower power consumption mode
  • a low-level switching signal 1 is used to control the image sensor to enter a normal mode.
  • the high-level switching signal 1 may also control the image sensor to switch to the normal mode
  • the low-level switching signal 1 may control the image sensor to enter a lower power consumption mode.
  • it is described by taking the low-level switching signal 1 to control the image sensor to switch to the normal mode, and the high-level switching signal 1 to control the image sensor to switch to the lower power consumption mode as an example.
  • the switching signal 2 also includes high and low levels.
  • the high-level switching signal 2 is used to control the image sensor to enter a low power consumption mode
  • the low-level switching signal 2 is used to control the image sensor to enter an ultra-low power consumption mode.
  • the high-level switching signal 2 may also control the image sensor to switch to the ultra-low power consumption mode
  • the low-level switching signal 2 may control the image sensor to switch to the low power consumption mode.
  • the low-level switching signal 2 controls the image sensor to switch to the low power consumption mode
  • the high-level switching signal 2 controls the image sensor to switch to the ultra-low power consumption mode as an example for illustration.
  • the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor generates the switching signal 1 and the switching signal 2, which are sent through the pin 2 of the processor, and then passed through the pin 2 of the image sensor It is transmitted to the image sensor, and the image sensor switches the working mode according to the switching signal 1 or the switching signal 2.
  • the sensor hub generates switching signal 2 and switching signal 1 as an example for illustration.
  • pins of the image sensor that receive the switching signal 1 and the pins that receive the switching signal 2 may also be different pins.
  • pin 2 of the image sensor receives switching signal 1 and the other pin receives switching signal 2 .
  • the following description takes pin 2 respectively receiving switching signal 1 and switching signal 2 as an example.
  • the image sensor can be configured such that the initial operation mode is an ultra-low power consumption mode. Based on this, the electronic device is turned on and powered on, and the image sensor of the front camera responds to the power-on command and operates in an ultra-low power consumption mode.
  • the electronic device If the image sensor is not configured with an initial operation mode, the electronic device is powered on, and the processor of the electronic device generates a first instruction to control the operation of the front camera. Specifically, the image sensor of the front camera will respond to the first instruction and operate in an ultra-low power consumption mode.
  • the first instruction may be generated by a camera driver module of the controller.
  • the image sensor can switch the working mode according to the different operating states of the electronic device.
  • FIG. 12c for simplicity of drawing, the pins 1 and 2 of the processor are not shown in FIG. 12c ).
  • MIPI interface and I2C interface the image sensor does not show the I2C interface and pin 2
  • the I2C control module is drawn outside the image sensor, but this does not constitute a limitation on the internal structure of the processor and the image sensor) and Figure 12d, Introduce the switching control method of the working mode provided by this embodiment.
  • the working mode switching control method provided by this embodiment is applied to the aforementioned electronic device.
  • the image sensor When the electronic device is powered on, the image sensor enters the ultra-low power consumption mode by default, or usually enters the ultra-low power consumption mode under the control of the camera driver.
  • the switching control method of the working mode provided by this embodiment includes the following steps:
  • the image sensor in the ultra-low power consumption mode generates ultra-low-resolution images at intervals.
  • the electronic device is powered on, the display screen is turned on and a lock screen interface is output, and the image sensor operates in an ultra-low power consumption mode to generate an ultra-low-resolution image at a configured time interval.
  • the image sensor when the electronic device is on standby and the display screen is off, the image sensor will also operate in an ultra-low power consumption mode to generate ultra-low-resolution images at a configured time interval.
  • the image sensor in the ultra-low power consumption mode uses the image data of the ultra-low resolution image to detect light and shadow changes, and obtain a light and shadow change detection result.
  • the image sensor utilizes the image data of the ultra-low resolution image to detect the change of light and shadow, and is used to determine whether there is a change of light and shadow in front of the front camera of the electronic device.
  • the image sensor sends the detection result of light and shadow changes to the sensor hub.
  • the image sensor can input the detection result of light and shadow changes to the sensor hub through pin 1.
  • the sensor hub can use the detection results of light and shadow changes to execute some low-power events. If it is detected that the front camera detects light and shadow changes, the display will not turn off, the ringtone volume will be reduced, and the display will be automatically rotated according to the direction of the face. Trigger screen capture/play pause, page turning and other functions.
  • the sensor hub uses the detection result of the light and shadow change to determine that the front camera of the electronic device detects the light and shadow change, and then generates an I2C address 2 and a low-level switching signal 2 .
  • the sensor hub determines that the front camera of the electronic device detects light and shadow changes, indicating that the user may be using the electronic device and needs to switch the working mode of the image sensor, so the sensor hub generates I2C address 2 and a low-level switching signal 2 to control the image sensor by Ultra-low power mode switches to low power mode.
  • the image sensor will continue to be in an ultra-low power consumption mode.
  • I2C address 2 and switching signal 2 can also be generated by IFE, IFE lite or AP.
  • the IFE, IFE lite or AP interacts with the sensor hub.
  • the detection result of the light and shadow change is used to determine the light and shadow change detected by the front camera of the electronic device, and the I2C address 2 and low Level switching signal 2.
  • the IFE, IFE lite or AP receives the detection result of the light and shadow change, uses the detection result of the light and shadow change to determine that the front camera of the electronic device detects the light and shadow change, and generates the I2C address 2 and the low-level switching signal 2.
  • sensor hub sends I2C address 2 and low-level switching signal 2 to the image sensor.
  • the sensor hub can send a low-level switching signal 2 to the image sensor through pin 2, and transmit the I2C address 2 through the I2C interface.
  • the image sensor receives the I2C address 2 through the I2C interface, and the I2C address 2 is further transmitted to the I2C control module whose address is set to 2 in the image sensor.
  • the I2C control module whose address is set to 2 is triggered and can transmit control signals to the image sensor.
  • the sensor detects the I2C control module that transmits the control signal, and can enter the low-power mode or ultra-low power mode. Further, the image sensor receives the low-level switching signal 2, responds to the low-level switching signal 2, and switches the working mode from the ultra-low power consumption mode to the low power consumption mode.
  • the switching between the ultra-low power consumption mode and the low power consumption mode of the image sensor can also be performed in an active switching manner. Specifically, the image sensor determines that the front camera of the electronic device has detected a change in light and shadow according to the detection result of the change in light and shadow, then switches the working mode to a low power consumption mode, and determines that the front camera of the electronic device has not detected a change in light and shadow, then maintains the ultra-low power consumption mode. low power mode.
  • the image sensor in the low power consumption mode generates a low-resolution image at a time interval.
  • the image sensor uses the image data of the low-resolution image to perform face and/or gesture detection, and obtain a face and/or gesture detection result.
  • the image sensor can use the image data of the low-resolution image to perform face detection, and when a face is detected, it can be determined that the user is using an electronic device.
  • the image sensor can also use the image data of the low-resolution image for gesture detection, and when a pre-configured gesture is detected, it can also determine that the user is using the electronic device.
  • the image sensor can also use the image data of the low-resolution image to perform face detection and gesture detection to obtain the face and gesture detection results.
  • the image sensor can also use the face and/or gesture detection results to determine that the image captured by the front camera of the electronic device does not appear face and/or gesture, maintaining the image sensor in a low-power mode. Or within a period of time, such as 5 seconds, use the face and/or gesture detection results to continuously determine that no face and/or gesture appear in the image captured by the front camera of the electronic device, then control the image sensor to switch to ultra-low power consumption model.
  • the image sensor sends the face and/or gesture detection result to the sensor hub.
  • the detection result of the face or gesture can be sent to the sensor hub.
  • the image sensor can send the face and/or gesture detection results to the sensor hub through pin 1.
  • the sensor hub determines that a human face and/or gesture appears in the image captured by the front camera of the electronic device according to the face and/or gesture detection result, and generates an I2C address 1 and a high-level switching signal 1.
  • the control image sensor can continue to operate in a low power consumption mode.
  • the sensor hub generates and sends an I2C address 2 and a high-level switching signal 2 to the image sensor to control the image sensor to switch to an ultra-low power consumption mode.
  • the sensor hub sends an I2C address 1 and a high-level switching signal 1 to the image sensor, and controls the image sensor to switch to a lower power consumption mode.
  • the image sensor receives the I2C address 1 and the high-level switching signal 1, and combines the I2C address 1 and the high-level switching signal 1 to switch to a lower power consumption mode for operation.
  • the sensor hub can send a high-level switching signal 1 to the image sensor through pin 2, and transmit the I2C address 1 through the I2C interface.
  • the image sensor After the image sensor receives the I2C address 1 through the I2C interface, it transmits to the I2C control module whose address is set to 1 in the image sensor, the I2C control module whose address is set to 1 is triggered, and can transmit the control signal to the image sensor, and the image sensor detects the transmission
  • the I2C control module of the control signal can enter normal mode or lower power consumption mode. Further, the image sensor receives the high-level switching signal 1 to control the image sensor to enter a lower power consumption mode.
  • the image sensor in a lower power consumption mode generates a lower resolution image at intervals.
  • the image sensor in a lower power consumption mode sends image data of a lower resolution image to the simplest image front end IFE lite.
  • the image sensor sends the image data of the lower resolution image to the simplest image front end IFE lite through the MIPI pin.
  • the minimal image front end IFE lite can store image data of lower resolution images in a secure buffer in memory.
  • the controller of the processor can use the image data of the low-resolution image to perform various events, such as the human eye gazing at the display Screen detection (also known as AO solution), control the automatic rotation of the display screen according to the direction of the face, trigger screen capture/play pause, page turning and other functions.
  • the controller of the processor uses the image data of the lower-resolution image to execute various event processes. If the image data of the lower-resolution image is used, it is determined that the exit condition of the executed event is satisfied, such as Executing the AO scheme, the controller continues to determine that none of the human eyes in the image is looking at the display screen as described above, then the image sensor can continue to operate in a lower power consumption mode, or control the image sensor to switch to a mode with lower power consumption than the low power consumption mode. Operating mode.
  • the sensor hub generates an I2C address 2 and a low-level switching signal 2, and sends them to the image sensor to control the image sensor to switch to a low power consumption mode.
  • the sensor hub generates an I2C address 2 and a high-level switching signal 2, and sends them to the image sensor to control the image sensor to switch to an ultra-low power consumption mode. Normally, when the display screen of the electronic device is in the off-screen state, the sensor hub generates an I2C address 2 and a high-level switching signal 2 .
  • IFE, IFE lite or AP can generate I2C address 2
  • IFE, IFE lite or AP can generate switching signal 2
  • different components can be used to generate I2C address 2 and switching signal respectively. signal 2.
  • step S1201 If the image sensor operates in an ultra-low power consumption mode, the aforementioned step S1201 is performed, and if the image sensor operates in a low power consumption mode, the aforementioned step S1206 is performed.
  • sensor hub generates I2C address 1 and low-level switching signal 1.
  • the user activates the front camera of the electronic device, and the display screen of the electronic device displays the shooting mode of the front camera.
  • the electronic device will respond to the user's operation, and the front camera will take pictures to obtain images, and the captured images will be displayed on the display screen.
  • the front camera of the electronic device is activated by a control command, and the image sensor needs to enter a normal mode.
  • the sensor hub will generate an I2C address 1 and a low-level switching signal 1 to control the image sensor to enter the normal mode.
  • step S1213 is not limited to that shown in FIG. 12d.
  • the image front-end IFE When the image sensor is running in ultra-low power mode or low power mode, if the front camera is triggered to start, the image front-end IFE will also generate a low-level switching signal 1 to switch the image sensor to run in normal mode.
  • sensor hub sends I2C address 1 and low-level switching signal 1 to the image sensor.
  • the sensor hub sends the low-level switching signal 1 to the image sensor through pin 2, and transmits the I2C address 1 through the I2C interface.
  • the I2C control module with the address set to 1 in the image sensor is in a triggered state and can transmit control signals
  • the image sensor detects the I2C control module that transmits the control signal, and can enter the normal mode or the lower power consumption mode. Further, the image sensor switches to the normal mode upon receiving the low-level switching signal 1 .
  • the image sensor in the normal mode generates a standard resolution image.
  • the image sensor can also configure the time interval as a period to generate a standard resolution image.
  • the image sensor in the normal mode sends the image data of the standard resolution image to the image front-end IFE.
  • the image sensor in the normal mode can send the image data of the standard resolution image to the image front-end IFE through the MIPI pin.
  • the image front-end IFE processes the image data of the standard resolution image, and displays the processed image data on the display screen.
  • sensor hub generates I2C address 1 and high level switching signal 1.
  • the electronic device responds to the user's operation and executes a corresponding process. Specifically, the front camera of the electronic device is triggered to be turned off by a control command, and the image sensor needs to be switched from a normal mode to another working mode.
  • the sensor hub can generate an I2C address 1 and a high-level switching signal 1 to control the image sensor to switch from the normal mode to the lower power consumption mode.
  • sensor hub sends I2C address 1 and high-level switching signal 1 to the image sensor.
  • the I2C address 1 will be transmitted to the image sensor through the I2C interface, and the I2C control module with the address set to 1 transmits the control signal to the image sensor, and the image sensor Combining I2C address 1 and high-level switching signal 1, switch to a lower power consumption mode.
  • the image sensor switches to a lower power consumption mode and operates in a lower power consumption mode, and the aforementioned step 1211 can be performed.
  • the front camera of the electronic device is turned off by the control instruction, and the processor can also switch the working mode of the image sensor according to the current state of the electronic device.
  • the sensor hub when the display screen of the electronic device is in the off-screen state, the sensor hub generates an I2C address 2 and a high-level switching signal 2 to control the image sensor to switch from the normal mode to the ultra-low power consumption mode.
  • the sensor hub when the display screen of the electronic device is in the off-screen state, the sensor hub generates an I2C address 2 and a low-level switching signal 2 to control the image sensor to switch from the normal mode to the low power consumption mode.
  • the automatic exposure module or the automatic exposure control module of the image sensor can adjust the exposure parameters according to the brightness and darkness of the image generated by the image sensor.
  • the image sensor produces an image with the required brightness.
  • I2C address 1 can correspond to any one of the first group of working modes
  • I2C address 2 can correspond to the second group of working modes Any of the modes
  • the lower power consumption mode can be determined by switching signal 1, or can be determined by I2C address and switching signal 1
  • the low power consumption mode can be determined by switching signal 2, or can be determined by I2C address and switching signal 2 .
  • the I2C address 1 corresponds to the normal mode
  • the I2C address 2 corresponds to the ultra-low power consumption mode
  • the lower power consumption mode corresponds to the high-level switching signal 1
  • the low-power consumption mode corresponds to the low-level switching signal 2 .
  • the sensor hub in the aforementioned step S1204, the sensor hub generates a low-level switching signal 2 to control the image sensor to operate in a low power consumption mode.
  • the sensor hub can also determine that a human face and/or gesture appears in the image captured by the front camera of the electronic device according to the detection result of the human face and/or gesture, and generate a high-level switching signal 1 to control the image sensor Run in a lower power mode.
  • the sensor hub In the aforementioned step S1213, the sensor hub generates I2C address 1, and controls the image sensor to operate in normal mode.
  • the sensor hub In the aforementioned step S1217, the sensor hub generates an I2C address 2, and controls the image sensor to switch to an ultra-low power consumption mode. Or the sensor hub generates a low-level switching signal 2 to control the image sensor to switch to a low-power mode. Alternatively, the sensor hub generates a high-level switching signal 1 to control the image sensor to switch to a lower power consumption mode.
  • the four working modes of the image sensor are divided into two groups, which may be different from the foregoing content, which may be: the normal mode is the first group of working modes, the lower power mode, the low power mode and the super power mode
  • the low power consumption mode is the second group of working modes.
  • the image sensor operates in the second group of working modes with low power consumption.
  • the second group of working modes can also be called the low power consumption sensing mode.
  • the switch between the first group of working modes and the second group of working modes of the image sensor can be realized by using high and low level switching signals, and the switching between the second group of working modes of the image sensor
  • the switching can be realized through different I2C addresses.
  • the switching signals of high and low levels and the I2C address can be generated by the same part of the processor, or can be generated by different parts.
  • the high and low level switching signals can be generated by the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor; the I2C address can be driven by the camera of the controller, or the IFE lite or IFE in the ISP of the processor , or generated by the processor's AP or sensor hub.
  • the processor generates and sends a low-level switching signal to the image sensor to control the image sensor to operate in normal mode; the processor generates and sends a high-level switching signal and different I2C addresses to the image sensor to control the image
  • the sensor operates in the second group of working modes; specifically, the processor generates and sends I2C address 1 to the image sensor to control the image sensor to operate in a lower power consumption mode, and the processor generates and sends I2C address 2 to the image sensor to control the image sensor Running in low-power mode, the processor generates and sends address 3 to the image sensor I2C to control the image sensor to run in ultra-low power mode.
  • the switching between the first group of working modes and the second group of working modes of the image sensor can be realized by using high and low level switching signals, and the switching between the second group of working modes of the image sensor Switching between them can be realized by adding different control signals to the same I2C address.
  • the switching signals of high and low levels, and the I2C address plus control signal can be generated by the same part of the processor, or can be generated by different parts.
  • the high and low level switching signals can be generated by the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor; the I2C address plus control signal can be driven by the camera of the controller, or the IFE in the ISP of the processor lite or IFE, or AP or sensor hub generated by the processor.
  • the processor generates and sends a low-level switching signal to the image sensor to control the image sensor to operate in normal mode; the processor generates and sends a high-level switching signal to the image sensor, as well as an I2C address plus a control signal, Control the image sensor to run in the second group of working modes; specifically, the processor generates and sends I2C address 1 and the first control signal to the image sensor, controls the image sensor to run in a lower power consumption mode, and the processor generates and sends to the image sensor The I2C address 1 and the second control signal control the image sensor to operate in a low power consumption mode, and the processor generates and sends the address 1 and the third control signal to the image sensor I2C to control the image sensor to operate in an ultra-low power consumption mode.
  • the first control signal, the second control signal and the third control signal can all be understood as switching command signals.
  • the first control signal is used to control the image sensor to run in a low power consumption mode
  • the second control signal is used to control the image sensor to run in a low power consumption mode
  • the third control signal is used to control the image sensor to run in an ultra-low power consumption mode.
  • the I2C interface of the image sensor receives the I2C address 1 and the control signal, the I2C control module with the address set to 1 is triggered, and further transmits the control signal to the image sensor, the image sensor analyzes the control signal, determines the working mode specified by the control signal, and switches the image sensor Enter the working mode specified by the control signal.
  • the processor sends the I2C address 1 plus the control signal, which can be understood as the processor sends information carrying the I2C address 1 and the control signal to the image sensor through the I2C bus.
  • the processor when it sends a low-level switching signal to the image sensor, it can also send an I2C address 1 plus a control signal, but the control signal is a conventional control signal, and does not have the ability to control the image sensor to a certain level. The function of operating in one working mode.
  • the switching between the first group of working modes and the second group of working modes of the image sensor can be realized by using high and low level switching signals.
  • the first group of working modes The mode corresponds to a high-level switching signal
  • the lower power consumption mode corresponds to a low-level switching signal, and vice versa.
  • the ultra-low power consumption mode and the low power consumption mode can be realized by the image sensor in an active switching manner.
  • the processor can also generate an I2C address, to control the image sensor to operate in a lower power consumption mode.
  • the switching between the first group of working modes and the second group of working modes of the image sensor can be realized by using different I2C addresses, and the switching between the second group of working modes of the image sensor , can be realized by adding different control signals to the same I2C address.
  • the controller can be driven by the camera of the controller, or the IFE lite or IFE in the ISP of the processor, or the AP or sensor hub of the processor, and send the I2C address to the image sensor, or the I2C address plus different control signals.
  • the processor generates and sends I2C address 1 to the image sensor to control the image sensor to operate in normal mode; the processor generates and sends I2C address 2 plus a control signal to the image sensor to control the image sensor to operate in the second group of working modes ; Specifically, the processor generates and sends I2C address 2 and the first control signal to the image sensor to control the image sensor to operate in a lower power consumption mode, and the processor generates and sends I2C address 2 and the second control signal to the image sensor to control The image sensor operates in a low power consumption mode, and the processor generates and sends address 2 and a third control signal to the image sensor I2C to control the image sensor to operate in an ultra-low power consumption mode.
  • the first control signal, the second control signal and the third control signal can all be understood as switching command signals.
  • the first control signal is used to control the image sensor to run in a low power consumption mode
  • the second control signal is used to control the image sensor to run in a low power consumption mode
  • the third control signal is used to control the image sensor to run in an ultra-low power consumption mode.
  • the I2C interface of the image sensor receives the I2C address 1, and the I2C control module with the address set to 1 is triggered to control the image sensor to run in normal mode.
  • the I2C interface of the image sensor receives the I2C address 2 and the control signal, the I2C control module with the address set to 2 is triggered, and further transmits the control signal to the image sensor, the image sensor analyzes the control signal, determines the working mode specified by the control signal, and switches the image sensor Enter the working mode specified by the control signal.
  • the processor can also send an I2C address 1 plus a control signal to the image sensor, and the control signal can be a conventional control signal or a switching command signal to control the image sensor to operate in normal mode .
  • the switching between the first group of working modes and the second group of working modes of the image sensor can be realized by using different I2C addresses, and the switching between the second group of working modes of the image sensor Among them, the ultra-low power consumption mode and the low power consumption mode can be realized by the image sensor in an active switching manner.
  • the processor can generate an I2C address to control the image sensor to operate in a normal mode, and generate another I2C address to control the image sensor to operate in a lower power consumption mode.
  • the processor and the image sensor implement the specific switching process of the working mode of the image sensor in the switching manner provided in the implementation manners, and reference may be made to the content provided in Embodiment 5.
  • two I2C control modules are set in the image sensor, and each I2C control module is provided with two addresses, one address of the first I2C control module corresponds to the normal mode of the image sensor, and the other One address corresponds to a low power consumption mode, one address of the second I2C control module corresponds to a low power consumption mode, and the other address corresponds to an ultra-low power consumption mode.
  • the address of one I2C control module is set to 1, the addresses of the other I2C control module are set to 2 and 3, and the two addresses are time-division multiplexed.
  • the sensor hub issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters normal mode.
  • the sensor hub sends the I2C address to 2, and the I2C control module with the address set to 2 and 3 transmits the control signal to the image sensor, and the image sensor enters the lower power consumption mode corresponding to address 2.
  • the sensor hub sends the I2C address 3, and the I2C control module with the address set to 2 and 3 transmits the control signal to the image sensor, and the image sensor enters the lower power consumption mode corresponding to the address 3.
  • four I2C control modules are set in the image sensor, the address of one I2C control module corresponds to the normal mode of the image sensor, the address of one I2C control module corresponds to the lower power consumption mode, and one I2C control module The address of the control module corresponds to the low power consumption mode, and the address of an I2C control module corresponds to the ultra-low power consumption mode.
  • the address of one I2C control block is set to 1, the address of another I2C control block is set to 2, the address of another I2C control block is set to 3, and the address of another I2C control block is set to 4 .
  • the sensor hub issues an I2C address of 1, and the I2C control module with the address set to 1 transmits a control signal to the image sensor, and the image sensor enters normal mode.
  • the sensor hub issues an I2C address of 2
  • the I2C control module with the address set to 2 transmits control signals to the image sensor, and the image sensor enters a lower power consumption mode.
  • the sensor hub issues the I2C address as 3, and the I2C control module with the address set to 3 transmits the control signal to the image sensor, and the image sensor enters the low power consumption mode.
  • the sensor hub sends the I2C address to 3, and the I2C control module with the address set to 3 transmits the control signal to the image sensor, and the image sensor enters the ultra-low power consumption mode.
  • another embodiment of the present application also provides an electronic device, which includes a processor and an image sensor that can interact with the processor.
  • the image sensor is generally set on the electronic device front camera.
  • the hardware structure and software architecture of the electronic device may be as described above, and will not be described here.
  • the processor includes a controller, an image signal processor ISP, an intelligent sensor hub sensor hub, and an application processor AP, and the ISP includes at least an image front end IFE.
  • the controller can include three logic units: AO module, automatic exposure module and camera driver. The functions of the AO module, the automatic exposure module and the camera driver can be as described above.
  • the smart sensor hub is a conventional part of the processor, common processors are equipped with a smart sensor hub, therefore, using the smart sensor hub to work with the low power consumption mode of the image sensor can avoid the use of the simplest image front end IFE lite, restrictions on processor usage.
  • the image sensor includes three working modes, which are: normal mode, low power consumption mode and ultra-low power consumption mode, which are as described in Embodiment 1 above, and will not be described here.
  • the normal mode belongs to the normal working mode of the image sensor.
  • the resolution of the image generated in this mode is the resolution configured by the image sensor, for example, it can be 4208*3120.
  • the low power mode consumes less power than the normal mode, and the ultra-low power mode Lowest power consumption.
  • the resolution of ultra-low-resolution images generated is usually: 16*12; for image sensors in low-power mode, the resolution of low-resolution images generated is usually: 64*48 , 72*54, 80*60, 96*72, 120*90, 128*96.
  • the image sensor and the processor are respectively provided with MIPI interface, pin 1 and I2C interface.
  • the pin 1 of the image sensor and the processor is used to transmit the level signal that the image sensor needs to send to the outside, such as the detection result indicated by the high level or the low level.
  • the MIPI interface of the image sensor and the processor is used to transfer the image data of the image generated by the image sensor.
  • the I2C interface of the image sensor and the processor is used to transmit different I2C addresses and control the image sensor to enter different working modes.
  • the image sensor is provided with an I2C control module connected to the I2C interface, and the I2C control module can interact with the processor through the I2C interface.
  • the I2C control module can be understood as a logic module provided in the image sensor, and can also be understood as a hardware component of the image sensor.
  • the normal mode of the image sensor corresponds to one of the high and low level signals
  • the low power consumption mode and the ultra-low power consumption mode of the image sensor correspond to the other of the high and low level signals
  • the image sensor's The switch between low power consumption mode and ultra-low power consumption mode is completed by the processor issuing the I2C address plus different control signals.
  • the normal mode of the image sensor corresponds to an I2C address
  • the low power mode and ultra-low power mode of the image sensor correspond to an I2C address
  • the image sensor can be provided with two I2C control modules, and each I2C control module corresponds to an I2C address.
  • the low-power mode and ultra-low power mode of the image sensor can be completed by the image sensor by actively switching, or by the processor sending high and low level signals through a pin, or through the I2C address Add different control signals to complete.
  • the normal mode of the image sensor corresponds to an I2C address
  • the low power mode of the image sensor corresponds to an I2C address
  • the ultra-low power mode of the image sensor corresponds to an I2C address
  • the image sensor can be provided with three I2C control modules, and each I2C control module corresponds to an I2C address.
  • the image sensor can be set with two I2C control modules, one I2C control module corresponds to the I2C address of the normal mode, and one I2C control module sets the I2C address of the ultra-low power consumption mode and the I2C address of the low power consumption mode, and the two I2C addresses are divided into time reuse.
  • the camera driver of the controller, the IFE lite or IFE in the ISP of the processor, or the AP of the processor generates an I2C address, and sends the I2C address through the I2C interface of the processor, and the I2C address passes through the I2C of the image sensor.
  • the interface is transmitted to the image sensor, and the image sensor switches the working mode according to the I2C address.
  • the normal mode of the image sensor corresponds to an I2C address
  • the low-power mode and ultra-low power mode of the image sensor correspond to an I2C address
  • the sensor hub generates an I2C address to switch the working mode of the image sensor
  • the image sensor adopts active switching low
  • the power consumption mode and the ultra-low power consumption mode are taken as examples to describe the working modes of the image sensor.
  • the switching manner of the working mode of the image sensor includes the following steps:
  • the image sensor in the ultra-low power consumption mode generates an ultra-low-resolution image every other time interval, and uses the image data of the ultra-low-resolution image to detect light and shadow changes, and obtain a light and shadow change detection result.
  • the image sensor in the ultra-low power consumption mode sends the detection result of the light and shadow change to the sensor hub.
  • the image sensor can input the detection result of light and shadow changes to the sensor hub through pin 1.
  • the sensor hub can use the detection results of light and shadow changes to execute some low-power events. If it is detected that the front camera detects light and shadow changes, the display will not turn off, the ringtone volume will be reduced, and the display will be automatically rotated according to the direction of the face. Trigger screen capture/play pause, page turning and other functions.
  • the image sensor in the ultra-low power consumption mode determines that the front camera of the electronic device has detected the light and shadow change according to the detection result of the light and shadow change, and then switches the working mode to the low power consumption mode.
  • the image sensor in the low power consumption mode generates a low-resolution image at a time interval.
  • the image sensor uses the image data of the low-resolution image to perform face and/or gesture detection, and obtain a face and/or gesture detection result.
  • the image sensor can use the image data of the low-resolution image to perform face detection, and when a face is detected, it can be determined that the user is using an electronic device.
  • the image sensor can also use the image data of the low-resolution image for gesture detection, and when a pre-configured gesture is detected, it can also determine that the user is using the electronic device.
  • the image sensor can also use the image data of the high-resolution image to perform face detection and gesture detection to obtain face and gesture detection results.
  • the image sensor sends the face and/or gesture detection result to the sensor hub.
  • the detection result of the face or gesture can be sent to the sensor hub.
  • the image sensor can send the face and/or gesture detection results to the sensor hub through pin 1.
  • the sensor hub can use the face and/or gesture detection results to perform some low-power events. If a face and/or gesture is detected, the display screen will not be turned off, the ringtone volume will be reduced, and the control display screen will be automatically activated according to the face direction. Rotate, trigger screen capture/pause playback, turn pages, etc.
  • the front camera of the electronic device is triggered to start by the control command, and the image sensor needs to enter the normal mode.
  • the sensor hub will generate an I2C address 1 to control the image sensor to enter the normal mode.
  • the front camera of the electronic device is turned off by the control instruction, and the image sensor needs to exit the normal mode and enter the ultra-low power consumption mode or the low power consumption mode.
  • the sensor hub will generate I2C address 2, the image sensor receives the I2C address 2, and controls the image sensor to enter an ultra-low power consumption mode.
  • the sensor hub will generate the I2C address 2, and the image sensor will receive the I2C address 2, and control itself to be in the low-power mode or the ultra-low power mode according to the state of the electronic device, such as the state of the display screen.
  • the front camera of the electronic device is turned off by the control instruction, and the electronic device is in the off-screen state, the image sensor receives the I2C address 2, and the image sensor switches to the ultra-low power consumption mode.
  • the front camera of the electronic device is turned off by the control command, the display screen is still in the bright state, the image sensor is connected to the I2C address 2, and the image sensor is switched to a low power consumption mode.
  • Another embodiment of the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer or a processor, the computer or the processor executes any one of the above-mentioned methods. one or more steps.
  • Another embodiment of the present application also provides a computer program product including instructions.
  • the computer program product is run on the computer or the processor, the computer or the processor is made to perform one or more steps in any one of the above methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

一种工作模式的切换控制方法、电子设备及计算机可读存储介质，所述电子设备包括处理器和摄像头，所述摄像头中的图像传感器包括至少两种工作模式，所述工作模式的切换控制方法包括：所述处理器通过I2C接口向所述图像传感器发送信号，其中，所述信号用于控制所述图像传感器以所述信号对应的工作模式运行。

Description

工作模式的切换控制方法、电子设备及可读存储介质
本申请要求于2021年8月20日提交中国国家知识产权局、申请号为202110971250.X、发明名称为“工作模式的切换方法、电子设备及介质”，于2022年1月27日提交中国国家知识产权局、申请号为202210099723.6、发明名称为“工作模式的切换控制方法、电子设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域，尤其涉及一种工作模式的切换控制方法、电子设备及可读存储介质。
背景技术
电子设备，例如手机，其摄像头包括图像传感器，图像传感器包括两种工作模式，以适配不同的功耗要求。图像传感器的工作模式的切换过程中，存在下述问题：
图像传感器的两个工作模式的互相切换，需要处理器生成高电平或者低电平，作用于图像传感器的切换引脚，处理器的控制逻辑较为复杂。
发明内容
本申请提供了一种工作模式的切换控制方法、电子设备、程序产品及存储介质，目的在于解决切换图像传感器的工作模式时存在的控制逻辑复杂的问题。
为了实现上述目的，本申请提供了以下技术方案：
第一方面，本申请提供了一种工作模式的切换控制方法，应用于电子设备，电子设备包括处理器和摄像头，摄像头中的图像传感器包括至少两种工作模式，工作模式的切换控制方法包括：处理器通过I2C接口向图像传感器发送信号，其中，信号用于控制图像传感器以信号对应的工作模式运行。
由上述内容可以看出：处理器通过I2C接口向图像传感器发送信号，控制图像传感器以信号对应的工作模式运行，实现了处理器采用通过I2C接口发送的信号控制图像传感器的工作模式的切换，不需要处理器生成高低电平来切换图像传感器的工作模式，解决处理器因采用高低电平来切换图像传感器的工作模式出现的控制逻辑复杂的问题。
在一个可能的实施方式中，信号为I2C地址。
在一个可能的实施方式中，图像传感器包括I2C控制模块，处理器通过I2C接口向图像传感器发送信号，包括：处理器通过I2C接口向图像传感器发送I2C地址，触发I2C地址对应的I2C控制模块运行，以控制图像传感器以I2C地址对应的工作模式运行。
在一个可能的实施方式中，一个I2C控制模块对应两个I2C地址，一个I2C地址对应图像传感器的至少一种工作模式。
在一个可能的实施方式中，图像传感器包括与图像传感器的工作模式的数量相同的I2C控制模块，且每一个I2C控制模块对应一个I2C地址，一个I2C地址对应图像传感器的一种工作模式。
在一个可能的实施方式中，一个I2C地址对应图像传感器的至少两种工作模式，工作模式的切换控制方法还包括：图像传感器响应切换信号，控制图像传感器以对应同一个I2C 地址的工作模式中的一种工作模式运行，切换信号由处理器向图像传感器发送，或由图像传感器生成。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式和第二工作模式，第二工作模式比第一工作模式功耗低；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第一图像分辨率>第二图像分辨率。图像传感器的第一工作模式对应第一I2C地址，图像传感器的第二工作模式对应第二I2C地址；其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式和第三工作模式，第二工作模式比第一工作模式功耗低，第三工作模式比第二工作模式功耗低，图像传感器的第一工作模式对应第一I2C地址，图像传感器的第二工作模式对应第二I2C地址，图像传感器的第三工作模式对应第三I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第一图像分辨率>第二图像分辨率>第三图像分辨率。其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行，或者向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行，或者向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行；处理器利用光影变化的检测结果，确定摄像头检测到光影变化，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；处理器利用较低分辨率图像的图像数据，确定电子设备满足第二工作模式的退出条件，向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行，较低分辨率图像由处于第二工作模式的图像传感器生成。
在本可能的实施方式中，由于图像传感器设置了超低功耗模式，超低功耗模式的功耗比较低功耗模式低，在电子设备未被使用时，图像传感器是以超低功耗模式运行，可以降低功耗。其中，电子设备未被使用可以理解成：电子设备待机，显示屏处于灭屏状态，或者电子设备处于锁屏状态。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式和第三工作模式，第二工作模式比第一工作模式功耗低，第三工作模式比第二工作模式 功耗低，图像传感器的第一工作模式对应第一I2C地址，图像传感器的第二工作模式和第三工作模式对应第二I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第一图像分辨率>第二图像分辨率>第三图像分辨率。其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式或第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式或第三工作模式运行。
在本可能的实施方式中，由于图像传感器设置了超低功耗模式，超低功耗模式的功耗比较低功耗模式低，在电子设备未被使用时，图像传感器是以超低功耗模式运行，可以降低功耗。其中，电子设备未被使用可以理解成：电子设备待机，显示屏处于灭屏状态，或者电子设备处于锁屏状态。并且，图像传感器的第二工作模式和第三工作模式对应第二I2C地址，可以实现分时复用第二I2C地址。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行，包括：图像传感器响应第一切换信号，控制图像传感器以第二工作模式运行，第一切换信号由处理器或图像传感器利用光影变化的检测结果，确定摄像头检测到光影变化时生成，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；图像传感器响应第二切换信号，控制图像传感器以第三工作模式运行，第二切换信号由处理器或图像传感器，利用较低分辨率图像的图像数据，确定电子设备满足第二工作模式的退出条件而生成，较低分辨率图像由处于第二工作模式的图像传感器生成。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行，包括：图像传感器接收第二I2C地址对应的I2C控制模块传输的控制信号，并控制图像传感器以控制信号指定的工作模式运行，控制信号用于指定第三工作模式或第二工作模式。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，第二工作模式比第一工作模式功耗低，第四工作模式比第二工作模式功耗低，第三工作模式比第四工作模式功耗低，第一工作模式对应第一I2C地址，第二工作模式对应第二I2C地址，第四工作模式对应第三I2C地址，第三工作模式对应第四I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第二图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制 图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行，或者向图像传感器发送第三I2C地址，控制图像传感器以第四工作模式运行；或者向图像传感器发送第四I2C地址，控制图像传感器以第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第三I2C地址，控制图像传感器以第四工作模式运行，或者向图像传感器发送第四I2C地址，控制图像传感器以第三工作模式运行；处理器利用光影变化的检测结果，确定摄像头检测到光影变化，向图像传感器发送第三I2C地址，控制图像传感器以第四工作模式运行，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像出现人脸和/或手势，向图像传感器发送第二I2C地址，控制图像传感器以第二工作模式运行，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到；处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势，向图像传感器发送第四I2C地址，控制图像传感器以第三工作模式运行；处理器利用较低分辨率图像的图像数据，确定电子设备满足第二工作模式的退出条件，向图像传感器发送第四I2C地址，控制图像传感器以第三工作模式运行，较低分辨率图像由处于第二工作模式的图像传感器生成。
在本可能的实施方式中，图像传感器额外设置低功耗模式和超低功耗模式，超低功耗模式的功耗比低功耗模式低，低功耗模式的功耗比较低功耗模式低，可以保证电子设备控制图像传感器在四种工作模式中运行，进一步可以保证电子设备控制图像传感器选择合适功耗的模式运行，降低电子设备的功耗。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，第二工作模式比第一工作模式功耗低，第四工作模式比第二工作模式功耗低，第三工作模式比第四工作模式功耗低，第一工作模式和第二工作模式对应第一I2C地址，第四工作模式和第三工作模式对应第二I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第二图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第三工作模式运行；处理器确定显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第三工作模式运行；处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势时，向图像传感器发送第二I2C地址，控制图像传感器以第三工作模式运行，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行包括：图像传感器响应第一切换信号，控制图像传感器以第四工作模式运行，第一切换信号由处理器或图像传感器利用光影变化的检测结果，确定摄像头检测到光影变化时生成，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；图像传感器响应第二切换信号，控制图像传感器以第二工作模式运行，第二切换信号由处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像出现人脸和/或手势时生成，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
在一个可能的实施方式中，一个I2C地址对应图像传感器的至少两种工作模式，处理器通过I2C接口向图像传感器发送信号，包括：处理器向图像传感器发送I2C地址和切换信号，控制图像传感器以I2C地址和切换信号对应的工作模式运行。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，第二工作模式比第一工作模式功耗低，第四工作模式比第二工作模式功耗低，第三工作模式比第四工作模式功耗低，第一工作模式和第二工作模式对应第一I2C地址，第四工作模式和第三工作模式对应第二I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第二图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器向图像传感器发送I2C地址和切换信号，控制图像传感器以I2C地址和切换信号对应的工作模式运行，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址和第一切换信号，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第一I2C地址和第二切换信号，控制图像传感器以第二工作模式运行，或者向图像传感器发送第二I2C地址和第三切换信号，控制图像传感器以第四工作模式运行，或者向图像传感器发送第二I2C地址和第四切换信号，控制图像传感器以第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址和第三切换信号，控制图像传感器以第四工作模式运行；或者向图像传感器发送第二I2C地址和第四切换信号，控制图像传感器以第三工作模式运行；处理器利用光影变化的检测结果，确定摄像头检测到光影变化，向图像传感器发送第二I2C地址和第三切换信号，控制图像传感器以第四工作模式运行，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像出现人脸和/或手势，向图像传感器发送第一I2C地址和第二切换信号，控制图像传感器以第二工作模式运行，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到；所处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势，向图像传感器发送第二I2C地址和第四切换信号，控制图像传感器以第三工 作模式运行；处理器利用较低分辨率图像的图像数据，确定电子设备满足第二工作模式的退出条件，向图像传感器发送第二I2C地址和第四切换信号，控制图像传感器以第三工作模式运行，较低分辨率图像由处于第二工作模式的图像传感器生成。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，第二工作模式比第一工作模式功耗低，第四工作模式比第二工作模式功耗低，第三工作模式比第四工作模式功耗低，第一工作模式对应第一I2C地址，第二工作模式，第四工作模式和第三工作模式对应第二I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第二工作模式也可以称为较低功耗模式，在第二工作模式下生成的图像的分辨率为第二图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第二图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行，包括：图像传感器接收第二I2C地址对应的I2C控制模块传输的控制信号，并控制图像传感器以控制信号指定的工作模式运行，控制信号用于指定第三工作模式，第二工作模式或第四工作模式。
在一个可能的实施方式中，第二I2C地址对应的I2C控制模块传输的控制信号包括：第一控制信号，第二控制信号和第三控制信号；其中：第一控制信号指定的工作模式为第三工作模式，由处理器确定摄像头被关闭时生成，或者由处理器确定电子设备的显示屏被灭屏时生成，或者由所处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势时生成，或者由处理器利用较低分辨率图像的图像数据，确定电子设备满足第二工作模式的退出条件时生成；第二控制信号指定的工作模式为第四工作模式，由处理器确定摄像头被关闭时生成，或者由处理器确定电子设备的显示屏被灭屏时生成，或者由处理器利用光影变化的检测结果，确定摄像头检测到光影变化时生成，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；第三控制信号指定的工作模式为第二工作模式，由处理器确定摄像头被关闭时生成，或者，由处理器根据人脸和/或手势检测结果，确定出摄像头拍摄的图像出现人脸和/或手势时生成。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第三工作模式和第四工作模式，第四工作模式比第一工作模式功耗低，第三工作模式比第四工作模式功耗低，图像传感器的第一工作模式对应第一I2C地址，图像传感器的第四工作模式对应第二I2C地址，图像传感器的第三工作模式对应第三I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器通过I2C 接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第四工作模式运行，或者向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第四工作模式运行，或者向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行；处理器利用光影变化的检测结果，确定摄像头检测到光影变化，向图像传感器发送第二I2C地址，控制图像传感器以第四工作模式运行，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；处理器利用人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势，向图像传感器发送第三I2C地址，控制图像传感器以第三工作模式运行，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
在一个可能的实施方式中，图像传感器的工作模式包括：第一工作模式，第三工作模式和第四工作模式，第四工作模式比第一工作模式功耗低，第三工作模式比第四工作模式功耗低，图像传感器的第一工作模式对应第一I2C地址，图像传感器的第四工作模式和第三工作模式对应第二I2C地址；第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率；第三工作模式也可以称为超低功耗模式，在第三工作模式下生成的图像的分辨率为第三图像分辨率；第四工作模式可以称为低功耗模式，第四工作模式下生成的图像的分辨率为第四图像分辨率；第一图像分辨率>第四图像分辨率>第三图像分辨率；其中：处理器通过I2C接口向图像传感器发送信号，包括：处理器确定摄像头被启动，向图像传感器发送第一I2C地址，控制图像传感器以第一工作模式运行；处理器确定摄像头被关闭，向图像传感器发送第二I2C地址，控制图像传感器以第四工作模式或第三工作模式运行；处理器确定电子设备的显示屏被灭屏，向图像传感器发送第二I2C地址，控制图像传感器以第四工作模式或第三工作模式运行。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行包括：图像传感器响应第一切换信号，控制图像传感器以第四工作模式运行，第一切换信号由处理器或图像传感器利用光影变化的检测结果，确定摄像头检测到光影变化时生成，光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；图像传感器响应第二切换信号，控制图像传感器以第三工作模式运行，第二切换信号由处理器或图像传感器，利用人脸和/或手势检测结果，确定出摄像头拍摄的图像未出现人脸和/或手势而生成，人脸和/或手势检测结果，由处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
在一个可能的实施方式中，图像传感器响应切换信号，控制图像传感器以对应同一个I2C地址的工作模式中的一种工作模式运行，包括：图像传感器接收第二I2C地址对应的I2C控制模块传输的控制信号，并控制图像传感器以控制信号指定的工作模式运行，控制信号用于指定第三工作模式或第四工作模式。
在一个可能的实施方式中，发送信号的对象，包括：处理器的摄像头驱动，或处理器的图像信号处理器中的最简图像前端或图像前端，或处理器的应用处理器，或处理器的智能传感集线器。
在一个可能的实施方式中，切换信号由处理器的图像信号处理器中的最简图像前端或图像前端，或处理器的应用处理器，或智能传感集线器，向图像传感器发送。
在本可能的实施方式中，智能传感集线器因功耗较低，因此，由智能传感集线器生成并发送切换信号，具有功耗低的优点。
第二方面，本申请提供了一种电子设备，包括：显示屏；摄像头，摄像头包括图像传感器，图像传感器包括至少两种工作模式；一个或多个处理器；存储器，其上存储有程序；当程序被一个或多个处理器执行时，使得电子设备执行如第一方面任意一项的工作模式的切换控制方法。
第三方面，本申请提供了一种计算机可读存储介质，其上存储有计算机程序，其中，计算机程序被处理器执行时实现如第一方面任意一项的工作模式的切换控制方法。
第四方面，本申请提供了一种计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行如第一方面任意一项的工作模式的切换控制方法。
附图说明
图1a和图1b为本申请提供的电子设备的一种应用场景图；
图2为本申请提供的电子设备的图像传感器和处理器的连接示意图；
图3a为本申请提供的电子设备的结构示意图；
图3b为本申请提供的电子设备中的逻辑单元的运行过程展示图；
图4a为本申请实施例提供的一种电子设备的结构示意图；
图4b为本申请实施例提供的一种电子设备的软件结构示例图；
图5a和图5b为本申请实施例一提供的电子设备的图像传感器和处理器的连接示意图；
图5c本申请实施例一提供的工作模式的切换控制方法的时序图；
图6位本申请实施例一提供的电子设备的工作状态的展示图；
图7a为本申请实施例二提供的电子设备的图像传感器和处理器的连接示意图；
图7b为本申请实施例二提供的图像传感器的工作模式的切换示意图；
图8a为本申请实施例三提供的电子设备的图像传感器和处理器的连接示意图；
图8b为本申请实施例三提供的电子设备的图像传感器和处理器的连接示意图
图9a和图9b为本申请实施例四提供的电子设备的图像传感器和处理器的连接示意图；
图9c和图9d为本申请实施例四提供的图像传感器的工作模式的切换示意图；
图10a为本申请实施例四提供的电子设备的图像传感器和处理器的连接示意图；
图10b为本申请实施例四提供的图像传感器的工作模式的切换示意图；
图11a，图11b，图11c和图11d为本申请实施例四提供的电子设备的图像传感器和处理器的连接示意图；
图12a为本申请实施例五提供的电子设备的图像传感器和处理器的连接示意图；
图12b和图12c为本申请实施例五提供的图像传感器的工作模式的切换示意图；
图12d本申请实施例五提供的工作模式的切换控制方法的时序图；
图13a和图13b为本申请实施例五提供的电子设备的图像传感器和处理器的连接示意图；
图14为本申请实施例六提供的电子设备的图像传感器和处理器的连接示意图。
具体实施方式
本申请说明书和权利要求书及附图说明中的术语“第一”、“第二”和“第三”等是用于区别不同对象，而不是用于限定特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
用户可以通过电子设备拍照或自拍，也可以通过电子设备查阅网页、新闻、文章等，还可以通过电子设备玩游戏，看视频。用户通过电子设备查阅网页、新闻、文章，玩游戏或看视频时，用户会长时间注视电子设备的显示屏，为配合用户长时间注视显示屏，电子设备的前置摄像头会对用户进行拍照，在利用前置摄像头拍摄的图像，检测到用户长时间注视显示屏后可执行多种对应事件，例如显示屏不熄屏、铃声音量降低等。
图1a展示了用户通过电子设备的前置摄像头进行自拍的场景。图1b展示了用户通过电子设备查阅网页的场景。以下以此两种场景为例进行方案介绍。并且，因在此两种场景均涉及到电子设备的前置摄像头，所以以电子设备的前置摄像头为例进行说明。但这些均不构成对方案应用场景的限定。
参见图2，电子设备的前置摄像头中的图像传感器，一般设置有normal模式和较低功耗模式两种工作模式，较低功耗模式比normal模式功耗低。normal模式也可以称为正常工作模式，属于普通成像模式。电子设备采用前置摄像头进行摄像，图像传感器以normal模式运行。
电子设备上电，前置摄像头(图中以图像传感器来代替展示，以下用图像传感器代替前置摄像头进行介绍)也上电，前置摄像头上电后，可在摄像头驱动的控制下进入较低功耗模式。
处于较低功耗模式的图像传感器，会每隔一个时间间隔，生成较低分辨率图像，通常为640*480，并将较低分辨率图像的图像数据，通过移动产业处理器接口(mobile industry processor interface，MIPI)发送到最简图像前端(Image Front End lite，IFE lite)单元。IFE lite输出的较低分辨率图像的图像数据可用于存储于内存的安全buffer。
参见图3a，最简图像前端(Image Front End lite，IFE lite)单元，是图像信号处理器(image signal processor，ISP)中的集成单元，电子设备的每个摄像头输出的低分辨率图像的图像数据，可到达IFE lite，由其将摄像头输出的图像存储于内存的安全buffer。
处理器的控制器利用较低分辨率图像的图像数据，可执行多种事件，如人眼注视显示屏的检测(也称AO方案)。
控制器执行AO方案涉及到自动曝光模块、AO(always on)模块和摄像头驱动三个逻辑单元，以下先对三个逻辑单元进行介绍。
自动曝光模块属于控制器的一个逻辑单元，由控制器运行自动曝光(automatic exposure，AE)算法得到。
AO(always on)模块也属于控制器的一个逻辑单元，由控制器运行AO(always on)方案得到。AO方案，是指基于AO camera(always on camera)实现的智慧感知的解决方案，通常包含人眼注视识别、机主识别、手势识别等功能，典型特征为长时间低功耗运行。
摄像头驱动也属于控制器的一个逻辑单元，用于对摄像头配置参数、工作模式等，还用于打开或关闭摄像头等。
处理器的控制器利用较低分辨率图像的图像数据，执行AO方案的具体方式如下：
电子设备的显示屏显示网页，用户注视电子设备的显示屏查阅网页。如图3a所示，电子设备发送指令，电子设备的前置摄像头响应指令后运行，执行步骤S1、拍摄用户的人脸图像。最简图像前端单元执行步骤S2读取人脸图像，基于安全机制将人脸图像存储于内存的安全buffer。AO模块执行步骤S3-1、获取内存的安全buffer存储的人脸图像的图像数据，通过分析图像数据，确定用户人眼是否注视显示屏。在AO模块确定用户人眼注视显示屏时，执行步骤S4、控制电子设备的显示屏不熄屏。
摄像头拍摄的人脸图像的图像质量高低，制约着AO模块确定用户人眼是否注视显示屏的准确性。尤其是摄像头拍摄的人脸图像的图像亮度较高或者较低时，AO模块确定用户人眼是否注视显示屏的误差较大。为此，自动曝光模块执行步骤S3-2，获取内存存储的人脸图像的图像数据；利用图像数据，计算人脸图像的图像亮度，将计算得到的图像亮度与标准亮度做比对，得到比对结果；依据比对结果调整摄像头的曝光参数，一般为曝光时长和增益，得到曝光时长调整值和增益调整值。自动曝光模块还执行步骤S5、将计算得到的曝光时长调整值和增益调整值，传输至AO模块，AO模块再按照图3a中步骤S6所示，将曝光时长调整值和增益调整值发送至摄像头驱动，摄像头驱动按照图3a中步骤S7所示，配置摄像头以曝光时长调整值和增益调整值运行。电子设备可再发送指令，摄像头响应电子设备的指令，以曝光时长调整值和增益调整值运行拍摄图像。
以下结合图3b，对AO模块分析图像数据，确定用户人眼是否注视显示屏，以及自动曝光模块调整摄像头的曝光参数的具体方式进行说明。
参见图3b，图像序列包括摄像头拍摄的多帧图像，比如图像帧1，2，3，4…n，其中，摄像头开始以通用的曝光时长和增益运行。通常来讲，通用的曝光时长和增益可以是预先设定的。自动曝光模块按照图像的存储顺序，依次获取图像序列的每一帧图像的图像数据。针对第一帧图像(也称图像帧1)，自动曝光模块利用图像帧1的图像数据，计算图像帧1的图像亮度，将图像帧1的图像亮度与标准亮度做比对，得到比对结果。若比对结果反映出图像帧1的图像亮度与标准亮度的差值小于预设值(如±10％)，自动曝光模块则不执行操作，摄像头还是以原始曝光时长和增益运行，该原始曝光时长和增益是指前述通用的曝光时长和增益。若比对结果反映出图像帧1的图像亮度与标准亮度的差值不小于预设值，自动曝光模块依据比对结果调整摄像头的曝光时长和增益，得到曝光时长1调整值和增益1调整值。自动曝光模块将曝光时长1调整值和增益1调整值，经AO模块向摄像头驱动传输。摄像头驱动配置摄像头以曝光时长1调整值和增益1调整值运行拍摄图像。
受自动曝光模块和摄像头驱动执行一次流程，滞后摄像头的拍摄一帧图像的影响，假设图像帧2和图像帧3，是摄像头以原始曝光时长和增益运行拍摄得到，自动曝光模块按照上述处理方式，利用图像帧2和图像帧3的图像数据，计算得到曝光时长1调整值和增益1调整值；摄像头驱动配置摄像头以曝光时长1调整值和增益1调整值运行拍摄图像。图像帧4为摄像头被配置以曝光时长1调整值和增益1调整值拍摄得到。自动曝光模块也采样上述处理方式，利用图像帧4的图像数据，计算得到曝光时长2调整值和增益2调整值；摄像头驱动配置摄像头以曝光时长2调整值和增益2调整值运行拍摄图像。如此反复，直到自动曝光模块比对出图像帧的图像亮度和标准亮度的差值小于预设值，如±10％停止。
AO模块也按照图像的存储顺序，依次获取图像序列的每一帧图像的图像数据。针对AO模块获取的每一帧图像的图像数据，AO模块均执行下述流程，得到每一帧图像中人眼注视显示屏的判断结果。以下以AO模块处理图像帧1的图像数据为例说明。
AO模块比对图像帧1的图像数据和样本特征库，根据图像帧1的图像数据和样本特征库的比对结果，对图像帧1配置置信度，该置信度用于表征图像帧1中的人眼注视显示屏的概率。AO模块判断图像帧1的置信度是否小于门限值，图像帧1的置信度不小于门限值，则确定图像帧1中的人眼注视显示屏，图像帧1的置信度小于门限值，则确定图像帧1中的人眼未注视显示屏。
一些实施例中，样本特征库中包括人眼注视显示屏图像的特征数据。该特征数据的确定方式为：获取大量的样本图像，样本图像包括人眼注视显示屏的样本图像和人眼未注视显示屏的样本图像，利用每一个样本图像的图像数据进行学习，得到表征人眼注视显示屏图像的特征数据。人眼注视显示屏的样本图像和人眼未注视显示屏的样本图像，均指代电子设备的前置摄像头拍摄人脸图像。
AO模块确定一定时长内的图像存在一帧人眼注视显示屏的图像，则执行控制电子设备的显示屏不熄屏、铃声音量降低等对应事件。
在一些实施例中，AO模块确定连续多帧图像有一帧图像，这一帧图像中的人眼注视显示屏，则执行控制电子设备的显示屏不熄屏，与按照电子设备的显示屏被设定的屏幕熄灭时间来控制显示屏熄屏的方案进行结合，结合的方式如下。
假设电子设备被设定的屏幕熄灭时间为15秒。在电子设备的显示屏被启动显示数据时刻起启动计时，计时满规定时间，如7秒后，电子设备的前置摄像头拍摄图像，AO模块依次获取图像，并针对获取的每一帧图像，执行下述操作：
比对图像的图像数据和样本特征库，根据图像的图像数据和样本特征库的比对结果，对图像配置置信度。判断图像被配置的置信度是否小于筛查出的置信度门限值，若图像被配置的置信度不小于置信度门限值，则确定图像中人眼注视显示屏；若图像被配置的置信度小于置信度门限值，则确定图像中人眼未注视显示屏。
在计时满7秒开始，一直到计时满15秒的时间段内，AO模块确定有一帧图像中人眼注视显示屏，则控制显示屏不熄屏。若AO模块持续确定图像中人眼均未注视显示屏，则在计时满15秒的时刻，控制显示屏熄屏。
需要说明的是，以上内容是以屏幕熄灭时间为15秒为示例对AO模块的处理流程进行介绍，但这并不构成对其的限定。
在用户注视显示屏，电子设备的控制器执行前述的AO方案过程中，若用户启动电子设备的前置摄像头，进入图1a展示的利用前置摄像头进行自拍的场景，电子设备会响应用户操作，前置摄像头拍照得到图像，并由显示屏显示拍摄的图像。
具体的，图2展示的处理器中，应用处理器(application processor，AP)发送低电平的上电切换信号到图像传感器的切换引脚，图像传感器的工作模式由较低功耗模式切换为normal模式。
处于normal模式的图像传感器，每隔一个时间间隔，生成标准分辨率图像，并将标准分辨率图像的图像数据，通过MIPI引脚发送到ISP的图像前端(Image Front End，IFE)。图像前端IFE对标准分辨率图像的图像数据进行处理，并由显示屏显示处理后的图像数据。
若用户关闭前置摄像头，电子设备响应用户的操作，执行对应流程。具体的，图2展示的AP会发送高电平的上电切换信号到图像传感器的切换引脚，图像传感器从normal模式切换为较低功耗模式。
由上述内容可以看出：图像传感器的工作模式的切换过程中，存在下述问题：
1、图像传感器处于较低功耗模式，图像传感器生成较低分辨率图像，处理器根据较低分辨率图像，可执行如AO方案等多种事件。在图像传感器利用较低分辨率图像，执行如AO方案等多种事件，存在有功耗高的问题。
需要说明的是，在用户并没有使用电子设备，但电子设备的图像传感器也会进入较低功耗模式，且处理器配合执行如AO方案等多种事件，如此会造成处理器的功耗较大。
2、图像传感器的较低功耗模式和normal模式的互相切换，需要处理器生成高电平或者低电平，作用于图像传感器的切换引脚，控制逻辑较为复杂。
具体的，对于一个图像传感器，采用高电平或者低电平的上电切换信号切换图像传感器的工作模式时，电子设备的处理器需要针对图像传感器的两种工作模式，在控制逻辑上将图像传感器虚拟为两个器件，即配置虚拟图像传感器的两种控制逻辑，通过控制两个虚拟图像传感器的切换运行，来实现上电切换信号的发送，以实现图像传感器的工作模式的切换，如此控制逻辑较为复杂。
3、为了配合图像传感器的较低功耗模式的运行，电子设备的处理器需要配置有最简图像前端(Image Front End lite，IFE lite)单元，如此对处理器的使用就形成了限定，配置有最简图像前端单元的处理器，才可配合图像传感器在功耗比较低的工作模式下运行。
基于上述技术方案中存在的问题，本申请通过下述实施例提出方案。
本申请实施例提供的方案，均可以适用于手机，平板电脑，桌面型、膝上型、笔记本电脑，超级移动个人计算机(Ultra-mobile Personal Computer，UMPC)，手持计算机，上网本，个人数字助理(Personal Digital Assistant，PDA)，可穿戴电子设备，智能手表等电子设备。
图4a为本申请实施例提供的一种电子设备的组成示例。以手机为例，电子设备400可以包括处理器410，外部存储器接口420，内部存储器421，显示屏430，摄像头440，天线1，天线2，移动通信模块450，以及无线通信模块460等。
可以理解的是，本实施例示意的结构并不构成对该电子设备的具体限定。在另一些实施例中，该电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器410可以包括一个或多个处理单元，例如：处理器410可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备400的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
视频编解码器用于对数字视频压缩或解压缩。电子设备可以支持一种或多种视频编解码器。这样，电子设备可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
处理器410中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器410中的存储器为高速缓冲存储器。该存储器可以保存处理器410刚用过或循环使用的指令或数据。如果处理器410需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器410的等待时间，因而提高了系统的效率。
在一些实施例中，处理器410可以包括一个或多个接口。接口可以包括集成电路总线(Inter-Integrated Circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器410可以包含多组I2C总线。处理器410可以通过不同的I2C总线接口分别耦合摄像头440等。
I2S接口可以用于音频通信。
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。
MIPI接口可以被用于连接处理器410与显示屏430，摄像头440等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface， DSI)等。在一些实施例中，处理器410和摄像头440通过CSI接口通信，实现电子设备400的拍摄功能。处理器410和显示屏430通过DSI接口通信，实现电子设备400的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器410与摄像头440，显示屏430，无线通信模块460等。
可以理解的是，本实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备400的结构限定。在本申请另一些实施例中，电子设备400也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
外部存储器接口420可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备的存储能力。外部存储卡通过外部存储器接口420与处理器410通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器421可以用于存储计算机可执行程序代码，可执行程序代码包括指令。处理器410通过运行存储在内部存储器421的指令，从而执行电子设备400的各种功能应用以及数据处理。内部存储器421可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器421可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器410通过运行存储在内部存储器421的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备的各种功能应用以及数据处理。
电子设备通过GPU，显示屏430，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏430和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器410可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏430用于显示图像，视频等。显示屏430包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oled，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备可以包括1个或N个显示屏430，N为大于1的正整数。
电子设备的显示屏430上可以显示一系列图形用户界面(graphical user interface，GUI)，这些GUI都是该电子设备的主屏幕。一般来说，电子设备的显示屏430的尺寸是固定的，只能在该电子设备的显示屏430中显示有限的控件。控件是一种GUI元素，它是一种软件组件，包含在应用程序中，控制着该应用程序处理的所有数据以及关于这些数据的交互操作，用户可以通过直接操作(direct manipulation)来与控件交互，从而对应用程序的有关信息进行读取或者编辑。一般而言，控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
电子设备可以通过ISP，摄像头440，视频编解码器，GPU，显示屏430以及应用处理器等实现拍摄功能。
ISP用于处理摄像头440反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头图像传感器上，光信号转换为电信号，摄像头图像传感器将电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头440中。
摄像头440包含镜头和图像传感器。摄像头440用于捕获静态图像或视频。物体通过镜头生成光学图像投射到图像传感器。图像传感器可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。图像传感器把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备可以包括1个或N个摄像头440，N为大于1的正整数。
DSP用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
电子设备的无线通信功能可以通过天线1，天线2，移动通信模块450，无线通信模块460，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块450可以提供应用在电子设备上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块450可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块450可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块450还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块450的至少部分功能模块可以被设置于处理器410中。在一些实施例中，移动通信模块450的至少部分功能模块可以与处理器410的至少部分模块被设置在同一个器件中。
无线通信模块460可以提供应用在电子设备上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块460可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块460经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器410。无线通信模块460还可以从处理器410接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
另外，在上述部件之上，运行有操作系统。例如iOS操作系统，Android操作系统，Windows操作系统等。在操作系统上可以安装运行应用程序。
图4b是本申请实施例的电子设备的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。如图4b所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，显示，音乐，铃声，以及短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。如图4b所示，应用程序框架层可以包括窗口管理器，内容提供器，电话管理器，资源管理器，通知管理器，视图系统等，在本申请的一些实施例中，应用程序框架层还可以包括感知服务。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
电话管理器用于提供电子设备的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
感知服务用于执行前述提出的AO方案。感知服务执行AO方案过程中，若检测到一帧图像中人眼注视显示屏，则控制应用程序层的显示采用不熄屏方式进行显示，铃声在需要输出响铃时，以降低音量的方式进行输出。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。在本申请一些实施例中，应用冷启动会在Android runtime中运行，Android runtime由此获取到应用的优化文件状态参数，进而Android runtime可以通过优化文件状态参数判断优化文件是否因系统升级而导致过时，并将判断结果返回给应用管控模块。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，二维图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染、合成和图层处理等。
二维图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。摄像头驱动用于对摄像头配置参数，打开或关闭摄像头。
需要说明的是，本申请实施例虽然以Android系统为例进行说明，但是其基本原理同样适用于基于iOS、Windows等操作系统的电子设备。
实施例一
针对前述功耗高的问题1，本申请实施例提供了一种电子设备，该电子设备包括处理器以及可与处理器进行交互的图像传感器，该图像传感器一般设置于电子设备的前置摄像头。电子设备的硬件结构和软件架构，可如前述内容，此处不展开说明。
参见图5a，本实施例提供的电子设备中，处理器包括控制器、图像信号处理器ISP和应用处理器AP，ISP至少包括图像前端IFE和最简图像前端IFE lite两个集成单元。控制器可以包括AO模块、自动曝光模块和摄像头驱动三个逻辑单元。AO模块、自动曝光模块和摄像头驱动的功能可如前述内容。
图像传感器，设置的工作模式包括第一工作模式，第二工作模式，以及第三工作模式。第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率，例如可以为4208*3120。第二工作模式也可以称为较低功耗模式，第二工作模式的功耗比第一工作模式功耗低，且在第二工作模式下生成的图像的分辨率为第二图像分辨率，通常可为320*240、520*392、640*480、720*540、800*600、960*720、1040*784。第三工作模式也可以称为超低功耗模式，第三种工作模式的功耗最低，在第三工作模式下生成的图像的分辨率为第三图像分辨率，通常为16*12。因此，可以看出：第一图像分辨率>第二图像分辨率>第三图像分辨率。
图像传感器和处理器上分别设置有MIPI接口，引脚1和引脚2。
图像传感器的MIPI接口，可传输图像传感器生成图像的图像数据。该图像数据可经处理器的MIPI接口，传输到处理器的ISP。一些实施例中，ISP中的图像前端IFE获取处理器的MIPI接口接收的图像数据；另一些实施例中，ISP中的最简图像前端IFE lite获取处理器的MIPI接口接收的图像数据。
图像传感器的引脚1，用于传输一位电平信号。同理，处理器的引脚1也用于传输一位电平信号。在一些实施例中，图像传感器需对外发送的电平信号，如由高电平或低电平所指示的检测结果，可通过引脚1向外传输，经处理器的引脚1传输到处理器的ISP。一些实施例中，ISP中的最简图像前端IFE lite获取处理器的引脚1接收的电平信号。
在一些示例中，图像传感器的引脚1可称之为GPO引脚，INT引脚。
图像传感器的引脚2也用于传输一位电平信号。同理，处理器的引脚2也用于传输一位电平信号。在一些实施例中，图像传感器的引脚2属于切换引脚，用于接收切换图像传感器的工作模式的切换信号。
一些实施例中，处理器的ISP中的IFE lite或IFE，或处理器的AP生成切换信号，通过处理器的引脚2发送切换信号，该切换信号再经图像传感器的引脚2传输到图像传感器，图像传感器根据切换信号进行工作模式的切换。
需要说明的是，IFE lite，IFE或AP均具有生成切换信号的能力，但采用IFE lite，IFE或AP生成切换信号，功耗依次增加，控制复杂性依次降低。
在一些示例中，图像传感器的引脚2可称之为PONV引脚，Xshutdown2引脚。
结合图5b(为了绘图简单，图5b未绘示处理器的引脚1和MIPI接口)和图5c，本实施例提供的工作模式的切换控制方法，应用于前述提出的电子设备，电子设备上电，图像传感器默认进入超低功耗模式，或者在摄像头驱动的控制下通常进入超低功耗模式。本实施例提供的工作模式的切换控制方法，包括下述步骤：
S501、处于超低功耗模式的图像传感器，每隔一个时间间隔，生成超低分辨率图像。
一些实施例中，电子设备开机上电，显示屏亮屏并输出如图6中的(a)所示的锁屏界面，图像传感器以超低功耗模式运行，以被配置的时间间隔，生成超低分辨率图像。
另一些实施例中，电子设备待机，显示屏处于灭屏状态时，图像传感器也会以超低功耗模式运行，以配置的时间间隔，生成超低分辨率图像。
S502、处于超低功耗模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测，得到光影变化的检测结果。
其中，图像传感器利用超低分辨率图像的图像数据，进行光影变化的检测，用于确定电子设备的前置摄像头前是否有光影变化。
S503、图像传感器向最简图像前端IFE lite发送光影变化的检测结果。
其中，图像传感器可将光影变化的检测结果通过引脚1输入到最简图像前端IFE lite。
S504、最简图像前端IFE lite，利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则生成切换信号。
最简图像前端IFE lite确定电子设备的前置摄像头检测到光影变化，说明用户可能在使用电子设备，需要切换图像传感器的工作模式，因此IFE lite生成切换信号，控制图像传感器由超低功耗模式切换到较低功耗模式。
一些实施例中，切换信号包括高、低电平。在一个示例中，高电平的切换信号，用于控制图像传感器进入超低功耗模式，低电平的切换信号，用于控制图像传感器进入较低功耗模式。当然，也可以由高电平的切换信号控制图像传感器切换到较低功耗模式，低电平的切换信号，控制图像传感器切换到超低功耗模式。
一些实施例中，切换信号也可由IFE或者AP生成。具体的，IFE或者AP与最简图像前端IFE lite交互，在最简图像前端IFE lite，利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，由IFE或者AP生成切换信号。或者，IFE或者AP接收光影变化的检测结果，利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，生成切换信号。
参见图6中的(b)，电子设备的显示屏输出网页，用户阅读网页而长时间注视显示屏。基于此，图像传感器利用超低分辨率图像的图像数据，进行光影变化的检测时，会检测到光影变化，最简图像前端IFE lite需要生成切换信号。
需要说明的是，最简图像前端IFE lite可利用光影变化的检测结果，执行一些低功耗事件，如若检测到前置摄像头检测到光影变化，则执行显示屏不熄屏，铃声音量降低，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
还需要说明的是，最简图像前端IFE lite确定电子设备的前置摄像头未检测到光影变化，则控制图像传感器持续处于超低功耗模式。
S505、最简图像前端IFE lite向图像传感器发送切换信号。
最简图像前端IFE lite可通过引脚2向图像传感器发送切换信号，图像传感器响应切换信号，切换工作模式由超低功耗模式到较低功耗模式。
S506、处于较低功耗模式的图像传感器，每隔一个时间间隔，生成较低分辨率图像。
S507、处于较低功耗模式的图像传感器，向最简图像前端IFE lite发送较低分辨率图像的图像数据。
其中，图像传感器将较低分辨率图像的图像数据，通过MIPI引脚发送到最简图像前端IFE lite。
一些实施例中，最简图像前端IFE lite可将较低分辨率图像的图像数据存储于内存的安全buffer。在最简图像前端IFE lite将低分辨率图像的图像数据存储于内存的安全buffer后，处理器的控制器可利用较低分辨率图像的图像数据，可执行多种事件，如人眼注视显示屏的检测(也称AO方案)，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
还需要说明的是，控制器利用较低分辨率图像的图像数据，执行多种事件过程中，若利用较低分辨率图像的图像数据，确定出电子设备满足被执行事件的退出条件，如执行AO方案，控制器如前述内容，持续确定图像中人眼均未注视显示屏，则电子设备可不继续执行被执行事件。并且，图像传感器可持续以较低功耗模式运行，或者控制图像传感器切换到超低功耗模式。
一些实施例中，最简图像前端IFE lite生成切换信号，控制图像传感器切换到超低功耗模式。
S508、图像前端IFE生成低电平的上电切换信号。
用户启动电子设备的前置摄像头，电子设备的显示屏如图6中的(c)，展示前置摄像头的拍摄模式。电子设备会响应用户操作，前置摄像头拍照得到图像，并由显示屏显示拍摄的图像。
具体的，电子设备的前置摄像头被控制指令触发而启动，图像传感器需进入normal模式。一些实施例中，在电子设备的前置摄像头被启动后，图像前端IFE会生成低电平的上电切换信号，控制图像传感器进入normal模式。另一些实施例中，也可以由IFE lite或者AP生成低电平的上电切换信号，控制图像传感器进入normal模式。本实施例以图像前端IFE生成低电平的上电切换信号为例进行说明。
上电切换信号，包括高、低电平。一些实施例中，低电平的上电切换信号，用于控制图像传感器切换到normal模式，高电平的上电切换信号，用于控制图像传感器退出normal模式。当然，也可以由高电平的上电切换信号控制图像传感器切换到normal模式，低电平的上电切换信号，控制图像传感器退出normal模式。本实施例中，是以低电平的上电切换信号控制图像传感器切换到normal模式，高电平的上电切换信号控制图像传感器退出normal模式为例进行说明。
还需要说明的是，步骤S508的执行位置，并不限定于图5c展示。在图像传感器以超低功耗模式下运行时，若前置摄像头被触发启动，图像前端IFE也会生成低电平的上电切换信号，切换图像传感器以normal模式运行。
S509、图像前端IFE向图像传感器发送低电平的上电切换信号。
图像前端IFE通过引脚2，将低电平的上电切换信号发送到图像传感器，图像传感器接收到低电平的上电切换信号，切换为normal模式。
S510、处于normal模式的图像传感器，生成标准分辨率图像。
一些实施例中，图像传感器也可以配置的时间间隔为周期，生成标准分辨率图像。
S511、处于normal模式的图像传感器，向图像前端IFE发送标准分辨率图像的图像数据。
一些实施例中，处于normal模式的图像传感器可将标准分辨率图像的图像数据，通过MIPI引脚发送到图像前端IFE。图像前端IFE对标准分辨率图像的图像数据进行处理，并由显示屏显示处理后的图像数据。
S512、图像前端IFE生成高电平的上电切换信号。
若用户关闭前置摄像头，电子设备响应用户的操作，执行对应流程。
具体的，电子设备的前置摄像头被控制指令触发而关闭，图像传感器需退出normal模式。图像前端IFE生成高电平的上电切换信号，控制图像传感器退出normal模式。
一些实施例中，图像前端IFE生成高电平的上电切换信号，可控制图像传感器由normal模式退出，并切换到超低功耗模式或较低功耗模式。
一些实施例中，电子设备的前置摄像头被控制指令触发而关闭，也可以由IFE lite或者AP生成高电平的上电切换信号，控制图像传感器进入超低功耗模式或较低功耗模式。本实施例以图像前端IFE生成高电平的上电切换信号为例进行说明。
S513、图像前端IFE向图像传感器发送高电平的上电切换信号。
其中，图像前端IFE可通过引脚2，将高电平的上电切换信号发送到图像传感器。
一些实施例中，图像传感器接收到高电平的上电切换信号，可直接切换到超低功耗模式。
另一些实施例中，图像传感器接收到高电平的上电切换信号，可根据显示屏的状态控制自身处于较低功耗模式还是超低功耗模式。一些实施例中，电子设备的前置摄像头被控制指令触发而关闭，且电子设备如图6中的(d)所示，处于灭屏状态，图像传感器接收图像前端IFE发送的高电平的上电切换信号后，图像传感器切换到超低功耗模式。一些实施例中，电子设备的前置摄像头被控制指令触发而关闭，电子设备如图6中的(e)所示，显 示屏还处于亮屏状态，图像传感器接收图像前端IFE发送的高电平的上电切换信号后，图像传感器切换到较低功耗模式。
还需要说明的是，图像传感器以超低功耗模式运行，返回执行步骤S501，图像传感器以较低功耗模式运行，返回执行步骤S506。
还需要说明的是，图像传感器从normal模式切换到其他工作模式，还可采用下述实现方式：
图像前端IFE生成并发送高电平的上电切换信号到图像传感器，并且，AP，或者IFE，或者IFE lite，可根据电子设备的当前状态，如显示屏的状态，生成并向图像传感器发送切换信号。图像传感器接收高电平的上电切换信号以及切换信号，结合高电平的上电切换信号以及切换信号，将normal模式切换到较低功耗模式或超低功耗模式。在一个示例中，图像传感器接收到高电平的上电切换信号以及第一种切换信号(如高电平)，图像传感器将normal模式切换到较低功耗模式；图像传感器接收到高电平的上电切换信号以及第二种切换信号(如低电平)，图像传感器将normal模式切换到超低功耗模式。
一些实施例中，图像传感器接收切换信号的引脚，也可以为图5b绘示的引脚2。基于此，图像传感器的引脚2可分时接收上电切换信号和切换信号，结合高电平的上电切换信号以及切换信号，切换到较低功耗模式或超低功耗模式。
另一实施例中，图像传感器还可采用不同的引脚，接收切换信号和上电切换信号。图像传感器根据两个引脚接收的切换信号和上电切换信号，切换到较低功耗模式或超低功耗模式。
本实施例中，由于图像传感器设置了超低功耗模式，超低功耗模式的功耗比较低功耗模式低，在电子设备未被使用时，图像传感器是以超低功耗模式运行，可以降低功耗。其中，电子设备未被使用可以理解成：电子设备待机，显示屏处于灭屏状态，或者电子设备处于锁屏状态。
需要说明的是，图像传感器位于超低功耗模式和较低功耗模式，均可由图像传感器的自动曝光控制模块或者控制器的自动曝光模块，根据图像传感器生成图像的明暗调整曝光参数使得图像传感器生成亮度达到要求的图像。
还需要说明的是，在图像传感器设置超低功耗模式，且处理器以高、低电平完成图像传感器在超低功耗模式和较低功耗模式进行切换的应用场景下，处理器需要针对图像传感器的超低功耗模式，配置出虚拟图像传感器的控制逻辑，可以理解成超低功耗模式下的虚拟图像传感器，通过控制超低功耗模式下的虚拟图像传感器的切换运行，来实现切换信号的发送，以实现图像传感器切换到超低功耗模式。
还需要说明的是，前述内容提出的由IFE lite，IFE或者AP生成切换信号，控制图像传感器在超低功耗模式和较低功耗模式切换运行，属于图像传感器在超低功耗模式和较低功耗模式的被动切换。被动切换可以理解成：图像传感器的工作模式的切换，需要受控于是否接收到IFE lite，IFE或者AP发送的切换信号，在接收到切换信号时，图像传感器依据切换信号进行工作模式的切换。当然，图像传感器在超低功耗模式和较低功耗模式两种工作模式切换，还可以采用图像传感器主动切换的控制方式。主动切换可以理解成图像传 感器主动在超低功耗模式和较低功耗模式两种工作模式切换，不用受限于IFE lite，IFE或者AP发送的切换信号，具体如下：
图像传感器利用超低分辨率图像的图像数据，进行光影变化的检测，得到光影变化的检测结果。图像传感器利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则切换图像传感器以较低功耗模式运行。
图像传感器确定电子设备的前置摄像头未检测到光影变化，则控制图像传感器持续处于超低功耗模式。
图像传感器位于较低功耗模式，生成较低分辨率图像，图像传感器也可以利用较低分辨率图像的图像数据，确定出满足被处理器执行事件的退出条件，如执行AO方案，图像传感器，持续确定图像中人眼均未注视显示屏，则图像传感器可持续以较低功耗模式运行，或者控制图像传感器切换到超低功耗模式。
实施例二
为解决前述功耗高的问题1，本申请另一实施例还提供了一种电子设备，与前述实施例不同的是，本实施例的电子设备，如图7a所示，处理器包括控制器、图像信号处理器ISP、应用处理器AP和智能传感集线器(sensor hub)。
其中，智能传感集线器，提供一种基于低功耗MCU和轻量级RTOS操作系统之上的软硬件结合的解决方案，其主要功能是连接并处理来自各种传感器设备的数据。并且，智能传感集线器是处理器中的常规部件，常见的处理器均设置有智能传感集线器。
sensor hub可连接引脚1和引脚2，通过引脚1接收图像传感器发送的如由高电平或低电平所指示的检测结果，也可生成切换信号或上电切换信号，并通过引脚2发送切换信号或上电切换信号到图像传感器。
还需要说明的是，sensor hub因功耗较低，因此，由sensor hub生成切换信号或上电切换信号，具有功耗低的优点。
本实施例中，电子设备上电，图像传感器默认进入超低功耗模式，或者在摄像头驱动的控制下通常进入超低功耗模式。或者，电子设备待机，显示屏处于灭屏状态，图像传感器也会以超低功耗模式运行。
图7a展示了电子设备的处理器包括的部件，以及图像传感器的三种工作模式。图像传感器受处理器中部件生成的信号驱动，在三种工作模式进行切换的具体方式，由图7b展示。参见图7b，处于超低功耗模式的图像传感器，执行前述步骤S501和步骤S502，得到光影变化的检测结果，并将光影变化的检测结果，向sensor hub传输。
sensor hub接收到光影变化的检测结果后，可利用光影变化的检测结果，确定电子设备的前置摄像头是否检测到光影变化。若确定电子设备的前置摄像头检测到光影变化，则生成切换信号，并发送到图像传感器。图像传感器响应切换信号，切换到较低功耗模式。
一些实施例中，sensor hub确定电子设备的前置摄像头检测到光影变化，也可由AP，或者IFE，或者IFE lite生成切换信号。本实施例以sensor hub生成切换信号为例进行说明。
处于较低功耗模式的图像传感器，执行前述步骤S506和步骤S507，将较低分辨率图像的图像数据发送到最简图像前端IFE lite。
若电子设备的前置摄像头被启动，sensor hub可生成低电平的上电切换信号，并通过引脚2将低电平的上电切换信号向图像传感器，控制图像传感器的工作模式切换到normal模式。
处于normal模式的图像传感器，可执行前述步骤S510和步骤S512。
若电子设备的前置摄像头被关闭，sensor hub可生成高电平的上电切换信号，并通过引脚2将高电平的上电切换信号向图像传感器，控制图像传感器的工作模式切换到超低功耗模式或较低功耗模式。
一些实施例中，图像传感器接收到高电平的上电切换信号，可直接切换到超低功耗模式。另一些实施例中，图像传感器接收到高电平的上电切换信号，可根据显示屏的状态控制自身处于较低功耗模式还是超低功耗模式。一些实施例中，电子设备的前置摄像头被控制指令触发而关闭，且电子设备处于灭屏状态，图像传感器接收sensor hub发送的高电平的上电切换信号后，图像传感器切换到超低功耗模式。一些实施例中，电子设备的前置摄像头被控制指令触发而关闭，电子设备处于亮屏状态，图像传感器接收sensor hub发送的高电平的上电切换信号后，图像传感器切换到较低功耗模式。
还需要说明的是，sensor hub发送高电平的上电切换信号到图像传感器，图像传感器可从normal模式切换到其他工作模式时，AP，或者IFE，或者IFE lite，或者sensor hub，可根据电子设备的当前状态，如显示屏的状态，生成切换信号。图像传感器接收高电平的上电切换信号以及切换信号，结合高电平的上电切换信号以及切换信号，切换到较低功耗模式或超低功耗模式。
还需要说明的是，前述提出的高电平或低电平的上电切换信号，也可由AP，或者IFE，或者IFE lite生成，并发送到图像传感器。
一些实施例中，在电子设备出现错误时，sensor hub还可以实现对图像传感器的复位，即控制图像传感器初始化为超低功耗模式。
实施例三
针对前述处理器的控制逻辑复杂的问题2，本申请实施例提供了一种电子设备，该电子设备包括处理器以及可与处理器进行交互的图像传感器，该图像传感器一般设置于电子设备的前置摄像头。电子设备的硬件结构和软件架构，可如前述内容，此处不展开说明。
参见图8a，本实施例提供的电子设备中，处理器包括控制器、图像信号处理器ISP和应用处理器AP，ISP至少包括图像前端IFE和最简图像前端IFE lite两个集成单元。控制器可以包括AO模块、自动曝光模块和摄像头驱动三个逻辑单元。AO模块、自动曝光模块和摄像头驱动功能如前述内容，此处不展开说明。
图像传感器，设置有normal模式和较低功耗模式两种工作模式，normal模式属于图像传感器的正常工作模式，该模式下生成的图像的分辨率为图像传感器配置的分辨率，例如可以为4208*3120，较低功耗模式比normal模式功耗低，图像传感器处于较低功耗模式，可生成较低分辨率图像，通常可为320*240、520*392、640*480、720*540、800*600、960*720、1040*784。
其中，normal模式也可以如实施例一所述，称为第一工作模式，较低功耗模式也可以称为第二工作模式，具体参见前述实施例一内容，此处不再展开说明。
图像传感器和处理器上分别设置有MIPI接口和I2C接口。图像传感器设置有连接I2C接口的I2C控制模块，该I2C控制模块可通过I2C接口与处理器进行交互。其中，I2C控制模块可以理解设置于图像传感器的逻辑模块，也可以理解成图像传感器的硬件部件。
一些实施例中，控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP生成I2C地址，通过处理器的I2C接口发送I2C地址，该I2C地址再经图像传感器的I2C接口传输到图像传感器，图像传感器根据I2C地址进行工作模式的切换。本实施例以AP生成I2C地址为例进行说明。
需要说明的是，AP下发不同的I2C地址控制图像传感器进入不同的工作模式。
在一个可能的实施方式中，参见图8a，AP通过I2C总线连接图像传感器，图像传感器设置两个I2C控制模块，且两个I2C控制模块的地址设定不同，AP下发I2C地址，地址设定为AP下发的I2C地址的I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入对应的工作模式。
在一个示例中：一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2。AP下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。AP下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。
在另一个可能的实施方式中，参见图8b，AP通过I2C总线连接图像传感器，图像传感器设置一个I2C控制模块，且一个I2C控制模块设置有两个地址，两个地址分时复用。AP下发一个I2C地址，I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入地址对应的工作模式。
还需要说明的是，上述提出的示例中，I2C控制模块的地址是为了方便说明，采用数值1和数值2来表示。通常情况下，I2C控制模块的地址可采用一个字节的二进制数据来表示，该一个字节的二进制数据中，前7位数值可用于指示I2C控制模块的地址。
图像传感器的MIPI接口，可传输图像传感器生成图像的图像数据。该图像数据可经处理器的MIPI接口，传输到处理器的ISP。一些实施例中，ISP中的图像前端IFE获取处理器的MIPI接口接收的图像数据；另一些实施例中，ISP中的最简图像前端IFE lite获取处理器的MIPI接口接收的图像数据。
本实施例中，处于normal模式和较低功耗模式的图像传感器的功能，以及与处理器的交互流程，可参见前述实施例一内容，此处不展开说明。
还需要说明的是，本实施例中，处理器采用下发不同I2C地址的方式，替换采用上电切换信号，控制图像传感器切换工作模式，控制逻辑简单。
实施例四
针对前述问题功耗高的1和控制逻辑复杂的问题2，本申请实施例提供了另一种电子设备，该电子设备包括处理器以及可与处理器进行交互的图像传感器，该图像传感器一般 设置于电子设备的前置摄像头。电子设备的硬件结构和软件架构，可如前述内容，此处不展开说明。
本实施例提供的电子设备中，处理器包括控制器、图像信号处理器ISP和应用处理器AP，ISP至少包括图像前端IFE和最简图像前端IFE lite两个集成单元。控制器可以包括AO模块、自动曝光模块和摄像头驱动三个逻辑单元。AO模块、自动曝光模块和摄像头驱动功能如前述内容，此处不展开说明。
图像传感器，设置有normal模式、较低功耗模式和超低功耗模式三种工作模式，具体如前述实施例一内容，此处不展开说明。超低功耗模式和较低功耗模式，可以统称为低功耗感知模式。本实施例，图像传感器需要在normal模式、较低功耗模式和超低功耗模式三个工作模式之间进行切换，以下进行具体说明。
在一个可能的实施方式中，参见图9a，图像传感器和处理器上分别设置有MIPI接口，引脚1，引脚2和I2C接口。图像传感器和处理器设置的MIPI接口和引脚1的功能，可如前述实施例一内容，此处不展开说明。
normal模式对应一个I2C地址，低功耗感知模式对应另一个I2C地址。处理器和图像传感器的I2C接口，用于传输切换图像传感器以normal模式运行的I2C地址，或者以低功耗感知模式运行的I2C地址。
并且，图像传感器设置有连接I2C接口的I2C控制模块，该I2C控制模块可通过I2C接口与处理器进行交互。其中，I2C控制模块可以理解设置于图像传感器的逻辑模块，也可以理解成图像传感器的硬件部件。
一些实施例中，控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP生成I2C地址，通过处理器的I2C接口发送I2C地址，该I2C地址再经图像传感器的I2C接口传输到图像传感器，图像传感器根据I2C地址进行低功耗感知模式或normal模式的切换。本实施例以AP生成I2C地址为例进行说明。
如图9c所示，AP下发不同的I2C地址，来实现控制图像传感器进入低功耗感知模式或normal模式。
一个实施方式中，如图9a所示，AP通过I2C总线连接图像传感器，图像传感器设置两个I2C控制模块，两个I2C控制模块的地址设定不同，AP下发I2C地址，地址设定为AP下发的I2C地址的I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入对应的工作模式。
如在一个示例中：一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2。AP下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。AP下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器进入低功耗感知模式。以下以该示例为例进行说明。
另一个实施方式中，AP通过I2C总线连接图像传感器，图像传感器设置一个I2C控制模块，且一个I2C控制模块设置有两个地址，两个地址分时复用。AP下发一个I2C地址，I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入地址对应的工作模式。
图像传感器的引脚2用于传输一位电平信号。同理，处理器的引脚2也用于传输一位电平信号。在一些实施例中，图像传感器的引脚2属于切换引脚，用于接收切换图像传感器以较低功耗模式或超低功耗模式运行的切换信号。在一些示例中，图像传感器的引脚2可称之为PONV引脚，Xshutdown2引脚。
一些实施例中，处理器的ISP中的IFE lite或IFE，或处理器的AP可生成切换信号，通过处理器的引脚2发送切换信号，该切换信号再经图像传感器的引脚2传输到图像传感器，图像传感器根据切换信号进行较低功耗模式和超低功耗模式两种工作模式的切换。
在另一个可能的实施方式中，参见图9b，图像传感器和处理器上分别设置有MIPI接口，引脚1和I2C接口。图像传感器和处理器设置的MIPI接口和引脚1的功能，可如前述实施例一内容，此处不展开说明。I2C接口的功能可如前述实施方式内容，此处也不再展开说明。
本可能的实施方式中，图像传感器的超低功耗模式和较低功耗模式，采用主动切换的方式，具体如下：
图像传感器处于超低功耗模式运行，利用超低分辨率图像的图像数据，进行光影变化的检测，得到光影变化的检测结果。图像传感器利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则切换图像传感器以较低功耗模式运行。
图像传感器确定电子设备的前置摄像头未检测到光影变化，则控制图像传感器持续处于超低功耗模式。
图像传感器位于较低功耗模式，生成较低分辨率图像，图像传感器也可以利用较低分辨率图像的图像数据，确定出满足被处理器执行事件的退出条件，如执行AO方案，图像传感器，持续确定图像中人眼均未注视显示屏，则图像传感器可持续以较低功耗模式运行，或者控制图像传感器切换到超低功耗模式。
本可能的实施方式中，图像传感器的超低功耗模式和较低功耗模式，由图像传感器根据光影变化的检测结果来进行切换，图像传感器的normal模式和低功耗感知模式，由处理器利用不同的I2C地址进行切换，如此可以实现采用简单的控制逻辑完成图像传感器的工作模式的切换。以下以本可能的实施方式为例，展开说明。
图9d展示了图像传感器受处理器中部件生成的信号驱动，在normal模式、超低功耗模式和较低功耗模式三种工作模式进行切换的具体方式。参见图9d(为了绘图简单，图9d未绘示处理器的引脚1，引脚2，MIPI接口和I2C接口，图像传感器也未绘示I2C接口和引脚2，且将I2C控制模块绘示在图像传感器外，但并不构成对处理器和图像传感器内部结构的限定)，电子设备上电，图像传感器默认进入超低功耗模式，或在摄像头驱动控制下进入超低功耗模式。或者，电子设备待机，显示屏处于灭屏状态，图像传感器以超低功耗模式运行。
处于超低功耗模式的图像传感器，会每隔一个时间间隔，生成超低分辨率图像，并利用超低分辨率图像的图像数据，进行光影变化的检测，并将光影变化的检测结果通过引脚1输入到IFE lite。IFE lite可利用光影变化的检测结果，执行一些低功耗事件，如若检测到前置摄像头检测到光影变化，则执行显示屏不熄屏，铃声音量降低，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
图像传感器利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则生成并向图像传感器发送切换信号，将图像传感器的模式由超低功耗模式切换为较低功耗模式。
处于较低功耗模式的图像传感器，也会每隔一个时间间隔，生成较低分辨率图像，并将较低分辨率图像的图像数据，通过MIPI引脚发送到IFE lite。IFE lite输出的较低分辨率图像的图像数据存储于内存的安全buffer。
处理器利用较低分辨率图像的图像数据，可执行多种事件，如由控制器执行前述内容提出的人眼注视显示屏的检测(也称AO方案)，或者，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
若电子设备的前置摄像头被启动，AP下发I2C地址1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。
处于normal模式的图像传感器，每隔一个时间间隔，生成标准分辨率图像，并将标准分辨率图像的图像数据，通过MIPI引脚发送到IFE。IFE对标准分辨率图像的图像数据进行处理，并由显示屏显示处理后的图像数据。
若电子设备的前置摄像头被关闭，AP下发I2C地址2，控制图像传感器从normal模式切换为低功耗感知模式。其中，图像传感器接收I2C地址2，地址设定为2的I2C控制模块传输控制信号到图像传感器，控制图像传感器进入较低功耗模式或超低功耗模式，可有下述三种实施方式：
一个实施方式中，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器直接切换到超低功耗模式。
另一个实施方式中，图像传感器结合AP下发的I2C地址和显示屏的状态，控制自身切换到较低功耗模式或超低功耗模式，一般情况下，若显示屏灭屏，图像传感器切换到超低功耗模式，若显示屏亮屏，图像传感器可切换到较低功耗模式，也可切换到超低功耗模式。
另一个实施方式中，图像传感器结合AP下发的I2C地址和引脚2传输的切换信号，控制自身切换到较低功耗模式或超低功耗模式。引脚2传输的切换信号，可由处理器的ISP中的IFE lite或IFE，或处理器的AP，根据电子设备的当前状态，如显示屏的状态生成。若显示屏处于灭屏状态，生成的切换信号用于将图像传感器切换到超低功耗模式。
与前述实施例三相同，采用数值1和数值2来表示I2C控制模块的地址，是为了表达方便，并不构成对I2C控制模块的地址的具体实现方式的限定。通常情况下，I2C控制模块的地址可采用一个字节的二进制数据来表示，该一个字节的二进制数据中，前7位数值可用于指示I2C控制模块的地址。并且，下述内容提出的I2C控制模块的地址，也会以数值为示例进行说明，同样下述内容中提出的数值不构成对I2C控制模块的地址的具体实现方式的限定。
还需要说明的是，在考虑到降低功耗的要求下，本实施例提供的电子设备，也可如图10a所示，采用处理器中的sensor hub接收图像传感器处于超低功耗模式时发送的光影变化的检测结果。
图10b展示图10a中绘示的图像传感器在normal模式、较低功耗模式和超低功耗模式下的切换实现方式。具体的，参见图10b(为了绘图简单，图10b未绘示处理器的引脚1，引脚2，MIPI接口和I2C接口，图像传感器也未绘示I2C接口和引脚2，且将I2C控制模块绘示在图像传感器外，但并不构成对处理器和图像传感器内部结构的限定)，sensor hub接收到光影变化的检测结果后，可利用光影变化的检测结果，确定电子设备的前置摄像头是否检测到光影变化。若确定电子设备的前置摄像头检测到光影变化，则生成切换信号，并发送到图像传感器。图像传感器响应切换信号，将超低功耗模式切换为较低功耗模式。
一些实施例中，sensor hub确定电子设备的前置摄像头检测到光影变化，也可由AP，或者IFE，或者IFE lite生成切换信号。
sensor hub也可用于下发不同的I2C地址来实现发送上电切换信号，控制图像传感器进入低功耗感知模式或normal模式。
电子设备的前置摄像头被启动，sensor hub可下发I2C地址1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。
电子设备的前置摄像头被关闭，sensor hub可下发I2C地址2，控制图像传感器从normal模式切换为低功耗感知模式。其中，图像传感器接收到I2C地址2，地址设定为2的I2C控制模块传输控制信号到图像传感器，控制图像传感器进入较低功耗模式或超低功耗模式的方式，可如前述提出的三种实施方式，此处不再展开说明。
还需要说明的是，图像传感器和处理器上分别设置有MIPI接口，引脚1，引脚2和I2C接口。引脚1和MIPI接口的功能与前述内容相同，此处不展开说明。
与前述内容不同的是，引脚2可以传输图像传感器的低功耗感知模式和normal模式之间切换的切换信号，该切换信号包括高、低电平。一个示例中，处理器通过自身的引脚2和图像传感器的引脚2传输高电平，图像传感器以低功耗感知模式运行，处理器通过自身的引脚2和图像传感器的引脚2传输低电平，图像传感器以normal模式运行。
一个可能的实施方式中，I2C接口传输的不同的I2C地址，用于控制图像传感器的超低功耗模式和较低功耗模式之间切换运行，一个I2C地址对应超低功耗模式，另一个I2C地址对应较低功耗模式。对应的，图像传感器可设置两个I2C控制模块或一个I2C控制模块。
图像传感器的工作模式的具体切换方式，可如前述内容，此处不展开说明。
以高、低电平的切换信号，控制图像传感器在低功耗感知模式和normal模式之间切换，可以保证图像传感器中的全部模块仅在normal模式启动，图像传感器运行在低功耗感知模式下，仅由部分低功耗的模块运行即可，能够保证图像传感器的功耗低。
并且，采用高、低电平的切换信号，切换图像传感器进入normal模式，还具有图像传感器接收到电平信号，全部模块能够快速启动的优点，保证图像传感器的性能。
另一个可能的实施方式中，I2C接口传输I2C地址加不同的控制信号，用于控制图像传感器的超低功耗模式和较低功耗模式之间切换运行。对应的，图像传感器可设置一个I2C控制模块。在一个示例中，I2C接口传输I2C地址1加第一控制信号，控制图像传感器以 较低功耗模式运行，I2C接口传输I2C地址1加第二控制信号，控制图像传感器以超低功耗模式运行。
需要说明的是，处理器发送I2C地址加控制信号，可以理解成处理器通过I2C总线，向图像传感器发送携带I2C地址和控制信号的信息。并且，控制信号可以理解成是切换命令性信号，第一控制信号用于控制图像传感器以较低功耗模式运行，第二控制信号用于控制图像传感器以超功耗模式运行。
图像传感器的I2C接口接收I2C地址1和控制信号，地址设定为1的I2C控制模块被触发，进一步传输控制信号到图像传感器，图像传感器解析控制信号，确定控制信号指定的工作模式，切换图像传感器进入控制信号指定的工作模式。
还需要说明的是，处理器控制图像传感器以normal模式运行的方式，还可以是处理器通过引脚2传输电平值，且通过I2C接口传输I2C地址，该I2C地址可以控制超低功耗模式和较低功耗模式之间切换所用的I2C地址相同，但可将该I2C地址和常规控制信号，由处理器向图像传感器发送。
常规控制信号，可以理解成不具有控制图像传感器以某种工作模式运行功能的信号。
还需要说明的是，因为采用不同的I2C地址，控制图像传感器切换到不同的模式，控制逻辑简单。因此，控制图像传感器在超低功耗模式或较低功耗模式之间切换的切换信号，也可采用不同的I2C地址，来实现发送。
在一种可能的实施方式中，图像传感器的normal模式和低功耗感知模式运行之间的切换，可不同的利用I2C地址来实现，图像传感器的低功耗感知模式内部之间的切换，可通过同一个I2C地址加不同的控制信号来实现。
其中，可由控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成，并向图像传感器发送I2C地址，或I2C地址加不同的控制信号。
在一个示例中，处理器生成并向图像传感器发送I2C地址1，控制图像传感器以normal模式运行；处理器生成并向图像传感器发送I2C地址2加控制信号，控制图像传感器以低功耗感知模式运行；具体的，处理器生成并向图像传感器发送I2C地址2和第一控制信号，控制图像传感器以较低功耗模式运行，处理器生成并向图像传感器发送I2C地址2和第二控制信号，控制图像传感器以超低功耗模式运行。
与前述相同，第一控制信号和第二控制信号，属于切换命令性信号，可以指定图像传感器以某种工作模式运行。
图像传感器的I2C接口接收到I2C地址1，地址设定为1的I2C控制模块被触发，控制图像传感以normal模式运行。图像传感器的I2C接口接收I2C地址2和控制信号，地址设定为2的I2C控制模块被触发，进一步传输控制信号到图像传感器，图像传感器解析控制信号，确定控制信号指定的工作模式，切换图像传感器进入控制信号指定的工作模式。
还需要说明的是，处理器控制图像传感器以normal模式运行，也可以是采用I2C地址1加控制信号，该控制信号可以是切换命令性信号，用于指定图像传感器以normal模式运行，也可以是常规控制信号。
在另一种可能的实施方式中，针对图像传感器的三种工作模式，均对应一个I2C地址。具体的，处理器通过I2C总线连接图像传感器，图像传感器设置I2C控制模块。AP下发不同的I2C地址，地址设定为AP下发的I2C地址的I2C控制模块被触发，图像传感器被控制进入AP下发的I2C地址所对应的工作模式。
需要说明的是，不同的I2C地址也可由控制器的摄像头驱动或者或处理器的ISP中的IFE lite或IFE生成及下发。以下以AP为例进行说明。
在一些实施例中，参见图11a，图像传感器中设置两个I2C控制模块，一个I2C控制模块的地址对应图像传感器的normal模式，一个I2C控制模块的一个地址对应较低功耗模式，另一个地址对应超低功耗模式。
如一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2和3，两个地址分时复用。AP下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。AP下发I2C地址为2，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址2对应的较低功耗模式。AP下发I2C地址为3，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址3对应的较低功耗模式。
在一些实施例中，参见图11b，图像传感器中设置三个I2C控制模块，一个I2C控制模块的地址对应图像传感器的normal模式，一个I2C控制模块的地址对应较低功耗模式，一个I2C控制模块的地址对应超低功耗模式。
如一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2，另一个I2C控制模块的地址设定为3。AP下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。AP下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。AP下发I2C地址为3，地址设定为3的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。
在处理器的sensor hub连接处理器的I2C接口时，sensor hub也可用于下发不同的I2C地址，以控制对应的I2C控制模块被触发，图像传感器被控制进入下发的I2C地址所对应的工作模式。
如图11c所示，图像传感器中设置两个I2C控制模块，一个I2C控制模块的地址对应图像传感器的normal模式，一个I2C控制模块的一个地址对应较低功耗模式，另一个地址对应超低功耗模式。
一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2和3，两个地址分时复用。sensor hub下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。sensor hub下发I2C地址为2，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址2对应的较低功耗模式。sensor hub下发I2C地址为3，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址3对应的较低功耗模式。
在一些实施例中，参见图11d，图像传感器中设置三个I2C控制模块，一个I2C控制模块的地址对应图像传感器的normal模式，一个I2C控制模块的地址对应较低功耗模式，一个I2C控制模块的地址对应超低功耗模式。
一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2，另一个I2C控制模块的地址设定为3。sensor hub下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。sensor hub下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。sensor hub下发I2C地址为3，地址设定为3的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。
实施例五
本申请另一实施例还提供了一种电子设备，该电子设备包括处理器以及可与处理器进行交互的图像传感器，该图像传感器一般设置于电子设备的前置摄像头。电子设备的硬件结构和软件架构，可如前述内容，此处不展开说明。
参见图12a，本实施例提供的电子设备中，处理器包括控制器、图像信号处理器ISP、智能传感集线器sensor hub和应用处理器AP，ISP至少包括图像前端IFE和最简图像前端IFE lite两个集成单元。控制器可以包括AO模块、自动曝光模块和摄像头驱动三个逻辑单元。AO模块、自动曝光模块和摄像头驱动的功能可如前述内容。
本实施例中，图像传感器包括四种工作模式，分别为：第一工作模式，第二工作模式，以及第四工作模式。第一工作模式也可以称为正常工作模式或normal模式，在第一工作模式下生成的图像的分辨率为第一图像分辨率，例如可以为4208*3120。第二工作模式也可以称为较低功耗模式，第二工作模式的功耗比第一工作模式功耗低，且在第二工作模式下生成的图像的分辨率为第二图像分辨率，通常可为320*240、520*392、640*480、720*540、800*600、960*720、1040*784。第三工作模式也可以称为超低功耗模式，第三种工作模式的功耗最低，在第三工作模式下生成的图像的分辨率为第三图像分辨率，通常为16*12。第四工作模式也可以称为低功耗模式，低功耗模式比较低功耗模式功耗低，第四工作模式下生成的图像的分辨率为第四图像分辨率，通常可为：64*48、72*54、80*60、96*72、120*90、128*96。因此，可以看出：第一图像分辨率>第二图像分辨率>第四图像分辨率>第三图像分辨率。
图像传感器和处理器上分别设置有MIPI接口，引脚1，引脚2和I2C接口。图像传感器和处理器设置的MIPI接口和引脚1的功能，可如前述实施例一内容，此处不展开说明。
图像传感器的四种工作模式可分为两组，第一组包括normal模式和较低功耗模式，第二组包括低功耗模式和超低功耗模式。处理器和图像传感器的I2C接口，通过I2C总线相连接，用于传输I2C地址，该I2C地址用于控制图像传感器以第一组的一个工作模式或者第二组的一个工作模式运行。
本实施例中，控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub可生成并通过处理器的I2C接口发送不同的I2C地址，该I2C地址再经 图像传感器的I2C接口传输到图像传感器，图像传感器切换到I2C地址对应的工作模式。本实施例以sensor hub生成I2C地址为例进行说明。
具体的，图像传感器设置连接I2C接口的I2C控制模块，该I2C控制模块可通过I2C接口与处理器进行交互，控制图像传感器的工作模式的切换。I2C控制模块可以理解设置于图像传感器的逻辑模块，也可以理解成设置于图像传感器的硬件部件。
在图像传感器中设置I2C控制模块，可具有下述两种实施方式：
在一个可能的实施方式中，如图12a所示，图像传感器设置两个I2C控制模块，且两个I2C控制模块的地址设定不同，sensor hub下发I2C地址，地址设定为sensor hub下发的I2C地址的I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入对应的工作模式。
如在一个示例中：一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2。参见图12b，sensor hub下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器可进入normal模式或较低功耗模式。sensor hub下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器可进入超低功耗模式或低功耗模式。本实施例以本示例进行说明。
在另一个可能的实施方式中，图像传感器设置一个I2C控制模块，且该I2C控制模块设置有两个地址，两个地址分时复用，一个地址对应normal模式和较低功耗模式，另一个地址对应低功耗模式和超低功耗模式。sensor hub下发I2C控制模块的一个I2C地址，I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，进入I2C地址对应的工作模式。
图像传感器的引脚2用于传输一位电平信号。同理，处理器的引脚2也用于传输一位电平信号。在一些示例中，图像传感器的引脚2可称之为PONV引脚，Xshutdown2引脚。
本实施例中，图像传感器的引脚2属于切换引脚，用于接收切换图像传感器的工作模式的切换信号1和切换信号2，切换信号1用于控制图像传感器以normal模式或较低功耗模式运行，切换信号2用于控制图像传感器的低功耗模式或超低功耗模式运行。
切换信号1包括高、低电平。在一个示例中，参见图12b，高电平的切换信号1，用于控制图像传感器进入较低功耗模式，低电平的切换信号1，用于控制图像传感器进入normal模式。当然，也可以由高电平的切换信号1控制图像传感器切换到normal模式，低电平的切换信号1，控制图像传感器进入较低功耗模式。本实施例中，是以低电平的切换信号1控制图像传感器切换到normal模式，高电平的切换信号1控制图像传感器切换到较低功耗模式为例进行说明。
切换信号2也包括高、低电平。在一个示例中，高电平的切换信号2，用于控制图像传感器进入低功耗模式，低电平的切换信号2，用于控制图像传感器进入超低功耗模式。当然，也可以由高电平的切换信号2控制图像传感器切换到超低功耗模式，低电平的切换信号2，控制图像传感器切换到低功耗模式。本实施例中，是以低电平的切换信号2控制图像传感器切换到进入低功耗模式，高电平的切换信号2控制图像传感器切换到超低功耗模式为例进行说明。
一些实施例中，处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成切换信号1和切换信号2，通过处理器的引脚2发送，再经图像传感器的引脚2传输到图像传感器，图像传感器根据切换信号1或切换信号2进行工作模式的切换。
本实施例中，以sensor hub生成切换信号2和切换信号1为例进行说明。
还需要说明的是，图像传感器接收切换信号1的引脚和接收切换信号2的引脚，也可以不同引脚。一个示例中，图像传感器的引脚2接收切换信号1，其他一个引脚接收切换信号2。下述以引脚2分别接收切换信号1和切换信号2为例进行说明。
还需要说明的是，图像传感器可被配置成：初始运行模式为超低功耗模式。基于此，电子设备开机上电，前置摄像头的图像传感器响应上电指令，以超低功耗模式运行。
若图像传感器未配置初始运行模式，电子设备开机上电，电子设备的处理器生成第一指令控制前置摄像头运行。具体的，前置摄像头的图像传感器会响应第一指令，以超低功耗模式运行。一些实施例中，第一指令可由控制器的摄像头驱动模块生成。
电子设备上电后运行过程中，图像传感器可根据电子设备的不同运行状态进行工作模式的切换，以下结合图12c(为了绘图简单，图12c未绘示处理器的引脚1，引脚2，MIPI接口和I2C接口，图像传感器也未绘示I2C接口和引脚2，且将I2C控制模块绘示在图像传感器外，但并不构成对处理器和图像传感器内部结构的限定)和图12d，介绍本实施例提供的工作模式的切换控制方法。
本实施例提供的工作模式的切换控制方法，应用于前述提出的电子设备，电子设备上电，图像传感器默认进入超低功耗模式，或者在摄像头驱动的控制下通常进入超低功耗模式。本实施例提供的工作模式的切换控制方法，包括下述步骤：
S1201、处于超低功耗模式的图像传感器，每隔一个时间间隔，生成超低分辨率图像。
一些实施例中，电子设备开机上电，显示屏亮屏并输出锁屏界面，图像传感器以超低功耗模式运行，以被配置的时间间隔，生成超低分辨率图像。
另一些实施例中，电子设备待机，显示屏处于灭屏状态时，图像传感器也会以超低功耗模式运行，以配置的时间间隔，生成超低分辨率图像。
S1202、处于超低功耗模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测，得到光影变化的检测结果。
其中，图像传感器利用超低分辨率图像的图像数据，进行光影变化的检测，用于确定电子设备的前置摄像头前是否有光影变化。
S1203、图像传感器向sensor hub发送光影变化的检测结果。
其中，图像传感器可将光影变化的检测结果通过引脚1输入到sensor hub。
sensor hub可利用光影变化的检测结果，执行一些低功耗事件，如若检测到前置摄像头检测到光影变化，则执行显示屏不熄屏，铃声音量降低，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
S1204、sensor hub利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则生成I2C地址2和低电平的切换信号2。
sensor hub确定电子设备的前置摄像头检测到光影变化，说明用户可能在使用电子设备，需要切换图像传感器的工作模式，因此sensor hub生成I2C地址2和低电平的切换信号2，控制图像传感器由超低功耗模式切换到低功耗模式。
还需要说明的是，sensor hub确定电子设备的前置摄像头未检测到光影变化，则图像传感器持续处于超低功耗模式。
一些实施例中，I2C地址2和切换信号2也可由IFE，IFE lite或者AP生成。具体的，IFE，IFE lite或者AP与sensor hub交互，在sensor hub，利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，由IFE，IFE lite或者AP生成I2C地址2和低电平的切换信号2。或者，IFE，IFE lite或者AP接收光影变化的检测结果，利用光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，生成I2C地址2和低电平的切换信号2。
S1205、sensor hub向图像传感器发送I2C地址2和低电平的切换信号2。
sensor hub可通过引脚2向图像传感器发送低电平的切换信号2，通过I2C接口传输I2C地址2。图像传感器通过I2C接口接收I2C地址2，I2C地址2进一步传输到图像传感器中地址设定为2的I2C控制模块，地址设定为2的I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，可进入低功耗模式或超低功耗模式。进一步，图像传感器接收到低电平的切换信号2，响应低电平的切换信号2，切换工作模式由超低功耗模式到低功耗模式。
还需要说明的是，图像传感器的超低功耗模式和低功耗模式之间的切换，还可以采用主动切换的方式。具体的，图像传感器根据光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则切换工作模式为低功耗模式，确定电子设备的前置摄像头未检测到光影变化，则维持超低功耗模式。
在采用主动切换超低功耗模式和低功耗模式时，前述步骤S1204和S1205，不需要执行。
S1206、处于低功耗模式的图像传感器，每隔一个时间间隔，生成低分辨率图像。
S1207、图像传感器利用低分辨率图像的图像数据，进行人脸和/或手势检测，得到人脸和/或手势检测结果。
图像传感器可利用低分辨率图像的图像数据，进行人脸检测，当检测到人脸，则可确定用户在使用电子设备。图像传感器也可利用低分辨率图像的图像数据，进行手势检测，当检测到预先配置的手势时，则也可确定用户在使用电子设备。当然，图像传感器也可以利用低分辨率图像的图像数据，进行人脸检测以及手势的检测，得到人脸和手势检测结果。
还需要说明的是，在图像传感器主动切换超低功耗模式或低功耗模式时，图像传感器还可以利用人脸和/或手势检测结果，确定出电子设备的前置摄像头拍摄的图像未出现人脸和/或手势，维持图像传感器处于低功耗模式。或者在一段时间内，如5秒，利用人脸和/或手势检测结果，持续确定电子设备的前置摄像头拍摄的图像未出现人脸和/或手势，则控制图像传感器切换到超低功耗模式。
S1208、图像传感器向sensor hub发送人脸和/或手势检测结果。
在图像传感器单独进行人脸或者手势检测时，可将人脸或者手势的检测结果，向sensor hub发送。
具体的，图像传感器可通过引脚1向sensor hub发送人脸和/或手势检测结果。
S1209、sensor hub根据人脸和/或手势检测结果，确定出电子设备的前置摄像头拍摄的图像出现人脸和/或手势，生成I2C地址1和高电平的切换信号1。
还需要说明的是，若sensor hub根据预设时间段内，如5秒，接收到的人脸和/或手势检测结果，确定电子设备的前置摄像头拍摄的图像未出现人脸和/或手势，则控制图像传感器可持续以低功耗模式运行。或者，sensor hub生成并向图像传感器发送I2C地址2和高电平的切换信号2，控制图像传感器切换到超低功耗模式运行。
S1210、sensor hub向图像传感器发送I2C地址1和高电平的切换信号1，控制图像传感器切换为较低功耗模式。
其中，图像传感器接收I2C地址1和高电平的切换信号1，结合I2C地址1和高电平的切换信号1，切换到较低功耗模式运行。
具体的，sensor hub可通过引脚2向图像传感器发送高电平的切换信号1，通过I2C接口传输I2C地址1。图像传感器通过I2C接口接收I2C地址1后，传输到图像传感器中地址设定为1的I2C控制模块，地址设定为1的I2C控制模块被触发，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，可进入normal模式或较低功耗模式。进一步，图像传感器接收到高电平的切换信号1，控制图像传感器进入较低功耗模式。
S1211、处于较低功耗模式的图像传感器，每隔一个时间间隔，生成较低分辨率图像。
S1212、处于较低功耗模式的图像传感器，向最简图像前端IFE lite发送较低分辨率图像的图像数据。
其中，图像传感器将较低分辨率图像的图像数据，通过MIPI引脚发送到最简图像前端IFE lite。
一些实施例中，最简图像前端IFE lite可将较低分辨率图像的图像数据存储于内存的安全buffer。在最简图像前端IFE lite将低分辨率图像的图像数据存储于内存的安全buffer后，处理器的控制器可利用较低分辨率图像的图像数据，可执行多种事件，如人眼注视显示屏的检测(也称AO方案)，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
还需要说明的是，处理器的控制器利用较低分辨率图像的图像数据，执行多种事件过程中，若利用较低分辨率图像的图像数据，确定出满足被执行事件的退出条件，如执行AO方案，控制器如前述内容，持续确定图像中人眼均未注视显示屏，则图像传感器可持续以较低功耗模式运行，或者控制图像传感器切换到功耗比较低功耗模式低的工作模式。
一些实施例中，sensor hub生成I2C地址2和低电平的切换信号2，并向图像传感器发送，控制图像传感器切换到低功耗模式。
另一些实施例中，sensor hub生成I2C地址2和高电平的切换信号2，并向图像传感器发送，控制图像传感器切换到超低功耗模式。通常情况下，在电子设备的显示屏处于灭屏状态时，sensor hub生成I2C地址2和高电平的切换信号2。
还需要说明的是，也可由控制器的摄像头驱动，IFE，IFE lite或者AP生成I2C地址2，由IFE，IFE lite或者AP生成切换信号2，或者采用不同的部件，分别生成I2C地址2和切换信号2。
图像传感器以超低功耗模式运行，则执行前述步骤S1201，图像传感器以低功耗模式运行，则执行前述步骤S1206。
S1213、sensor hub生成I2C地址1和低电平的切换信号1。
用户启动电子设备的前置摄像头，电子设备的显示屏展示前置摄像头的拍摄模式。电子设备会响应用户操作，前置摄像头拍照得到图像，并由显示屏显示拍摄的图像。
具体的，电子设备的前置摄像头被控制指令触发而启动，图像传感器需进入normal模式。一些实施例中，在电子设备的前置摄像头被启动后，sensor hub会生成I2C地址1和低电平的切换信号1，控制图像传感器进入normal模式。
还需要说明的是，步骤S1213的执行位置，并不限定于图12d展示。在图像传感器以超低功耗模式、低功耗模式运行时，若前置摄像头被触发启动，图像前端IFE也会生成低电平的切换信号1，切换图像传感器以normal模式运行。
S1214、sensor hub向图像传感器发送I2C地址1和低电平的切换信号1。
sensor hub通过引脚2，将低电平的切换信号1发送到图像传感器，通过I2C接口传输I2C地址1，图像传感器的中地址设定为1的I2C控制模块处于被触发状态，可传输控制信号到图像传感器，图像传感器检测传输控制信号的I2C控制模块，可进入normal模式或较低功耗模式。进一步的，图像传感器接收到低电平的切换信号1，切换为normal模式。
S1215、处于normal模式的图像传感器，生成标准分辨率图像。
一些实施例中，图像传感器也可以配置的时间间隔为周期，生成标准分辨率图像。
S1216、处于normal模式的图像传感器，向图像前端IFE发送标准分辨率图像的图像数据。
一些实施例中，处于normal模式的图像传感器可将标准分辨率图像的图像数据，通过MIPI引脚发送到图像前端IFE。图像前端IFE对标准分辨率图像的图像数据进行处理，并由显示屏显示处理后的图像数据。
S1217、sensor hub生成I2C地址1和高电平的切换信号1。
若用户关闭前置摄像头，电子设备响应用户的操作，执行对应流程。具体的，电子设备的前置摄像头被控制指令触发而关闭，图像传感器需从normal模式切换到其他工作模式。
一些实施例中，sensor hub可生成I2C地址1和高电平的切换信号1，控制图像传感器从normal模式切换到较低功耗模式。
S1218、sensor hub向图像传感器发送I2C地址1和高电平的切换信号1。
当然，在sensor hub生成I2C地址1和生成高电平的切换信号1时，I2C地址1会通过I2C接口传输到图像传感器，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器结合I2C地址1和高电平的切换信号1，切换到较低功耗模式。
图像传感器切换到较低功耗模式，以较低功耗模式运行，可执行前述步骤1211。
还需要说明的是，电子设备的前置摄像头被控制指令触发而关闭，处理器还可根据电子设备的当前状态，切换图像传感器的工作模式。
一些实施例中，在电子设备的显示屏处于灭屏状态，sensor hub生成I2C地址2和高电平的切换信号2，控制图像传感器从normal模式切换到超低功耗模式。
一些实施例中，在电子设备的显示屏处于灭屏状态，sensor hub生成I2C地址2和低电平的切换信号2，控制图像传感器从normal模式切换到低功耗模式。
需要说明的是，图像传感器位于超低功耗模式、低功耗模式和较低功耗模式，均可由自动曝光模块或图像传感器的自动曝光控制模块，根据图像传感器生成图像的明暗调整曝光参数使得图像传感器生成亮度达到要求的图像。
还需要说明的是，在处理器采用不同的引脚发送切换信号1和切换信号2的应用场景中，I2C地址1可对应第一组工作模式的任一个，I2C地址2可对应第二组工作模式的任一个；并且，较低功耗模式，可通过切换信号1确定，或通过I2C地址和切换信号1确定，低功耗模式，可由切换信号2确定，或通过I2C地址和切换信号2确定。
在一个示例中，I2C地址1对应normal模式，I2C地址2对应超低功耗模式，较低功耗模式对应高电平的切换信号1，低功耗模式对应低电平的切换信号2。以上述示例为例展开说明方案。
具体的，前述步骤S1204中，sensor hub生成低电平的切换信号2，控制图像传感器以低功耗模式运行。
前述步骤S1209中，sensor hub还可根据人脸和/或手势检测结果，确定出电子设备的前置摄像头拍摄的图像出现人脸和/或手势，生成高电平的切换信号1，控制图像传感器以较低功耗模式运行。
前述步骤S1213中，sensor hub生成I2C地址1，控制图像传感器以normal模式运行。
前述步骤S1217中，sensor hub生成I2C地址2，控制图像传感器切换为超低功耗模式。或者sensor hub生成低电平的切换信号2，控制图像传感器切换为低功耗模式。或者，sensor hub生成高电平的切换信号1，控制图像传感器切换为较低功耗模式。
还需要说明的是，图像传感器的四种工作模式划分为两组的方式，可与前述内容不同，可以为：normal模式为第一组工作模式，较低功耗模式、低功耗模式和超低功耗模式为第二组工作模式，图像传感器在第二组工作模式运行，功耗均较低，也可以称第二组工作模式为低功耗感知模式。
在一个可能的实施方式中，图像传感器的第一组工作模式和第二组工作模式之间的切换，可利用高、低电平的切换信号来实现，图像传感器的第二组工作模式之间的切换，可通过不同的I2C地址来实现。高、低电平的切换信号和I2C地址可以由处理器的同一个部件生成，也可以由不同部件生成。高、低电平的切换信号可由处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成；I2C地址可由控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成。
在一个示例中，处理器生成并向图像传感器发送低电平的切换信号，控制图像传感器以normal模式运行；处理器生成并向图像传感器发送高电平的切换信号和不同的I2C地址，控制图像传感器以第二组工作模式运行；具体的，处理器生成并向图像传感器发送I2C地 址1，控制图像传感器以较低功耗模式运行，处理器生成并向图像传感器发送I2C地址2，控制图像传感器以低功耗模式运行，处理器生成并向图像传感器I2C发送地址3，控制图像传感器以超低功耗模式运行。
在另一个可能的实施方式中，图像传感器的第一组工作模式和第二组工作模式之间的切换，可利用高、低电平的切换信号来实现，图像传感器的第二组工作模式之间的切换，可通过同一个I2C地址加不同的控制信号来实现。
当然，高、低电平的切换信号，以及I2C地址加控制信号可以由处理器的同一个部件生成，也可以由不同部件生成。高、低电平的切换信号可由处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成；I2C地址加控制信号可由控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成。
在一个示例中，处理器生成并向图像传感器发送低电平的切换信号，控制图像传感器以normal模式运行；处理器生成并向图像传感器发送高电平的切换信号，以及I2C地址加控制信号，控制图像传感器以第二组工作模式运行；具体的，处理器生成并向图像传感器发送I2C地址1和第一控制信号，控制图像传感器以较低功耗模式运行，处理器生成并向图像传感器发送I2C地址1和第二控制信号，控制图像传感器以低功耗模式运行，处理器生成并向图像传感器I2C发送地址1和第三控制信号，控制图像传感器以超低功耗模式运行。
第一控制信号，第二控制信号和第三控制信号，均可以理解为切换命令性信号。第一控制信号用于控制图像传感器以较低功耗模式运行，第二控制信号用于控制图像传感器以低功耗模式运行，第三控制信号用于控制图像传感器以超低功耗模式运行。
图像传感器的I2C接口接收I2C地址1和控制信号，地址设定为1的I2C控制模块被触发，进一步传输控制信号到图像传感器，图像传感器解析控制信号，确定控制信号指定的工作模式，切换图像传感器进入控制信号指定的工作模式。
需要说明的是，处理器发送I2C地址1加控制信号，可以理解成处理器通过I2C总线，向图像传感器发送携带I2C地址1和控制信号的信息。
还需要说明的是，本实施方式中，处理器向图像传感器发送低电平的切换信号时，也可以发送I2C地址1加控制信号，但控制信号属于常规控制信号，不具有控制图像传感器以某种工作模式运行的功能。
在另一个可能的实施方式中，图像传感器的第一组工作模式和第二组工作模式之间的切换，可利用高、低电平的切换信号来实现，在一个示例中，第一组工作模式对应高电平的切换信号，较低功耗模式对应低电平的切换信号，反之也可以。图像传感器的第二组工作模式内部之间的切换中，超低功耗模式和低功耗模式，可由图像传感器以主动切换的方式实现。
还需要说明的是，在高、低电平的切换信号中的一种对应图像传感器的第一组工作模式，另一种对应第二组工作模式的场景下，处理器还可生成I2C地址，来控制图像传感器以较低功耗模式运行。
在另一个可能的实施方式中，图像传感器的第一组工作模式和第二组工作模式之间的切换，可不同的利用I2C地址来实现，图像传感器的第二组工作模式内部之间的切换，可通过同一个I2C地址加不同的控制信号来实现。
其中，可由控制器的摄像头驱动，或处理器的ISP中的IFE lite或IFE，或处理器的AP或者sensor hub生成，并向图像传感器发送I2C地址，或I2C地址加不同的控制信号。
在一个示例中，处理器生成并向图像传感器发送I2C地址1，控制图像传感器以normal模式运行；处理器生成并向图像传感器发送I2C地址2加控制信号，控制图像传感器以第二组工作模式运行；具体的，处理器生成并向图像传感器发送I2C地址2和第一控制信号，控制图像传感器以较低功耗模式运行，处理器生成并向图像传感器发送I2C地址2和第二控制信号，控制图像传感器以低功耗模式运行，处理器生成并向图像传感器I2C发送地址2和第三控制信号，控制图像传感器以超低功耗模式运行。
与上述实施方式相同，第一控制信号，第二控制信号和第三控制信号，均可以理解为切换命令性信号。第一控制信号用于控制图像传感器以较低功耗模式运行，第二控制信号用于控制图像传感器以低功耗模式运行，第三控制信号用于控制图像传感器以超低功耗模式运行。
图像传感器的I2C接口接收到I2C地址1，地址设定为1的I2C控制模块被触发，控制图像传感以normal模式运行。图像传感器的I2C接口接收I2C地址2和控制信号，地址设定为2的I2C控制模块被触发，进一步传输控制信号到图像传感器，图像传感器解析控制信号，确定控制信号指定的工作模式，切换图像传感器进入控制信号指定的工作模式。
还需要说明的是，本实施方式中，处理器也可以向图像传感器发送I2C地址1加控制信号，且控制信号可属于常规控制信号，也可属于切换命令性信号，控制图像传感器以normal模式运行。
在另一个可能的实施方式中，图像传感器的第一组工作模式和第二组工作模式之间的切换，可不同的利用I2C地址来实现，图像传感器的第二组工作模式内部之间的切换中，超低功耗模式和低功耗模式，可由图像传感器以主动切换的方式实现。
其中，处理器可生成一个I2C地址，控制图像传感器以normal模式运行，生成另一个I2C地址，控制图像传感器以较低功耗模式运行。
前述几种可能的实施方式中，处理器和图像传感器，以实施方式提供的切换方式实现图像传感器的工作模式的具体切换流程，可参考实施例五提供的内容。
还需要说明的是，因为采用不同的I2C地址，控制图像传感器切换到不同的模式，控制逻辑简单。因此，控制图像传感器在四种工作模式之间的切换控制，可采用发送不同的I2C地址来实现。
在一些实施例中，如图13a所示，图像传感器中设置两个I2C控制模块，每个I2C控制模块均设置有两个地址，第一I2C控制模块的一个地址对应图像传感器的normal模式，另一个地址对应较低功耗模式，第二I2C控制模块的一个地址对应低功耗模式，另一个地址对应超低功耗模式。
一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2和3，两个地址分时复用。sensor hub下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。sensor hub下发I2C地址为2，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址2对应的较低功耗模式。sensor hub下发I2C地址为3，地址设定为2和3的I2C控制模块传输控制信号到图像传感器，图像传感器进入地址3对应的较低功耗模式。
在一些实施例中，如图13b所示，图像传感器中设置四个I2C控制模块，一个I2C控制模块的地址对应图像传感器的normal模式，一个I2C控制模块的地址对应较低功耗模式，一个I2C控制模块的地址对应低功耗模式，一个I2C控制模块的地址对应超低功耗模式。
一个示例中，一个I2C控制模块的地址设定为1，另一个I2C控制模块的地址设定为2，另一个I2C控制模块的地址设定为3，另一个I2C控制模块的地址设定为4。sensor hub下发I2C地址为1，地址设定为1的I2C控制模块传输控制信号到图像传感器，图像传感器进入normal模式。sensor hub下发I2C地址为2，地址设定为2的I2C控制模块传输控制信号到图像传感器，图像传感器进入较低功耗模式。sensor hub下发I2C地址为3，地址设定为3的I2C控制模块传输控制信号到图像传感器，图像传感器进入低功耗模式。sensor hub下发I2C地址为3，地址设定为3的I2C控制模块传输控制信号到图像传感器，图像传感器进入超低功耗模式。
实施例六
针对前述处理器使用受限的问题3，本申请另一实施例还提供了一种电子设备，该电子设备包括处理器以及可与处理器进行交互的图像传感器，该图像传感器一般设置于电子设备的前置摄像头。电子设备的硬件结构和软件架构，可如前述内容，此处不展开说明。
参见图14，本实施例提供的电子设备中，处理器包括控制器、图像信号处理器ISP、智能传感集线器sensor hub和应用处理器AP，ISP至少包括图像前端IFE。控制器可以包括AO模块、自动曝光模块和摄像头驱动三个逻辑单元。AO模块、自动曝光模块和摄像头驱动的功能可如前述内容。
因为智能传感集线器是处理器中的常规部件，常见的处理器均设置有智能传感集线器，因此，采用智能传感集线器配合图像传感器的功耗低的工作模式，可以避免采用最简图像前端IFE lite，对处理器使用的限定。
本实施例中，图像传感器包括三种工作模式，分别为：normal模式，低功耗模式和超低功耗模式，具体如前述实施例一内容，此处不展开说明。normal模式属于图像传感器的正常工作模式，该模式下生成的图像的分辨率为图像传感器配置的分辨率，例如可以为4208*3120，低功耗模式比normal模式功耗低，超低功耗模式功耗最低。超低功耗模式的图像传感器，生成的超低分辨率图像的分辨率通常为：16*12；低功耗模式的图像传感器，生成的低分辨率图像的分辨率通常可为：64*48、72*54、80*60、96*72、120*90、128*96。
图像传感器和处理器上分别设置有MIPI接口，引脚1和I2C接口。
图像传感器和处理器的引脚1，用于传输图像传感器需对外发送的电平信号，如由高电平或低电平所指示的检测结果。
图像传感器和处理器的MIPI接口，用于传输图像传感器生成图像的图像数据。
图像传感器和处理器的I2C接口，用于传输不同的I2C地址，控制图像传感器进入不同的工作模式。并且，图像传感器设置有连接I2C接口的I2C控制模块，该I2C控制模块可通过I2C接口与处理器进行交互。其中，I2C控制模块可以理解设置于图像传感器的逻辑模块，也可以理解成图像传感器的硬件部件。
一些实施例中，图像传感器的normal模式，对应高、低电平信号中的一个，图像传感器的低功耗模式和超低功耗模式对应高、低电平信号中的另一个，图像传感器的低功耗模式和超低功耗模式之间的切换，由处理器下发I2C地址加不同的控制信号来完成。
一些实施例中，图像传感器的normal模式对应一个I2C地址，图像传感器的低功耗模式和超低功耗模式，对应一个I2C地址。对应的，图像传感器可设置两个I2C控制模块，每一个I2C控制模块对应一个I2C地址。其中，图像传感器的低功耗模式和超低功耗模式可由图像传感器采用主动切换的方式完成，或者由处理器通过一个引脚下发高、低电平信号的方式来完成，或者通过I2C地址加不同的控制信号来完成。
一些实施例中，图像传感器的normal模式对应一个I2C地址，图像传感器的低功耗模式对应一个I2C地址，图像传感器的超低功耗模式，对应一个I2C地址。对应的，图像传感器可设置三个I2C控制模块，每一个I2C控制模块对应一个I2C地址。或者，图像传感器可设置两个I2C控制模块，一个I2C控制模块对应normal模式的I2C地址，一个I2C控制模块设置超低功耗模式的I2C地址和低功耗模式的I2C地址，两个I2C地址分时复用。
一些实施例中，控制器的摄像头驱动，处理器的ISP中的IFE lite或IFE，或处理器的AP生成I2C地址，通过处理器的I2C接口发送I2C地址，该I2C地址再经图像传感器的I2C接口传输到图像传感器，图像传感器根据I2C地址进行工作模式的切换。
以下以图像传感器的normal模式对应一个I2C地址，图像传感器的低功耗模式和超低功耗模式对应一个I2C地址，以sensor hub生成I2C地址切换图像传感器的工作模式，且图像传感器采用主动切换低功耗模式和超低功耗模式为例，对图像传感器的工作模式进行说明。
具体的，图像传感器的工作模式的切换方式，包括下述步骤：
S1301、处于超低功耗模式的图像传感器，每隔一个时间间隔，生成超低分辨率图像，并利用超低分辨率图像的图像数据，进行光影变化的检测，得到光影变化的检测结果。
S1302、处于超低功耗模式的图像传感器向sensor hub发送光影变化的检测结果。
其中，图像传感器可将光影变化的检测结果通过引脚1输入到sensor hub。
sensor hub可利用光影变化的检测结果，执行一些低功耗事件，如若检测到前置摄像头检测到光影变化，则执行显示屏不熄屏，铃声音量降低，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
S1303、处于超低功耗模式的图像传感器根据光影变化的检测结果，确定电子设备的前置摄像头检测到光影变化，则切换工作模式为低功耗模式。
S1304、处于低功耗模式的图像传感器，每隔一个时间间隔，生成低分辨率图像。
S1305、图像传感器利用低分辨率图像的图像数据，进行人脸和/或手势检测，得到人脸和/或手势检测结果。
图像传感器可利用低分辨率图像的图像数据，进行人脸检测，当检测到人脸，则可确定用户在使用电子设备。图像传感器也可利用低分辨率图像的图像数据，进行手势检测，当检测到预先配置的手势时，则也可确定用户在使用电子设备。
当然，图像传感器也可以利用分辨率图像的图像数据，进行人脸检测以及手势的检测，得到人脸和手势检测结果。
S1306、图像传感器向sensor hub发送人脸和/或手势检测结果。
在图像传感器单独进行人脸或者手势检测时，可将人脸或者手势的检测结果，向sensor hub发送。
具体的，图像传感器可通过引脚1向sensor hub发送人脸和/或手势检测结果。
sensor hub可利用人脸和/或手势检测结果，执行一些低功耗事件，如若检测到人脸和/或手势，则执行显示屏不熄屏，铃声音量降低，控制显示屏根据人脸方向自动旋转、触发截屏/播放暂停、翻页等功能。
S1307、电子设备的前置摄像头被控制指令触发而启动，图像传感器需进入normal模式。
一些实施例中，在电子设备的前置摄像头被启动后，sensor hub会生成I2C地址1，控制图像传感器进入normal模式。
S1308、电子设备的前置摄像头被控制指令触发而关闭，图像传感器需退出normal模式，进入超低功耗模式或低功耗模式。
一些实施例中，sensor hub会生成I2C地址2，图像传感器接收到I2C地址2，控制图像传感器进入超低功耗模式。
一些实施例中，sensor hub会生成I2C地址2，图像传感器接收到I2C地址2，根据电子设备的状态，如显示屏的状态，控制自身处于低功耗模式还是超低功耗模式。
电子设备的前置摄像头被控制指令触发而关闭，且电子设备处于灭屏状态，图像传感器接收到I2C地址2，图像传感器切换到超低功耗模式。电子设备的前置摄像头被控制指令触发而关闭，显示屏还处于亮屏状态，图像传感器接I2C地址2，图像传感器切换到低功耗模式。
本申请另一实施例还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机或处理器上运行时，使得计算机或处理器执行上述任一个方法中的一个或多个步骤。
本申请另一实施例还提供了一种包含指令的计算机程序产品。当该计算机程序产品在计算机或处理器上运行时，使得计算机或处理器执行上述任一个方法中的一个或多个步骤。

Claims (27)

  1. 一种工作模式的切换控制方法，其特征在于，应用于电子设备，所述电子设备包括处理器和摄像头，所述摄像头中的图像传感器包括至少两种工作模式，所述工作模式的切换控制方法包括：
    所述处理器通过I2C接口向所述图像传感器发送信号，其中，所述信号用于控制所述图像传感器以所述信号对应的工作模式运行。
  2. 根据权利要求1所述的工作模式的切换控制方法，其特征在于，所述信号为I2C地址。
  3. 根据权利要求2所述的工作模式的切换控制方法，其特征在于，所述图像传感器包括I2C控制模块，所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器通过I2C接口向所述图像传感器发送I2C地址，触发I2C地址对应的I2C控制模块运行，以控制所述图像传感器以所述I2C地址对应的工作模式运行。
  4. 根据权利要求3所述的工作模式的切换控制方法，其特征在于，一个所述I2C控制模块对应两个I2C地址，一个所述I2C地址对应所述图像传感器的至少一种工作模式。
  5. 根据权利要求3所述的工作模式的切换控制方法，其特征在于，所述图像传感器包括与所述图像传感器的工作模式的数量相同的I2C控制模块，且每一个所述I2C控制模块对应一个I2C地址，一个所述I2C地址对应所述图像传感器的一种工作模式。
  6. 根据权利要求4所述的工作模式的切换控制方法，其特征在于，所述一个所述I2C地址对应所述图像传感器的至少两种工作模式，所述工作模式的切换控制方法还包括：
    所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行，所述切换信号由所述处理器向所述图像传感器发送，或由所述图像传感器生成。
  7. 根据权利要求2至5中任意一项所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式和第二工作模式，所述第二工作模式比所述第一工作模式功耗低；所述图像传感器的第一工作模式对应第一I2C地址，所述图像传感器的第二工作模式对应第二I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行。
  8. 根据权利要求2至5中任意一项所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式和第三工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第三工作模式比所述第二工作模式功耗低，所 述图像传感器的第一工作模式对应第一I2C地址，所述图像传感器的第二工作模式对应第二I2C地址，所述图像传感器的第三工作模式对应第三I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行，或者向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行，或者向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用光影变化的检测结果，确定所述摄像头检测到光影变化，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行，所述光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述处理器利用较低分辨率图像的图像数据，确定所述电子设备满足所述第二工作模式的退出条件，向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行，所述较低分辨率图像由处于第二工作模式的图像传感器生成。
  9. 根据权利要求6所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式和第三工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第三工作模式比所述第二工作模式功耗低，所述图像传感器的第一工作模式对应第一I2C地址，所述图像传感器的第二工作模式和第三工作模式对应第二I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式或所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式或所述第三工作模式运行。
  10. 根据权利要求9所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行，包括：
    所述图像传感器响应第一切换信号，控制所述图像传感器以所述第二工作模式运行，所述第一切换信号由所述处理器或图像传感器利用光影变化的检测结果，确定所述摄像头检测到光影变化时生成，所述光影变化的检测结果，由处于所述第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述图像传感器响应第二切换信号，控制所述图像传感器以所述第三工作模式运行，所述第二切换信号由所述处理器或图像传感器，利用较低分辨率图像的图像数据，确定所述电子设备满足所述第二工作模式的退出条件而生成，所述较低分辨率图像由处于第二工作模式的图像传感器生成。
  11. 根据权利要求9所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行，包括：
    所述图像传感器接收所述第二I2C地址对应的I2C控制模块传输的控制信号，并控制所述图像传感器以所述控制信号指定的工作模式运行，所述控制信号用于指定所述第三工作模式或第二工作模式。
  12. 根据权利要求2至5中任意一项所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第四工作模式比所述第二工作模式功耗低，所述第三工作模式比所述第四工作模式功耗低，所述第一工作模式对应第一I2C地址，所述第二工作模式对应第二I2C地址，所述第四工作模式对应第三I2C地址，所述第三工作模式对应第四I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第二工作模式运行，或者向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第四工作模式运行；或者向所述图像传感器发送所述第四I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第四工作模式运行，或者向所述图像传感器发送所述第四I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用光影变化的检测结果，确定所述摄像头检测到光影变化，向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第四工作模式运行，所述光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述处理器根据人脸和/或手势检测结果，确定出所述摄像头拍摄的图像出现人脸和/或手势，向所述图像传感器发送第二I2C地址，控制所述图像传感器以所述第二工作模式运行，所述人脸和/或手势检测结果，由所述处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到；
    所处理器根据所述人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势，向所述图像传感器发送第四I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用较低分辨率图像的图像数据，确定所述电子设备满足所述第二工作模式的退出条件，向所述图像传感器发送所述第四I2C地址，控制所述图像传感器以所述第三工作模式运行，所述较低分辨率图像由处于第二工作模式的图像传感器生成。
  13. 根据权利要求6所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第四工作模式比所述第二工作模式功耗低，所述第三工作模式比所述第四工作模式功耗低，所述第一工作模式和所述第二工作模式对应第一I2C地址，所述第四工作模式和所述第三工作模式对应第二I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器确定所述显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器根据人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势时，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第三工作模式运行，所述人脸和/或手势检测结果，由处于所述第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
  14. 根据权利要求13所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行包括：
    所述图像传感器响应第一切换信号，控制所述图像传感器以所述第四工作模式运行，所述第一切换信号由所述处理器或图像传感器利用光影变化的检测结果，确定所述摄像头检测到光影变化时生成，所述光影变化的检测结果，由处于所述第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述图像传感器响应第二切换信号，控制所述图像传感器以第二工作模式运行，所述第二切换信号由所述处理器根据人脸和/或手势检测结果，确定出所述摄像头拍摄的图像出现人脸和/或手势时生成，所述人脸和/或手势检测结果，由处于所述第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
  15. 根据权利要求4所述的工作模式的切换控制方法，其特征在于，所述一个所述I2C地址对应所述图像传感器的至少两种工作模式，所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器向所述图像传感器发送I2C地址和切换信号，控制所述图像传感器以所述I2C地址和所述切换信号对应的工作模式运行。
  16. 根据权利要求15所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第四工作模式比所述第二工作模式功耗低， 所述第三工作模式比所述第四工作模式功耗低，所述第一工作模式和所述第二工作模式对应第一I2C地址，所述第四工作模式和所述第三工作模式对应第二I2C地址；
    所述处理器向所述图像传感器发送I2C地址和切换信号，控制所述图像传感器以所述I2C地址和所述切换信号对应的工作模式运行，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址和第一切换信号，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第一I2C地址和第二切换信号，控制所述图像传感器以所述第二工作模式运行，或者向所述图像传感器发送所述第二I2C地址和第三切换信号，控制所述图像传感器以所述第四工作模式运行，或者向所述图像传感器发送所述第二I2C地址和第四切换信号，控制所述图像传感器以所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址和所述第三切换信号，控制所述图像传感器以所述第四工作模式运行；或者向所述图像传感器发送所述第二I2C地址和所述第四切换信号，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用光影变化的检测结果，确定所述摄像头检测到光影变化，向所述图像传感器发送所述第二I2C地址和所述第三切换信号，控制所述图像传感器以所述第四工作模式运行，所述光影变化的检测结果，由处于所述第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述处理器根据人脸和/或手势检测结果，确定出所述摄像头拍摄的图像出现人脸和/或手势，向所述图像传感器发送所述第一I2C地址和所述第二切换信号，控制所述图像传感器以所述第二工作模式运行，所述人脸和/或手势检测结果，由所述处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到；
    所处理器根据所述人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势，向所述图像传感器发送所述第二I2C地址和所述第四切换信号，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用较低分辨率图像的图像数据，确定所述电子设备满足所述第二工作模式的退出条件，向所述图像传感器发送所述第二I2C地址和所述第四切换信号，控制所述图像传感器以所述第三工作模式运行，所述较低分辨率图像由处于第二工作模式的图像传感器生成。
  17. 根据权利要求6所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第二工作模式，第三工作模式和第四工作模式，所述第二工作模式比所述第一工作模式功耗低，所述第四工作模式比所述第二工作模式功耗低，所述第三工作模式比所述第四工作模式功耗低，所述第一工作模式对应第一I2C地址，所述第二工作模式，所述第四工作模式和所述第三工作模式对应第二I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行。
  18. 根据权利要求17所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行，包括：
    所述图像传感器接收所述第二I2C地址对应的I2C控制模块传输的控制信号，并控制所述图像传感器以所述控制信号指定的工作模式运行，所述控制信号用于指定所述第三工作模式，第二工作模式或第四工作模式。
  19. 根据权利要求18所述的工作模式的切换控制方法，其特征在于，所述第二I2C地址对应的I2C控制模块传输的控制信号包括：第一控制信号，第二控制信号和第三控制信号；其中：
    所述第一控制信号指定的工作模式为第三工作模式，由所述处理器确定所述摄像头被关闭时生成，或者由所述处理器确定所述电子设备的显示屏被灭屏时生成，或者由所处理器根据所述人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势时生成，或者由所述处理器利用较低分辨率图像的图像数据，确定所述电子设备满足所述第二工作模式的退出条件时生成；
    所述第二控制信号指定的工作模式为第四工作模式，由所述处理器确定所述摄像头被关闭时生成，或者由所述处理器确定所述电子设备的显示屏被灭屏时生成，或者由所述处理器利用光影变化的检测结果，确定所述摄像头检测到光影变化时生成，所述光影变化的检测结果，由处于所述第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述第三控制信号指定的工作模式为第二工作模式，由所述处理器确定所述摄像头被关闭时生成，或者，由所述处理器根据人脸和/或手势检测结果，确定出所述摄像头拍摄的图像出现人脸和/或手势时生成。
  20. 根据权利要求2至5中任意一项所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第三工作模式和第四工作模式，所述第四工作模式比所述第一工作模式功耗低，所述第三工作模式比所述第四工作模式功耗低，所述图像传感器的第一工作模式对应第一I2C地址，所述图像传感器的第四工作模式对应第二I2C地址，所述图像传感器的第三工作模式对应第三I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第四工作模式运行，或者向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第四工作模式运行，或者向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行；
    所述处理器利用光影变化的检测结果，确定所述摄像头检测到光影变化，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第四工作模式运行，所述光影 变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述处理器利用人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势，向所述图像传感器发送所述第三I2C地址，控制所述图像传感器以所述第三工作模式运行，所述人脸和/或手势检测结果，由所述处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
  21. 根据权利要求6所述的工作模式的切换控制方法，其特征在于，所述图像传感器的工作模式包括：第一工作模式，第三工作模式和第四工作模式，所述第四工作模式比所述第一工作模式功耗低，所述第三工作模式比所述第四工作模式功耗低，所述图像传感器的第一工作模式对应第一I2C地址，所述图像传感器的第四工作模式和第三工作模式对应第二I2C地址；其中：
    所述处理器通过I2C接口向所述图像传感器发送信号，包括：
    所述处理器确定所述摄像头被启动，向所述图像传感器发送所述第一I2C地址，控制所述图像传感器以所述第一工作模式运行；
    所述处理器确定所述摄像头被关闭，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第四工作模式或所述第三工作模式运行；
    所述处理器确定所述电子设备的显示屏被灭屏，向所述图像传感器发送所述第二I2C地址，控制所述图像传感器以所述第四工作模式或所述第三工作模式运行。
  22. 根据权利要求21所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行包括：
    所述图像传感器响应第一切换信号，控制所述图像传感器以所述第四工作模式运行，所述第一切换信号由所述处理器或图像传感器利用光影变化的检测结果，确定所述摄像头检测到光影变化时生成，所述光影变化的检测结果，由处于第三工作模式的图像传感器，利用超低分辨率图像的图像数据，进行光影变化的检测得到；
    所述图像传感器响应第二切换信号，控制所述图像传感器以所述第三工作模式运行，所述第二切换信号由所述处理器或图像传感器，利用人脸和/或手势检测结果，确定出所述摄像头拍摄的图像未出现人脸和/或手势而生成，所述人脸和/或手势检测结果，由所述处于第四工作模式的图像传感器，利用低分辨率图像的图像数据，进行人脸和/或手势检测得到。
  23. 根据权利要求21所述的工作模式的切换控制方法，其特征在于，所述图像传感器响应切换信号，控制所述图像传感器以对应同一个所述I2C地址的工作模式中的一种工作模式运行，包括：
    所述图像传感器接收所述第二I2C地址对应的I2C控制模块传输的控制信号，并控制所述图像传感器以所述控制信号指定的工作模式运行，所述控制信号用于指定所述第三工作模式或第四工作模式。
  24. 根据权利要求1至23中任意一项所述的工作模式的切换控制方法，其特征在于，发送所述信号的对象，包括：
    所述处理器的摄像头驱动，或所述处理器的图像信号处理器中的最简图像前端或图像前端，或所述处理器的应用处理器，或所述处理器的智能传感集线器。
  25. 根据权利要求6所述的工作模式的切换控制方法，其特征在于，所述切换信号由所述处理器的图像信号处理器中的最简图像前端或图像前端，或所述处理器的应用处理器，或智能传感集线器，向所述图像传感器发送。
  26. 一种电子设备，其特征在于，包括：
    显示屏；
    摄像头，所述摄像头包括图像传感器，所述图像传感器包括至少两种工作模式；
    一个或多个处理器；
    存储器，其上存储有程序；
    当所述程序被所述一个或多个处理器执行时，使得所述电子设备执行如权利要求1至25任意一项所述的工作模式的切换控制方法。
  27. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至25中任意一项所述的工作模式的切换控制方法。
PCT/CN2022/089195 2021-08-19 2022-04-26 工作模式的切换控制方法、电子设备及可读存储介质 WO2023019999A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/547,641 US20240137649A1 (en) 2021-08-19 2022-04-25 Operation mode switching control method, electronic device, and readable storage medium
EP22857317.6A EP4287605A1 (en) 2021-08-20 2022-04-26 Working mode switching control method, electronic device, and readable storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110971250.X 2021-08-19
CN202110971250 2021-08-20
CN202210099723.6A CN115714908A (zh) 2021-08-20 2022-01-27 工作模式的切换控制方法、电子设备及可读存储介质
CN202210099723.6 2022-01-27

Publications (1)

Publication Number Publication Date
WO2023019999A1 true WO2023019999A1 (zh) 2023-02-23

Family

ID=85230481

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/089195 WO2023019999A1 (zh) 2021-08-19 2022-04-26 工作模式的切换控制方法、电子设备及可读存储介质

Country Status (4)

Country Link
US (1) US20240137649A1 (zh)
EP (1) EP4287605A1 (zh)
CN (1) CN115714908A (zh)
WO (1) WO2023019999A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547158A (zh) * 2012-02-24 2012-07-04 北京思比科微电子技术股份有限公司 具有采样控制功能的图像传感器、多图像传感器系统及工作方法
US20170359499A1 (en) * 2016-06-14 2017-12-14 Intel Corporation Transmission of Image Data and Camera Management Commands
CN110177199A (zh) * 2019-06-28 2019-08-27 Oppo广东移动通信有限公司 图像传感器芯片、摄像头模组、电子设备以及影像拍摄方法
TW202110144A (zh) * 2019-08-28 2021-03-01 大陸商南京深視光點科技有限公司 基於行動產業處理器介面的資料傳輸系統及資料傳輸方法
CN112929560A (zh) * 2019-12-06 2021-06-08 三星电子株式会社 图像处理设备及其方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101505434A (zh) * 2009-03-12 2009-08-12 浙江大学 具有全局同步功能的高分辨率智能网络摄像机阵列系统
CN102929826B (zh) * 2012-11-16 2016-05-11 深圳市建恒测控股份有限公司 一种单一超声波信号总线传输多路超声波信号系统与方法
CN105430276A (zh) * 2015-12-18 2016-03-23 广东欧珀移动通信有限公司 成像装置和电子装置
US10795850B2 (en) * 2019-02-26 2020-10-06 Texas Instruments Incorporated Methods and apparatus to transition devices between operational states
JP7379932B2 (ja) * 2019-08-23 2023-11-15 オムロン株式会社 制御装置、ユーザプログラムの実行制御方法、およびシステムプログラム
KR20210090476A (ko) * 2020-01-10 2021-07-20 삼성전자주식회사 단일 카메라로 줌 시나리오에서 화질 향상 방법과 이를 포함하는 전자장치
CN112148662B (zh) * 2020-08-17 2024-02-09 上海赛昉科技有限公司 利用i2c地址匹配唤醒的低功耗芯片架构及唤醒方法
CN114202000A (zh) * 2020-08-31 2022-03-18 华为技术有限公司 一种业务处理方法及设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547158A (zh) * 2012-02-24 2012-07-04 北京思比科微电子技术股份有限公司 具有采样控制功能的图像传感器、多图像传感器系统及工作方法
US20170359499A1 (en) * 2016-06-14 2017-12-14 Intel Corporation Transmission of Image Data and Camera Management Commands
CN110177199A (zh) * 2019-06-28 2019-08-27 Oppo广东移动通信有限公司 图像传感器芯片、摄像头模组、电子设备以及影像拍摄方法
TW202110144A (zh) * 2019-08-28 2021-03-01 大陸商南京深視光點科技有限公司 基於行動產業處理器介面的資料傳輸系統及資料傳輸方法
CN112929560A (zh) * 2019-12-06 2021-06-08 三星电子株式会社 图像处理设备及其方法

Also Published As

Publication number Publication date
CN115714908A (zh) 2023-02-24
EP4287605A1 (en) 2023-12-06
US20240137649A1 (en) 2024-04-25

Similar Documents

Publication Publication Date Title
WO2021227770A1 (zh) 应用窗口显示方法和电子设备
WO2021027747A1 (zh) 一种界面显示方法及设备
WO2023016008A1 (zh) 参数的调整方法、显示的控制方法、电子设备及介质
US20230419570A1 (en) Image Processing Method and Electronic Device
WO2022105445A1 (zh) 基于浏览器的应用投屏方法及相关装置
WO2020155875A1 (zh) 电子设备的显示方法、图形用户界面及电子设备
CN116075808A (zh) 一种图像处理方法及电子设备
WO2023016017A1 (zh) 显示屏的控制方法及电子设备
WO2021042878A1 (zh) 一种拍摄方法及电子设备
WO2022262550A1 (zh) 一种拍摄视频的方法及电子设备
US11995317B2 (en) Method and apparatus for adjusting memory configuration parameter
CN114531519B (zh) 一种基于垂直同步信号的控制方法及电子设备
WO2023019999A1 (zh) 工作模式的切换控制方法、电子设备及可读存储介质
WO2023040670A1 (zh) 桌面动效显示方法及电子设备
WO2023030168A1 (zh) 界面显示方法和电子设备
WO2022089216A1 (zh) 一种界面显示的方法和电子设备
WO2023169276A1 (zh) 投屏方法、终端设备及计算机可读存储介质
WO2022247664A1 (zh) 图形界面显示方法、电子设备、介质以及程序产品
WO2022252816A1 (zh) 显示方法及电子设备
WO2023130929A1 (zh) 熄屏显示的控制方法、电子设备及存储介质
WO2024114285A1 (zh) 显示方法、显示装置和可穿戴设备
WO2023116012A1 (zh) 屏幕显示方法和电子设备
WO2023130927A1 (zh) 熄屏显示的控制方法、电子设备及存储介质
WO2023130928A1 (zh) 熄屏显示的控制方法、电子设备及介质
WO2022252805A1 (zh) 显示方法及电子设备

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 18547641

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022857317

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022857317

Country of ref document: EP

Effective date: 20230831

NENP Non-entry into the national phase

Ref country code: DE