WO2024179072A1 - 相机应用配置方法、设备及存储介质 - Google Patents

相机应用配置方法、设备及存储介质 Download PDF

Info

Publication number
WO2024179072A1
WO2024179072A1 PCT/CN2023/134117 CN2023134117W WO2024179072A1 WO 2024179072 A1 WO2024179072 A1 WO 2024179072A1 CN 2023134117 W CN2023134117 W CN 2023134117W WO 2024179072 A1 WO2024179072 A1 WO 2024179072A1
Authority
WO
WIPO (PCT)
Prior art keywords
sensor
thread
camera application
channel
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/134117
Other languages
English (en)
French (fr)
Other versions
WO2024179072A9 (zh
Inventor
沈涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Publication of WO2024179072A1 publication Critical patent/WO2024179072A1/zh
Publication of WO2024179072A9 publication Critical patent/WO2024179072A9/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Definitions

  • the present application relates to the field of terminal technology, and in particular to a camera application configuration method, device and storage medium.
  • terminal devices usually provide camera applications (application, APP), and the camera application supports photo taking and video recording functions.
  • application application, APP
  • the user when a user wants to capture images of certain scenes, the user triggers a camera application and takes a photo or video of the target scene image.
  • the user interface (UI) of the camera application takes a long time to display the image, resulting in the user being unable to capture the target scene image.
  • the embodiments of the present application provide a camera application configuration method, device, and storage medium, which are applied to the field of terminal technology and reduce the time consumed from when a camera application is triggered to when an image is displayed on the interface of the camera application.
  • an embodiment of the present application provides a camera application configuration method.
  • the method includes:
  • the camera parameters are obtained through the first thread
  • the sensor is initialized through the second thread;
  • the sensor includes a sensor corresponding to the camera application;
  • the sensor initialization is related to the camera parameters;
  • the data stream of the camera application is configured through the third thread; the data stream includes data streams corresponding to various functions of the camera application; the data stream configuration is related to the camera parameters and the parameters obtained in the sensor initialization, and the third moment is later than the second moment.
  • the sensor initialization start time of the sensor corresponding to the camera application is earlier than the start time of data stream configuration for the camera application, so that the sensor initialization of the sensor corresponding to the camera application is completed in advance, and the available memory of the electronic device and the available CPU resources of the electronic device before the third thread starts to execute can be effectively utilized, so that the sensor initialization can be completed quickly, and the competition of the sensor initialization for the memory and CPU resources required by other processes corresponding to the third thread is reduced, so that the third thread corresponding to other processes can obtain sufficient memory and CPU resources during execution, reducing the time consumption of sensor initialization and the time consumption of data stream configuration of the camera application, thereby reducing the time consumption from the camera application being triggered to the image being displayed on the interface of the camera application.
  • the method before the sensor is initialized by the second thread, the method further includes:
  • Detecting a logical identifier of a camera application where the logical identifier includes an identifier of a first mode or an identifier of a second mode; the first mode corresponds to m sensors, the second mode corresponds to one sensor, and m is an integer greater than 0;
  • the logical identifier is the identifier of the first mode, create the first channel corresponding to the camera application and create m second lines Procedure;
  • Initializing the sensor through the second thread includes: respectively initializing the m sensors through m second threads; wherein the first channel is used to manage the interface between the camera application and the m sensors.
  • each sensor can be initialized. If the camera application corresponds to multiple sensors, the total time consumption of initializing the sensors corresponding to the camera application can be reduced by concurrently initializing the m sensors through m second threads.
  • initializing the m sensors respectively by using the m second threads includes:
  • any second thread of the m second threads executes: applying for resource cache, obtaining the second channel of any sensor among the m sensors from the first channel, and writing first register data to any sensor; the second channel is used to power on any sensor.
  • the first register data of each sensor sent by the camera application can be transmitted to the kernel driver layer, and each sensor can be powered on through the second channel of each sensor so that the kernel driver layer can write the first register data into each sensor to realize sensor initialization of each sensor.
  • the resource cache includes a package cache
  • the first register data is transmitted from the application layer to the kernel driver layer, so that the first register data is written into each sensor through the kernel driver layer to implement sensor initialization of each sensor.
  • a second thread is created
  • Initializing the sensor through the second thread includes: initializing a sensor through a second thread.
  • a camera application corresponds to a sensor
  • a second thread can be directly created and the sensor corresponding to the camera application can be initialized through the second thread. This can save time for sensor initialization and facilitate management of the sensor initialization thread.
  • initializing a sensor through a second thread includes:
  • the following steps are executed through a second thread: applying for a resource cache, creating a first channel, obtaining a second channel of a sensor from the first channel, and writing first register data to a sensor; the second channel is used to power on a sensor.
  • the first register data of the sensor sent by the camera application can be transmitted to the kernel driver layer, and the sensor can be powered on through the second channel of the sensor so that the kernel driver layer can write the first register data into the sensor to implement sensor initialization of the sensor.
  • the process of creating the first channel is placed in the second thread, which saves the time of sensor initialization and facilitates the management of the sensor initialization thread.
  • the resource cache includes a package cache
  • the first register data is transmitted from the application layer to the kernel driver layer, so that the first register data is written into the sensor through the kernel driver layer to implement sensor initialization of the sensor.
  • the second thread includes a corresponding synchronization lock; the synchronization lock is used to lock the execution object corresponding to the second thread.
  • the method further includes:
  • the process state machines corresponding to the first channel, the second thread, and the second channel and the second thread are maintained in global variables; the process state machine includes a directed graph consisting of the process state, process transition, and process action of the process, and the process state machine corresponding to the second thread includes a sensor power-on state machine and a first register data writing sensor completion state machine.
  • the data stream configuration of the camera application is performed through a third thread, including:
  • the nodes include nodes for initializing the sensor
  • the first channel is obtained from the global variable
  • the second channel of each sensor is obtained from the global variable
  • the process of writing the first register data into the sensor is skipped, and the second register data is written into each sensor through the second channel of each sensor; the second register data includes the flow distribution register data selected according to the size of the data flow.
  • the method further includes:
  • the method further includes:
  • a logical identifier of the camera application is detected;
  • the logical identifier includes an identifier of a first mode or an identifier of a second mode, the first mode corresponds to m sensors, the second mode corresponds to one sensor, and m is an integer greater than 0;
  • the logic mark is the mark of the first mode
  • the first channel is released, the sensor power-on state machine is initialized, and the sensor is released.
  • the second channel of the sensor and the second thread are deregistered;
  • the sensor power-on state machine is initialized, the second channel of the sensor is released, and the second thread is deregistered.
  • the adaptability of the camera application to other applications can be improved, so that the second thread can be normally created and executed after other applications trigger the camera application, and further, after the camera application is triggered by other applications, the initialization of the sensor corresponding to the camera application and the data stream configuration of the camera application can be quickly completed.
  • the method further includes:
  • the threads and the first channel of each node corresponding to the camera application are destroyed; and the second channel is destroyed.
  • the electronic device 100 has more available CPU resources and more available input/output (I/O) resources.
  • an embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program.
  • the computer program is executed by a processor, the method of the first aspect is implemented.
  • an embodiment of the present application provides a computer program product, which includes a computer program.
  • the computer program When the computer program is executed, the computer executes the method of the first aspect.
  • an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method of the first aspect.
  • FIG1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application.
  • FIG. 2 is a schematic diagram of the software structure of the electronic device 100 provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of a user triggering a camera application to take a photo in a possible implementation
  • FIG4 is a schematic diagram of a process of displaying an image after a camera application is triggered in a possible implementation
  • FIG5 is a schematic diagram of a user triggering a camera application to take a photo according to an embodiment of the present application
  • FIG6 is a schematic diagram of a process of displaying an image after a camera application is triggered according to an embodiment of the present application
  • FIG. 7 is a diagram showing the corresponding architecture of a camera application in which a sensor initialization node is advanced according to an embodiment of the present application
  • FIG8 is a flowchart of a camera application configuration method according to an embodiment of the present application.
  • FIG9 is a second flow chart of a camera application configuration method provided in an embodiment of the present application.
  • FIG10 is a schematic diagram of a multi-camera mode provided in an embodiment of the present application.
  • FIG11 is a schematic diagram of a single-camera mode provided in an embodiment of the present application.
  • FIG12 is a schematic diagram of the hardware structure of the chip provided in an embodiment of the present application.
  • Sensor It can be called a photosensitive element, image sensor or sensor; it is used to convert light signals into electrical signals.
  • Session can be called session control; used to display the control interface of the camera device.
  • buffer can be called buffer or cache.
  • Package buffer It can be called packet buffer; it is used to store or transfer data to be transmitted during data transmission. The data to be transmitted needs to be encapsulated into encapsulation packets and then stored in the packet buffer.
  • Camera ID It can be called sensor ID. For example, one sensor corresponds to one sensor ID.
  • User case can be called an instance; it is used to create configuration parameters for camera data stream processing nodes; user case supports the creation and algorithm configuration of various camera modes, such as single-camera mode, multi-camera mode, etc.
  • handle It can be called a handle, which is used to create a camera device driver channel and also used to create or obtain a sensor driver channel.
  • Session handle It can be called session control handle or camera device control interface, which is used to create camera device driver channel.
  • Devices handle It can be called sensor handle or sensor control interface, which is used to create or obtain sensor drive channels.
  • Pipeline It can be called pipeline, assembly line or multi-threading, which includes threads of each node corresponding to the camera application, such as sensor initialization thread, image algorithm thread, etc.
  • ION It is a subsystem used for memory management in Android. ION means zero-copy. ION represents the memory that transfers data between the user space and kernel space of the system (such as the Android system). This memory can be directly called in the user space and kernel space to obtain data.
  • the words “exemplary” or “for example” are used to indicate examples, illustrations or descriptions. Any embodiment or design described as “exemplary” or “for example” in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as “exemplary” or “for example” is intended to present related concepts in a specific way.
  • At least one refers to one or more, and “more than one” refers to two or more.
  • “And/or” describes the association relationship of associated objects, indicating that three relationships may exist.
  • a and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.
  • the character “/” generally indicates that the previous and next associated objects are in an “or” relationship.
  • “At least one of the following” or similar expressions refers to any combination of these items, including any combination of single or plural items.
  • At least one of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple.
  • the "at" in the embodiment of the present application can be the instant when a certain situation occurs, or a period of time after a certain situation occurs, and the embodiment of the present application does not specifically limit this.
  • the display interface provided in the embodiment of the present application is only an example, and the display interface can also include more or less content.
  • the electronic devices of the embodiments of the present application may include handheld devices with image processing functions, vehicle-mounted devices, etc.
  • some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, etc.
  • the present invention relates to wireless terminals in the art city, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDA), handheld devices with wireless communication functions, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices (such as car computers), wearable devices, terminal devices in 5G networks or terminal devices in future evolved public land mobile communication networks (PLMN), etc., and the embodiments of the present application are not limited to this.
  • SIP session initiation protocol
  • WLL wireless local loop
  • PDA personal digital assistants
  • handheld devices with wireless communication functions computing devices or other processing devices connected to wireless modems
  • vehicle-mounted devices such as car computers
  • wearable devices terminal devices in 5G networks or terminal devices in future evolved public land mobile communication networks (PLMN), etc.
  • PLMN public land mobile communication networks
  • the electronic device may also be a wearable device.
  • Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as hearing aids, glasses, gloves, watches, clothing, and shoes.
  • a wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction.
  • wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.
  • the electronic device can also be a terminal device in the Internet of Things (IoT) system.
  • IoT Internet of Things
  • Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.
  • the electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.
  • UE user equipment
  • MS mobile station
  • MT mobile terminal
  • access terminal user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.
  • the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer.
  • the hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory).
  • the operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system.
  • the application layer includes applications such as cameras, browsers, address books, word processing software, and instant messaging software.
  • FIG. 1 shows a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver
  • the present invention relates to a speaker 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100.
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently.
  • the components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc.
  • AP application processor
  • GPU graphics processor
  • ISP image signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • Different processing units may be independent devices or integrated into one or more processors.
  • the controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (SIM) interface, and/or a universal serial bus (USB) interface, etc.
  • I2C inter-integrated circuit
  • I2S inter-integrated circuit sound
  • PCM pulse code modulation
  • UART universal asynchronous receiver/transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may include multiple groups of I2C buses.
  • the processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces.
  • the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.
  • the I2S interface can be used for audio communication.
  • the processor 110 can include multiple I2S buses.
  • the processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170.
  • the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.
  • the PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals.
  • the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 via a PCM interface to implement the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • the UART interface is generally used to connect the processor 110 and the wireless communication module 160.
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.
  • the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193.
  • the MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc.
  • the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100.
  • the processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc.
  • the GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.
  • the USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones.
  • the interface can also be used to connect other electronic devices, such as AR devices, etc.
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100.
  • the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
  • the charging management module 140 is used to receive charging input from a charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 140 may receive charging input from a wired charger through the USB interface 130.
  • the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc.
  • the power management module 141 can also be set in the processor 110.
  • the power management module 141 and the charging management module 140 can also be set in the same device.
  • the electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor.
  • the GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos, etc.
  • the display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display.
  • the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.
  • the electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
  • ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.
  • the camera 193 is used to capture still images or videos.
  • the object generates an optical image through the lens and projects it onto the photosensitive element.
  • the photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor.
  • CMOS complementary metal oxide semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • the DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.
  • the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • the digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
  • Video codecs are used to compress or decompress digital videos.
  • the electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • MPEG Moving Picture Experts Group
  • MPEG2 MPEG2, MPEG3, MPEG4, etc.
  • NPU is a neural network (NN) computing processor.
  • NN neural network
  • applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.
  • the internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions.
  • the internal memory 121 may include a program storage area and a data storage area.
  • the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.
  • the data storage area may store data created during the use of the electronic device 100 (such as audio data, image data, phone book, etc.), etc.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • the pressure sensor 180A can be set on the display screen 194.
  • the capacitive pressure sensor can be a parallel plate including at least two conductive materials.
  • the electronic device 100 determines the intensity of the pressure based on the change in capacitance.
  • the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A.
  • the electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.
  • the gyro sensor 180B can be used to determine the motion posture of the electronic device 100.
  • the angular velocity of the electronic device 100 around three axes i.e., x, y, and z axes
  • the gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.
  • the distance sensor 180F is used to measure the distance.
  • the electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.
  • the ambient light sensor 180L is used to sense the brightness of the ambient light.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
  • the touch sensor 180K is also called a "touch control device”.
  • the touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen”.
  • the touch sensor 180K is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194.
  • the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.
  • the key 190 includes a power key, a volume key, etc.
  • the key 190 may be a mechanical key or a touch key.
  • the electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
  • Motor 191 can generate vibration prompts.
  • Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
  • touch operations acting on different applications can correspond to different vibration feedback effects.
  • touch operations acting on different areas of the display screen 194 can also correspond to different vibration feedback effects.
  • Different application scenarios for example: time reminders, receiving messages, alarm clocks, games, etc.
  • the touch vibration feedback effect can also support customization.
  • Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture.
  • the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.
  • FIG. 2 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.
  • the layered architecture divides the software into several layers, each with a clear role and division of labor.
  • the layers communicate with each other through software interfaces.
  • the Android system is divided into five layers, from top to bottom: application layer, application framework layer, hardware abstraction layer (HAL), kernel driver layer, and hardware layer.
  • HAL hardware abstraction layer
  • the application layer may include a series of application packages.
  • the application package may include applications such as camera, gallery, calendar, phone, map, navigation, WLAN, Bluetooth, music, video, information, etc. as shown in FIG. 2 .
  • the application framework layer provides application programming interface (API) and programming framework for the application programs in the application layer.
  • API application programming interface
  • the application framework layer includes some predefined functions.
  • the application framework layer may include a camera access interface and a window manager.
  • the camera access interface is used to establish communication channels between the camera application and the camera management and camera device.
  • the camera access interface can include a Java native interface (JNI interface).
  • JNI interface Java native interface
  • the window manager is used to manage window programs.
  • the window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • the hardware abstraction layer may include a camera hardware abstraction layer, a camera algorithm library, and a camera service level (CSL).
  • the camera hardware abstraction layer includes at least one camera device.
  • the camera algorithm library includes image algorithms corresponding to the camera device of the camera hardware abstraction layer.
  • the camera device may include a sensor, an image signal processor, a digital signal processor, and an image processor.
  • the camera service layer is used for data interaction between the hardware abstraction layer and the kernel driver layer.
  • the camera service layer provides an interface for interaction between the hardware abstraction layer and the kernel driver layer.
  • the kernel driver layer is the layer between hardware and software.
  • the kernel driver layer contains at least camera device driver, digital signal processor driver, image processor driver.
  • the kernel driver layer can use the kernel driver framework of video device (Video for linux2, V4L2) to facilitate reading and writing camera devices such as sensors.
  • the hardware layer includes at least one sensor, an image signal processor, a digital signal processor, and an image processor.
  • FIG. 3 is a schematic diagram of a user triggering a camera application to take a photo in a possible implementation.
  • FIG. 4 is a schematic diagram of the process of displaying an image after the camera application is triggered in a possible implementation.
  • the user can trigger the camera application by touching and clicking the control of the camera application on the electronic device 100, and the user can also trigger the camera application by switching the current application of the electronic device 100 to the operation of the camera application.
  • the corresponding hardware interrupt is sent to the kernel driver layer.
  • the kernel driver layer processes the touch operation into raw input events (including touch coordinates, timestamp of the touch operation and other information).
  • the raw input events are stored in the kernel driver layer.
  • the application framework layer obtains the raw input event from the kernel driver layer and identifies that the control corresponding to the input event is the control of the camera application icon.
  • the camera application (APP) of the electronic device 100 calls the camera access interface of the application framework layer,
  • the first thread is executed through the HAL layer to obtain camera parameters and the first resource.
  • the first resource includes the created HAL executor.
  • the first thread is such as the startup thread (open thread) of the camera application.
  • the distribution thread of the HAL layer is called to configure the data flow of the camera application.
  • the distribution thread execution (also called distribution) is to create a first channel (such as a session handle) corresponding to the camera application and create each node of the camera application.
  • the first channel is also called the camera device driver channel.
  • the first channel is used to manage the interface between the camera application and at least one camera device.
  • the camera use case includes a multi-camera mode or a single-camera mode.
  • Creating each node of the camera application includes creating a thread (pipeline) of each node.
  • the node is such as a sensor initialization node, an image algorithm configuration node, a sensor start-up node, etc.
  • Sensor initialization includes powering on the sensor corresponding to the camera application by calling the kernel driver layer, and writing initialization register data and calibration register data to the sensor.
  • Sensor start-up includes starting the camera device driver after writing the distribution register data to the sensor corresponding to the camera application by calling the kernel driver layer, and collecting scene images and outputting scene images through the sensor.
  • the camera application of the electronic device 100 sends a preview request and a rotating buffer (rotating buffer) to the camera hardware abstraction layer. Sending a preview request and a rotating buffer (rotating buffer) is called repeating.
  • the camera hardware abstraction layer of the electronic device 100 calls the image algorithm in the camera algorithm library, processes the scene image output by the sensor, and writes the processed image to the rotating buffer.
  • the camera application displays the image preview in the rotating buffer to the user interface (UI) of the camera application.
  • UI user interface
  • the user can take a photo or record a video of the scene image.
  • the photo or video operation representation is used to save the image displayed on the interface.
  • the data flow configuration of the camera application will take a long time. Therefore, when the user triggers the camera application, the camera application interface will take a long time to display the scene image, which will cause the user to be unable to capture the target scene image they want to capture.
  • some possible camera application system architectures use a serial execution method of executing the first thread--distribution--repeating--(preview) displaying the image as shown in Figure 4 to realize the image display after the camera application is triggered.
  • the time consumption of any of the multiple threads executed in series is extended, the time consumption of the image display after the camera application is triggered is extended. Since the node creation of sensor initialization in the process corresponding to the distribution thread is later than the start time of the distribution thread execution, and the data volume of the initialization register data and the calibration register data is large, the node creation of sensor initialization takes a long time, which in turn makes the execution of the distribution thread take a long time.
  • an embodiment of the present application proposes a camera application configuration method, which is applied to an electronic device.
  • the start time of the sensor initialization node creation is made earlier than the start time of the distribution thread execution.
  • the sensor initialization node creation is completed in advance, and on the other hand, the electronic device memory and the electronic device CPU resources before the distribution thread starts to execute can be effectively utilized, so that the sensor initialization node creation can be completed quickly, and the competition of the sensor initialization node creation for the memory and CPU resources required by other nodes corresponding to the distribution thread is reduced, so that the distribution thread corresponding to other processes can obtain sufficient memory and CPU resources during the execution process, thereby reducing the time consumption of the sensor initialization node creation and the distribution thread execution, so that after the camera application is triggered, the camera application interface quickly displays the image.
  • FIG. 5 is a schematic diagram of a user triggering a camera application to take a photo provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a process flow of an image display after a camera application provided by an embodiment of the present application is triggered.
  • the touch sensor of the electronic device 100 receives a touch single-click operation by the user, the corresponding hardware interrupt is sent to the kernel driver layer.
  • the kernel driver layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation, and other information).
  • the original input event is stored in the kernel driver layer.
  • the application framework layer obtains the original input event from the kernel driver layer and identifies that the control corresponding to the input event is a control of the camera application icon.
  • the camera application (APP) of the electronic device 100 calls the camera access interface of the application framework layer, and executes the first thread through the HAL layer to obtain the camera parameters and the first resource.
  • the first resource includes the created HAL executor.
  • the first thread is such as the startup thread (open thread) of the camera application.
  • the camera application of the electronic device 100 initializes the sensor corresponding to the camera application through the second thread. Sensor initialization is related to the camera parameters.
  • the camera application configures the data flow of the camera application through the third thread. Data stream configuration is also called stream allocation.
  • the camera application configures the data stream of the camera application through the third thread, including initializing the camera device driver channel (session handle), creating other nodes corresponding to the camera application such as the image algorithm configuration node, and starting the stream.
  • the data stream includes data streams corresponding to various functions of the camera application.
  • the data stream configuration is related to the camera parameters and the parameters obtained in the sensor initialization.
  • the third moment is later than the second moment.
  • the execution start time of the second thread is earlier than the execution start time of the third thread, so that the sensor initialization of the sensor corresponding to the camera application can be completed in advance.
  • the camera application of the electronic device 100 sends a preview request and a rotating buffer (rotating buffer) to the camera hardware abstraction layer. Sending a preview request and a rotating buffer (rotating buffer) is also called repeating.
  • the camera hardware abstraction layer of the electronic device 100 calls the image algorithm in the camera algorithm library, processes the scene image output by the sensor, and writes the processed image into the rotating buffer.
  • the camera application of the electronic device 100 displays the image preview in the rotating buffer to the user interface (UI) of the camera application. The user can then take a photo or record a video of the scene image. The photo or video operation is used to save the image displayed on the interface.
  • UI user interface
  • the corresponding time taken for the camera application in the possible implementation shown in FIG3 is (T+730) milliseconds, while the corresponding time taken for the camera application provided by the embodiment of the present application shown in FIG5 is (T+600) milliseconds.
  • the scene image displayed on the interface of the camera application shown in FIG5 is the target scene image to be captured by the user. If the camera application on the user's electronic device 100 is the camera application shown in FIG3, the user will not be able to capture the target scene image shown in FIG5.
  • the interface of the camera application displays the implementation principle and technical effect of the scene image, which is similar to the implementation principle and technical effect of the camera application interface displaying the scene image after the user touches and clicks the camera application as shown in the embodiment of the present application, and this embodiment will not be repeated here.
  • the camera application configuration method provided in the embodiment of the present application enables the sensor initialization of the sensor corresponding to the camera application to be completed in advance, accelerates the data flow configuration of the camera application, and can reduce the time from the camera application being triggered to the camera application interface displaying the scene image. If the camera application on the user's electronic device adopts the camera application configuration method provided in the embodiment of the present application, after the user triggers the camera application, the interface of the camera application will promptly display the target scene image that the user wants to capture, allowing the user to complete the capture of the target scene image.
  • FIG. 7 is a diagram of the corresponding architecture of the camera application in advance of the sensor initialization node provided in an embodiment of the present application.
  • the architecture includes a camera application (APP), an application framework (framework) and a kernel driver layer (kernel sensor driver).
  • the application architecture includes: camera application API, hardware environment area, user end (sensor customer) and sensor node.
  • the camera application API can be used to trigger and close the camera application.
  • the hardware environment area can initialize global parameters, initialize the thread manager, provide a sensor power-on process entry, provide a thread data synchronization delay mechanism, provide a sensor initialization process global state machine configuration/acquisition entry, provide a process exit, clear data and reclaim memory process.
  • the user end can create a second thread, create a second thread synchronization lock, create a first channel, obtain a second channel, configure parameters such as process switches and logical identifiers, and release asynchronous threads such as the second thread and CSL resources.
  • the sensor node can use the first channel and the second channel, skip the first register data writing process, and release the first channel and second channel related resources.
  • the kernel driver layer can power on or off the sensor, and can also write the second register data.
  • FIG. 8 is a flowchart of the camera application configuration method provided in an embodiment of the present application.
  • Figure 9 is a flowchart of the camera application configuration method provided in an embodiment of the present application.
  • Figure 10 is a schematic diagram of a multi-camera mode provided in an embodiment of the present application.
  • Figure 11 is a schematic diagram of a single-camera mode provided in an embodiment of the present application.
  • the execution subject of the embodiment shown in Figure 8 can be the electronic device 100 in the embodiment shown in Figure 1. As shown in Figure 8, the method includes:
  • the electronic device 100 responds to an instruction to trigger a camera application, and at a first moment, the electronic device 100 calls a first thread, obtains camera parameters and creates a HAL executor through the first thread.
  • the first thread is such as an open thread.
  • the execution of the first thread is provided by the HAL layer.
  • the camera parameters include a sensor identifier.
  • the sensor is initialized through a second thread.
  • the sensor includes a sensor corresponding to the camera application.
  • the sensor initialization is related to the camera parameters.
  • the electronic device 100 initializes the sensor corresponding to the camera application through the second thread.
  • the electronic device 100 detects the logical ID of the camera application.
  • the logical ID includes an ID of the first mode or an ID of the second mode.
  • the first mode corresponds to m sensors, and the second mode corresponds to one sensor, where m is an integer greater than 0.
  • the first mode is, for example, the multi-camera mode shown in FIG. 10.
  • the second mode is, for example, the single-camera mode shown in FIG. 11.
  • the logical ID can be used to determine an instance (user case) of the camera application, such as a single-camera mode or a multi-camera mode.
  • a first channel corresponding to the camera application is created, and m second threads are created.
  • the first channel can be used to manage the interface between the camera application and the m sensors.
  • the electronic device 100 performs sensor initialization on the m sensors respectively through the m second threads.
  • any second thread of the m second threads any second thread executes: applying for resource cache, obtaining the second channel of any sensor of the m sensors from the first channel, and writing the first register data to any sensor.
  • the second channel is used to power on any sensor.
  • the electronic device 100 uses the session control handle of the CSL service to create a first channel between the camera application and the camera device.
  • the electronic device 100 creates a session control handle of the driver node of the camera device including the sensor through the CSL open function to obtain the first channel, so as to obtain the second channel of driving the sensor through the first channel.
  • the electronic device 100 implements communication between the camera application and the hardware abstraction layer through the first channel.
  • the interface between the camera application and the m sensors can be devices handle 1 and devices handle 2 as shown in Figure 10.
  • the created m second threads can be divided into They correspond to the sensor identifiers (camera ids) of the m sensors.
  • the electronic device 100 initializing the m sensors respectively through the m second threads may include: the electronic device 100 registers a callback function (such as a sensor advance power thread job callback function) to the second thread through a first function (such as a register job family function), configures the thread priority and parameter transfer buffer, and starts the second thread through a startup function (such as a post job function).
  • a callback function such as a sensor advance power thread job callback function
  • a first function such as a register job family function
  • a startup function such as a post job function
  • the electronic device 100 creates a corresponding synchronization lock while creating each second thread.
  • the synchronization lock is used to lock the execution object corresponding to the second thread to obtain an accurate execution result or clean execution result data.
  • the electronic device 100 creates two second threads, namely, the second thread 1 and the second thread 2.
  • the electronic device 100 performs sensor initialization on the sensor 1 and the sensor 2 as shown in FIG. 10 through the second thread 1 and the second thread 2, respectively.
  • the electronic device 100 uses the synchronization lock of the second thread 1 to lock the object H when the second thread 1 corresponds to the process h.
  • the electronic device 100 After the electronic device 100 executes the process h corresponding to the second thread 1, it unlocks the object H so that the electronic device 100 locks the object H when the process h corresponding to the second thread 2 is executed. In this way, multiple second threads executed in parallel will not simultaneously compete for the same execution object, so that the execution result corresponding to each second thread is accurate, and the execution result data obtained for each second thread is clean.
  • the electronic device 100 executes to obtain the second channel of any one of the m sensors from the first channel, for example: the electronic device 100 retrieves the control interface (devices handle) of any one of the m sensors through the first channel (or session handle) to obtain the second channel of any one of the m sensors.
  • the control interface of any sensor is devices handle 1 or devices handle 2 as shown in FIG10 .
  • the control interface (devices handle) of any one of the m sensors may be created or retrieved by the electronic device 100 through an acquisition function (such as a CSL acquire device function).
  • the resource cache includes a packet buffer.
  • the electronic device 100 executes an application for a resource cache, including the electronic device 100 applying for a packet buffer from the kernel driver layer through the camera service layer (CSL).
  • the packet buffer may be an ION buffer.
  • the packet buffer is used for data transmission.
  • the ION buffer may provide a common memory address for the hardware abstraction layer and the kernel driver layer.
  • the kernel driver layer may directly use the data in the ION buffer to execute subsequent processes without copying each other, thereby improving the efficiency of data transmission.
  • the process of the electronic device 100 executing writing the first register data to any sensor may include the following steps S1021-S1024:
  • the electronic device 100 obtains first register data from a camera application.
  • the electronic device 100 stores the first encapsulation packet obtained by encapsulating the first register data in a packet buffer, thereby transmitting the first register data from the application layer to the kernel driver layer.
  • the electronic device 100 parses the first encapsulated packet obtained from the packet buffer area to obtain the parsed first register data. For example, the electronic device 100 obtains the first encapsulated packet in the packet buffer area of the kernel driver layer. The electronic device 100 parses the first encapsulated packet in the kernel driver layer to obtain the parsed first register data.
  • the electronic device 100 writes the parsed first register data into any sensor.
  • the device 100 writes the first register data parsed by the kernel driver layer into any sensor through an I2C bus (inter-integrated circuit).
  • the electronic device 100 can write the parsed initialization register data into the sensor by calling a second function (such as load sensor init config cmd function), and write the parsed calibration register data into the sensor by calling a third function (such as load sensor config cmds function).
  • a second function such as load sensor init config cmd function
  • a third function such as load sensor config cmds function
  • step S1021 may be performed before the electronic device 100 executes the resource cache application process, so that the electronic device 100 applies for a packet buffer that matches the data size of the first register, thereby achieving reasonable and effective use of the memory resources of the electronic device 100.
  • a second thread is created. Since the camera application whose logical identifier is an identifier of the second mode corresponds to a sensor, the electronic device 100 initializes the sensor through a second thread. Exemplarily, the electronic device 100 executes through a second thread: applying for a resource cache, creating a first channel, obtaining a second channel of a sensor from the first channel, and writing first register data to the sensor. The second channel is used to power on the sensor.
  • the resource buffer includes a packet buffer.
  • the electronic device 100 executes the resource buffer application, including the electronic device 100 applying to the kernel driver layer for a packet buffer corresponding to the sensor.
  • the packet buffer may be an ION buffer.
  • the packet buffer is used for data transmission.
  • the electronic device 100 creates a first channel (the first channel shown in FIG. 11 ) between the camera application and the camera device by creating a session control handle (session handle) of the CSL service.
  • a session control handle session handle
  • the electronic device 100 executes to obtain the second channel of this sensor from the first channel.
  • the electronic device 100 calls the control interface of this sensor (devices handle 1 as shown in FIG. 11 ) through the first channel to obtain the second channel of this sensor.
  • the process of the electronic device 100 executing writing the first register data to the sensor may include the following steps S1025-S1028:
  • the electronic device 100 obtains first register data from the camera application.
  • the first register data may include initialization register data corresponding to the sensor.
  • the first register data may also include calibration register data corresponding to the sensor.
  • Calibration registers include phase detection correction (PDC) registers, sensitivity phase correction (SPC) registers, and quad sensitivity correction (QSC) registers.
  • the electronic device 100 stores the first encapsulation packet obtained by encapsulating the first register data in the packet buffer, thereby transmitting the first register data from the application layer to the kernel driver layer.
  • the electronic device 100 parses the first encapsulated packet obtained from the packet buffer area to obtain the parsed first register data. For example, the electronic device 100 obtains the first encapsulated packet in the packet buffer area of the kernel driver layer. The electronic device 100 parses the first encapsulated packet in the kernel driver layer to obtain the parsed first register data.
  • the electronic device 100 writes the parsed first register data into the sensor.
  • the electronic device 100 writes the parsed first register data of the kernel driver layer into the sensor via an I2C bus (inter-integrated circuit).
  • step S1025 may be performed before the electronic device 100 executes the resource cache application process, so that the electronic device 100 applies for a packet buffer that matches the first register data size corresponding to the sensor, thereby realizing Reasonable and effective use of the memory resources of the electronic device 100.
  • the electronic device 100 performs sensor initialization on the sensor through the second thread, further comprising: the electronic device 100 configures global state machine parameters of the sensor power-on process, and configures global state machine parameters of the first register data writing sensor state, so as to store the sensor power-on state machine and the first register data writing sensor completion state machine in the global variables of the camera application.
  • the state machine includes a directed graph consisting of states, transitions, and actions.
  • the electronic device 100 also creates a global variable maintenance interface for managing global variables of the camera application.
  • the electronic device 100 maintains the process state machines corresponding to the first channel, the second thread, the second channel, and the second thread in the global variables, so that the electronic device 100 calls the process state machines corresponding to the first channel, the second thread, the second channel, and the second thread when configuring the data flow of the camera application according to step S103, so as to reduce the time consumed by the electronic device 100 to configure the data flow of the camera application.
  • the process state machine includes a directed graph consisting of the process state, process transition, and process action of the process.
  • the process state machine corresponding to the second thread includes a sensor power-on state machine and a first register data writing sensor completion state machine.
  • the global variable maintenance interface also provides a sensor power-on process entry, a thread data synchronization delay mechanism, a sensor initialization process global state machine configuration/acquisition entry, and also provides process exit, data clearing and memory recovery process functions.
  • the electronic device 100 can perform global class initialization on the hardware environment (hw enviroment) of the camera application.
  • the electronic device 100 can perform the following global parameter initialization: initialize all sensor power-on state machine parameters, create sensor pointer addresses, create and manage sensor hardware information resource lists, create an asynchronous thread manager, and use the thread manager class provided by the operating system of the electronic device 100.
  • the global variable maintenance interface such as interface class 1, the sensor hardware information pointer address can be returned.
  • a thread manager can be provided to create and manage asynchronous threads such as the second thread.
  • the maximum number of threads p that the thread manager can maintain can be statically configured or dynamically configured according to m, for example, p is statically configured or dynamically configured according to p ⁇ m.
  • the electronic device 100 can also maintain the asynchronous thread data execution state machine and the CSL handle.
  • calling interface class 3 to maintain the sensor power-on process parameter state machine; calling interface class 4 to maintain the sensor initialization process execution state machine; calling interface class 5 to maintain the CSL handles (handles) of the single-camera mode and the multi-camera mode, and thus the first channel and the second channel of the CSL service of the single-camera mode and the multi-camera mode can be maintained.
  • the electronic device 100 executes step S103, it can conveniently, accurately and efficiently call the process state machines corresponding to the first channel, the second thread, the second channel and the second thread.
  • data stream configuration is performed on the camera application through a third thread; the data stream includes data streams corresponding to various functions of the camera application; the data stream configuration is related to the camera parameters and parameters obtained in sensor initialization, and the third moment is later than the second moment.
  • the electronic device 100 configures the data flow of the camera application through the third thread at the third moment after executing the process corresponding to the first thread.
  • the third moment is later than the second moment, but the third moment is earlier than the time point when the second thread is executed, so that the third thread can be executed in parallel with the second thread.
  • the electronic device 100 may configure the data flow of the camera application through the third thread in the manner shown in S1031-S1034 as follows:
  • the electronic device 100 initializes the first channel and creates threads for each node corresponding to the camera application.
  • the threads for each node are such as a realtime pipeline.
  • the nodes include nodes for initializing sensors corresponding to the camera application.
  • the nodes also include nodes for each instance (usercase) of the camera application, such as nodes for image algorithms related to the instance.
  • the thread of the node is also related to the first channel.
  • the examples may include night scene, portrait, dynamic photo, etc. as shown in FIG5 .
  • the electronic device 100 needs to detect the threads of each node. If the electronic device 100 detects that the global variable contains the thread of the node for initializing the sensor, such as the second thread, indicating that the global variable also contains the first channel, the electronic device 100 can obtain the first channel from the global variable.
  • the electronic device 100 detects again at an interval of 10ms whether the global variable contains a thread for initializing the sensor node.
  • the electronic device 100 starts timing from the first detection of whether the global variable contains a thread for initializing the sensor node. If the electronic device 100 still does not detect that the global variable contains a thread for initializing the sensor node after 3s, the electronic device 100 triggers the process protection mechanism of the camera application to terminate the process currently being executed by the camera application and the subsequent processes to be executed. 3s is a pre-set timeout period. In this way, the parameters of the first channel obtained can be not empty data, so that the second thread can be scheduled by the CPU in time, and the process synchronization of obtaining the first channel and obtaining the second channel can be achieved.
  • the electronic device 100 may also directly detect whether the global variable contains the first channel. If the electronic device 100 detects that the global variable contains the first channel, the first channel is obtained from the global variable.
  • the electronic device 100 After the electronic device 100 obtains the first channel from the global variable, it may execute the step shown in S1033.
  • the electronic device 100 detects whether the global variable contains the first channel again at an interval of 10ms. The electronic device 100 starts timing from the first detection of whether the global variable contains the first channel. If the electronic device 100 still does not detect that the global variable contains the first channel after 3s, the electronic device 100 triggers the process protection mechanism of the camera application to terminate the process currently being executed by the camera application and the subsequent processes to be executed.
  • the electronic device 100 if the electronic device 100 detects that the global variable contains the sensor power-on state machine and the first register data writing sensor completion state machine, the electronic device 100 obtains the sensor power-on state machine and the first register data writing sensor completion state machine from the global variable.
  • the electronic device 100 obtains the sensor power-on state machine and the first register data writing sensor completion state machine from the global variable, and can determine that the sensor corresponding to the camera application has completed power-on, and the sensor corresponding to the camera application has completed sensor initialization, then the electronic device 100 obtains the second channel of each sensor from the global variable to directly execute the subsequent steps, such as the step shown in S1034.
  • the electronic device 100 detects again at an interval of 10ms whether the global variables include the sensor power-on state machine and the first register data writing sensor completion state machine.
  • the electronic device 100 triggers the process protection mechanism of the camera application to terminate the process currently being executed by the camera application and the subsequent processes to be executed. 5 seconds can be the preset maximum waiting time.
  • the electronic device 100 skips the process of writing the first register data into the sensor, and writes the first register data into the sensor.
  • the second channel writes the second register data to each sensor corresponding to the camera application.
  • the second register data includes the allocation register data selected according to the data stream size. For example, the electronic device 100 writes the second register data to each sensor corresponding to the camera application by calling the third function (such as the load sensor config cmds function).
  • the electronic device 100 may flush the data of the data output thread (pipeline) corresponding to the kth data output mode of the target sensor, and reconfigure the data of the data output thread (pipeline) corresponding to the vth data output mode. For example, the electronic device 100 may rewrite the second register data to the target sensor to configure the data of the data output thread (pipeline) corresponding to the vth data output mode, and the target sensor will not be powered off.
  • step S1034 detects that the global variable includes a sensor power-on state machine and a first register data writing sensor completion state machine, the electronic device 100 no longer needs to initialize each sensor.
  • the electronic device 100 directly skips the process of writing the first register data to the sensor, and writes the second register data into each sensor corresponding to the camera application through the second channel of each sensor of the camera application, thereby starting the flow of each sensor.
  • step S1034 does not need to write the first register data to the sensor. If the execution of the threads of other nodes corresponding to the camera application created by step S1031 is parallel to step S1034, the execution of the threads of other nodes will obtain more abundant memory and CPU resources, which further enables step S103 to be completed quickly.
  • each sensor can collect scene images and transmit the collected scene images to the HAL layer of the electronic device 100.
  • the electronic device 100 calls the image algorithm in the camera algorithm library of the HAL layer, and after the HAL layer processes the scene image output by the sensor corresponding to the camera application, the processed image is written to the rotating buffer.
  • the camera application displays the image preview in the rotating buffer to the interface of the camera application.
  • the electronic device 100 can call the global variable maintenance interface to cancel the second thread in the global variable to release more memory.
  • the electronic device 100 fails to create the second thread when executing step S102, the electronic device 100 terminates the execution of step S102, and the electronic device 100 can continue to execute the steps shown in S103 to implement sensor initialization of the sensor corresponding to the camera application and data stream configuration of the camera application. In this way, the interface of the camera application can still display the scene image collected and output by the sensor.
  • the second thread also includes a thread switch.
  • the electronic device 100 may also turn off the thread switch when the total amount of available memory and CPU resources of the electronic device 100 is greater than a resource threshold. As shown in Figure 9, if the electronic device 100 detects that the thread switch is on, step 102 may be executed. Correspondingly, if the electronic device 100 detects that the thread switch is off, the electronic device 100 may skip step S102 and execute step S103 after executing step S101.
  • the electronic device 100 detects that the thread switch identifier is an on identifier, it indicates that the electronic device 100 detects that the thread switch is on; accordingly, if the electronic device 100 detects that the thread switch identifier is a off identifier, it indicates that the electronic device 100 detects that the thread switch is off.
  • the on identifier is such as identifier 1
  • the off identifier is such as identifier 0.
  • the electronic device 100 may also respond to the instruction to close the camera application, detect the logical identifier of the camera application, and obtain the sensor-related pointer. If the logical identifier is an identifier of the first mode, the electronic device 100 releases the first channel based on the obtained sensor-related pointer, initializes the sensor power-on state machine, releases the second channel of the sensor, and cancels the second thread; if the logical identifier is an identifier of the second mode, the electronic device 100 releases the first channel based on the obtained sensor-related pointer, initializes the sensor power-on state machine, releases the second channel of the sensor, and cancels the second thread; If the electronic device 100 receives a signal of the type, it initializes the sensor power-on state machine, releases the second channel of the sensor and cancels the second thread.
  • the adaptability of the camera application to other applications can be improved, so that the second thread can be normally created and executed after other applications trigger the camera application, and further, after the camera application is triggered by other applications, the initialization of the sensor corresponding to the camera application and the data stream configuration of the camera application can be quickly completed.
  • the electronic device 100 responds to the instruction to switch out of the camera application, destroys the threads and the first channel of each node corresponding to the camera application, and destroys the second channel. If the first channel and the second channel are the first channel and the second channel of the CSL service, after the camera application exits, all the first channels and the second channels of the CSL service will be released, so that after the camera application exits, the electronic device 100 has more available CPU resources and more available input/output (I/O) resources.
  • I/O input/output
  • the camera application configuration method provided in the embodiment of the present application enables the sensor initialization of the sensor corresponding to the camera application to be completed in advance, accelerates the data stream configuration of the camera application, and can reduce the time from the camera application being triggered to the camera application interface displaying the scene image. If the camera application on the user's electronic device adopts the camera application configuration method provided in the embodiment of the present application, after the user triggers the camera application, the interface of the camera application will promptly display the target scene image that the user wants to capture, allowing the user to complete the capture of the target scene image, thereby improving the user's experience of using the camera application. In addition, if the electronic device adopts the camera application configuration method provided in the embodiment of the present application, after the camera application is triggered, even if the second thread cannot be created, the interface of the camera application can still preview and display the scene image output after the sensor is collected.
  • An embodiment of the present application provides an electronic device, which terminal device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.
  • FIG12 is a schematic diagram of the hardware structure of the chip provided in the embodiment of the present application.
  • the chip includes one or more (including two) processors 81, a communication line 82, a communication interface 83 and a memory 84.
  • the processor 81 is used to call the computer program in the memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above-mentioned related embodiments, and will not be repeated here.
  • the embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the above method is implemented when the computer program is executed by the processor.
  • the method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium.
  • Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another.
  • the storage medium can be any target medium that can be accessed by a computer.
  • a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer.
  • any connection is appropriately referred to as a computer-readable medium.
  • a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave
  • DSL digital subscriber line
  • wireless technology such as infrared, radio and microwave
  • Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.
  • An embodiment of the present application provides a computer program product, which includes a computer program.
  • the computer program When the computer program is executed, the computer executes the above method.
  • each process and/or box in the flowchart and/or block diagram and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions.
  • These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Studio Devices (AREA)

Abstract

本申请实施例提供了一种相机应用配置方法、设备及存储介质，涉及终端技术领域。其中方法应用于电子设备，方法包括：第一时刻，通过第一线程获取相机参数；第二时刻，通过第二线程对传感器进行传感器初始化；所述传感器包括相机应用对应的传感器；所述传感器初始化与所述相机参数有关；第三时刻，通过第三线程对所述相机应用进行数据流配置；所述数据流包括所述相机应用的各功能对应的数据流；所述数据流配置与所述相机参数以及所述传感器初始化中得到的参数有关，所述第三时刻晚于所述第二时刻。这样，减少了相机应用被触发到相机应用的界面显示出图像的耗时。

Description

相机应用配置方法、设备及存储介质
本申请要求于2023年02月27日提交中国国家知识产权局、申请号为202310227589.8、申请名称为“相机应用配置方法、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及一种相机应用配置方法、设备及存储介质。
背景技术
随着终端技术的发展，终端设备通常会提供相机应用(application，APP)，相机应用支持拍照、录像功能。
在可能的实现方式中，当用户想要抓拍某些场景图像时，用户会触发相机应用后，对目标场景图像进行拍照或录像。然而，用户触发相机应用后，相机应用的界面(user interface，UI)需耗费较长时间才能显示出图像，导致用户无法抓拍到目标场景图像。
发明内容
本申请实施例提供一种相机应用配置方法、设备及存储介质，应用于终端技术领域，减少了相机应用被触发到相机应用的界面显示出图像的耗时。
第一方面，本申请实施例提出一种相机应用配置方法。该方法包括：
第一时刻，通过第一线程获取相机参数；
第二时刻，通过第二线程对传感器进行传感器初始化；传感器包括相机应用对应的传感器；传感器初始化与相机参数有关；
第三时刻，通过第三线程对相机应用进行数据流配置；数据流包括相机应用的各功能对应的数据流；数据流配置与相机参数以及传感器初始化中得到的参数有关，第三时刻晚于第二时刻。
这样，相机应用对应传感器的传感器初始化开始时间早于对相机应用进行数据流配置的开始时间，使相机应用对应传感器的传感器初始化提前完成，还可以有效利用第三线程开始执行之前的电子设备的可用内存和电子设备的可用CPU资源，使得传感器初始化能快速完成，减弱传感器初始化对第三线程对应的其他流程所需内存和CPU资源的竞争力度，使得第三线程对应其他流程在执行过程中获取到充裕的内存和CPU资源，减少了传感器初始化的耗时和相机应用的数据流配置的耗时，进而减少从相机应用被触发，到相机应用的界面显示出图像的耗时。
在一种可能的实现方式中，通过第二线程对传感器进行传感器初始化之前，还包括：
检测相机应用的逻辑标识，逻辑标识包括第一模式的标识或第二模式的标识；第一模式对应m个传感器，第二模式对应一个传感器，m为大于0的整数；
若逻辑标识为第一模式的标识，则创建相机应用对应的第一通道，并创建m个第二线 程；
通过第二线程对传感器进行传感器初始化，包括：通过m个第二线程分别对m个传感器进行传感器初始化；其中，第一通道用于管理相机应用与m个传感器之间的接口。
这样，无论相机应用对应一个传感器还是多个传感器，均可以对各传感器进行传感器初始化。若相机应用对应多个传感器，通过m个第二线程并发地对m个传感器进行传感器初始化，可以减少相机应用对应的传感器初始化的总耗时。
在一种可能的实现方式中，通过m个第二线程分别对m个传感器进行传感器初始化，包括：
对于m个第二线程的任一个第二线程，通过任一个第二线程执行：申请资源缓存，从第一通道中得到m个传感器中任一个传感器的第二通道，以及向任一个传感器写入第一寄存器数据；第二通道用于为任一个传感器上电。
这样，通过所申请的缓存资源，可以实现将相机应用下发的各传感器的第一寄存器数据传输到内核驱动层，通过各传感器的第二通道，为各传感器上电，以便内核驱动层将第一寄存器数据写入各传感器，实现对各传感器的传感器初始化。
在一种可能的实现方式中，资源缓存包括包缓存区；
向任一个传感器写入第一寄存器数据，包括：
从相机应用获取第一寄存器数据；
将第一寄存器数据封装后得到的第一封装包存储于包缓冲区；
解析从包缓存区获取的第一封装包，得到解析后的第一寄存器数据；
将解析后的第一寄存器数据写入任一个传感器。
这样，实现第一寄存器数据从应用层传输到内核驱动层，以便通过内核驱动层将第一寄存器数据写入各传感器，实现对各传感器的传感器初始化。
在一种可能的实现方式中，若逻辑标识为第二模式的标识，则创建一个第二线程；
通过第二线程对传感器进行传感器初始化，包括：通过一个第二线程对一个传感器进行传感器初始化。
这样，在相机应用对应一个传感器的情况下，不用在创建第二线程之前创建相机应用对应的第一通道，直接创建一个第二线程并通过一个第二线程对相近应用对应的这一个传感器进行传感器初始化，可以节省传感器初始化耗时，也便于对传感器初始化的线程进行管理。
在一种可能的实现方式中，通过一个第二线程对一个传感器进行传感器初始化，包括：
通过一个第二线程执行：申请资源缓存，创建第一通道，从第一通道中得到一个传感器的第二通道，以及向一个传感器写入第一寄存器数据；第二通道用于为一个传感器上电。
这样，通过所申请的缓存资源，可以实现将相机应用下发的这一个传感器的第一寄存器数据传输到内核驱动层，通过这一个传感器的第二通道，为这一个传感器上电，以便内核驱动层将第一寄存器数据写入这一个传感器，实现对这一个传感器的传感器初始化，将创建第一通道的流程放在第二线程中，节省传感器初始化耗时，也便于对传感器初始化的线程进行管理。
在一种可能的实现方式中，资源缓存包括包缓存区；
向一个传感器写入第一寄存器数据，包括：
从相机应用获取第一寄存器数据；
将第一寄存器数据封装后得到的第一封装包存储于包缓冲区；
解析从包缓存区获取的第一封装包，得到解析后的第一寄存器数据；
将解析后的第一寄存器数据写入一个传感器。
这样，实现第一寄存器数据从应用层传输到内核驱动层，以便通过内核驱动层将第一寄存器数据写入这一个传感器，实现对这一个传感器的传感器初始化。
在一种可能的实现方式中，第二线程包括对应的同步锁；同步锁用于对第二线程对应的执行对象进行锁定。
这样，使得并行执行的多个第二线程不会同时抢用同一个执行对象，使得每个第二线程对应的执行结果均是精准的，获得的每个第二线程的执行结果数据均是洁净的。
在一种可能的实现方式中，方法还包括：
创建用于管理相机应用的全局变量的全局变量维护接口；
将第一通道、第二线程、第二通道和第二线程对应的流程状态机维护在全局变量中；流程状态机包括流程的流程状态、流程转换、流程动作构成的有向图，第二线程对应的流程状态机包括传感器上电状态机和第一寄存器数据写入传感器完成状态机。
这样，通过调用全局变量维护接口，在对相机应用进行数据流配置时，可以便捷、精准、高效地对第一通道、第二线程、第二通道和第二线程对应的流程状态机进行调用。
在一种可能的实现方式中，通过第三线程对相机应用进行数据流配置，包括：
初始化第一通道，创建相机应用对应的各节点的线程；节点包括用于初始化传感器的节点；
若检测到全局变量中包含用于初始化传感器的节点的线程，则从全局变量中获取第一通道；
若检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，则从全局变量中获取各传感器的第二通道；
跳过将第一寄存器数据写入传感器的流程，通过各传感器的第二通道将第二寄存器数据分别写入各传感器；第二寄存器数据包括根据数据流大小选择的配流寄存器数据。
这样，节省了向传感器写入第一寄存器数据的时间，此外，由于无需向传感器写入第一寄存器数据，相机应用对应的其他节点的线程的执行若与写入第二寄存器数据的流程并行，则其他节点的线程的执行将获得更充裕的内存和CPU资源，也进一步使得相机应用的数据流配置能快速完成。
在一种可能的实现方式中，在通过各传感器的第二通道将第二寄存器数据分别写入各传感器之后，还包括：
注销全局变量中的第二线程。
这样，可以释放出更多内存。
在一种可能的实现方式中，在通过第三线程对相机应用进行数据流配置之后，还包括：
响应关闭相机应用的指令，检测相机应用的逻辑标识；逻辑标识包括第一模式的标识或第二模式的标识，第一模式对应m个传感器，第二模式对应一个传感器，m为大于0的整数；
若逻辑标识为第一模式的标识，则释放第一通道，初始化传感器上电状态机，释放传 感器的第二通道并注销第二线程；
若逻辑标识为第二模式的标识，则初始化传感器上电状态机，释放传感器的第二通道并注销第二线程。
这样，可以提高相机应用与其他应用的适配性能，使得其他应用触发相机应用后第二线程能被正常创建和执行，进而使得相机应用被其他应用触发后，能快速完成相机应用对应传感器的初始化和相机应用的数据流配置。
在一种可能的实现方式中，在通过第三线程对相机应用进行数据流配置之后，还包括：
响应切换退出相机应用的指令，摧毁相机应用对应的各节点的线程和第一通道；并摧毁第二通道。
这样，若第一通道和第二通道是CSL服务的第一通道和第二通道，这样，相机应用退出后，所有的CSL服务的第一通道和第二通道均会被释放，以使得相机应用退出后，电子设备100拥有更多可用的CPU资源以及更多可用的输入/输出(input/output，I/O)资源。
第二方面，本申请实施例提供一种电子设备，该电子设备包括：处理器和存储器；存储器存储计算机执行指令；处理器执行存储器存储的计算机执行指令，使得电子设备执行如第一方面的方法。
第三方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序。计算机程序被处理器执行时实现如第一方面的方法。
第四方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括计算机程序，当计算机程序被运行时，使得计算机执行如第一方面的方法。
第五方面，本申请实施例提供了一种芯片，芯片包括处理器，处理器用于调用存储器中的计算机程序，以执行如第一方面的方法。
应当理解的是，本申请的第二方面至第五方面与本申请的第一方面的技术方案相对应，各方面及对应的可行实施方式所取得的有益效果相似，不再赘述。
附图说明
图1为本申请实施例提供的电子设备100的结构示意图；
图2为本申请实施例提供的电子设备100的软件结构示意图；
图3为可能的实现方式中用户触发相机应用以拍照的示意图；
图4为可能的实现方式中相机应用被触发后图像显示的流程示意图；
图5为本申请实施例提供的用户触发相机应用以拍照的示意图；
图6为本申请实施例提供的相机应用被触发后图像显示的流程示意图；
图7为本申请实施例提供的传感器初始化节点提前的相机应用对应架构图；
图8为本申请实施例提供的相机应用配置方法流程图一；
图9为本申请实施例提供的相机应用配置方法流程图二；
图10为本申请实施例提供的多摄模式示意图；
图11为本申请实施例提供的单摄模式示意图；
图12为本申请实施例提供的芯片的硬件结构示意图。
具体实施方式
为了便于清楚描述本申请实施例的技术方案，以下，对本申请实施例中所涉及的部分术语和技术进行简单介绍：
1、sensor：可以称为感光元件、图像传感器或传感器；用于将光信号转换为电信号。
2、configure_streams：可以称为数据流配置或配流；例如对相机应用相关的功能对应数据流的配置。
3、session：可以称为会话控制；用于展示相机设备的控制接口。
4、buffer：可以称为缓冲区或缓存。
5、package buffer：可以称为包缓冲区；用于在数据传输过程中存储或中转待传输的数据，待传输的数据需封装成封装包后存入包缓冲区。
6、camera id：可以称为传感器标识。例如，一个传感器对应一个传感器标识。
7、user case：可以称为实例；用于创建相机数据流处理节点的配置参数；user case支持相机各种模式的创建及算法配置，模式例如单摄模式、多摄模式等。
8、handle：可以称为句柄，用于相机设备驱动通道的创建，也用于传感器驱动通道的创建或获取。
9、session handle：可以称为会话控制句柄或相机设备控制接口，用于相机设备驱动通道的创建。
10、devices handle：可以称为传感器句柄或传感器控制接口，用于传感器(sensor)驱动通道的创建或获取。
11、open：可以称为启动。
13、pipeline：可以称为管道、流水线或多线程，包含相机应用对应的各节点的线程，如传感器初始化的线程、图像算法的线程等。
14、ION：是安卓(android)用于内存管理的一个子系统。ION的意思是zero-copy。ION表征在系统(如android系统)的用户空间和内核空间传递数据的内存，该内存可以在用户空间和内核空间直接调用获取数据。
15、其他术语
本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
本申请实施例中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b，或c中的至少一项(个)，可以表示：a，b，c，a-b，a-c，b-c，或a-b-c，其中a，b，c可以是单个，也可以是多个。
需要说明的是，本申请实施例中的“在……时”，可以为在某种情况发生的瞬时，也可以为在某种情况发生后的一段时间内，本申请实施例对此不作具体限定。此外，本申请实施例提供的显示界面仅作为示例，显示界面还可以包括更多或更少的内容。
16、电子设备
本申请实施例的电子设备可以包括具有图像处理功能的手持式设备、车载设备等。例如，一些电子设备为：手机(mobile phone)、平板电脑、掌上电脑、笔记本电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端、蜂窝电话、无绳电话、会话启动协议(session initiation protocol，SIP)电话、无线本地环路(wireless local loop，WLL)站、个人数字助理(personal digital assistant，PDA)、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、车载设备(如车机)、可穿戴设备，5G网络中的终端设备或者未来演进的公用陆地移动通信网络(public land mobile network，PLMN)中的终端设备等，本申请实施例对此并不限定。
作为示例而非限定，在本申请实施例中，该电子设备还可以是可穿戴设备。可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如助听器、眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。
此外，在本申请实施例中，电子设备还可以是物联网(internet of things，IoT)系统中的终端设备，IoT是未来信息技术发展的重要组成部分，其主要技术特点是将物品通过通信技术与网络连接，从而实现人机互连，物物互连的智能化网络。
本申请实施例中的电子设备也可以称为：终端设备、用户设备(user equipment，UE)、移动台(mobile station，MS)、移动终端(mobile terminal，MT)、接入终端、用户单元、用户站、移动站、移动台、远方站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置等。
在本申请实施例中，电子设备或各个网络设备包括硬件层、运行在硬件层之上的操作系统层，以及运行在操作系统层上的应用层。该硬件层包括中央处理器(central processing unit，CPU)、内存管理单元(memory management unit，MMU)和内存(也称为主存)等硬件。该操作系统可以是任意一种或多种通过进程(process)实现业务处理的计算机操作系统，例如，Linux操作系统、Unix操作系统、Android操作系统、iOS操作系统或windows操作系统等。该应用层包含相机、浏览器、通讯录、文字处理软件、即时通信软件等应用。
为了能够更好地理解本申请实施例，下面对本申请实施例的电子设备的结构进行介绍：
图1示出了电子设备100的结构示意图。
电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受 话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音 频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。I2S接口和PCM接口都可以用于音频通信。
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液 晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据、图像数据、电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例 中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测电子设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消电子设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。
距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本发明实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。
图2是本发明实施例的电子设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为五层，从上至下分别为应用层，应用框架层，硬件抽象层(hardware abstraction layer，HAL)，内核驱动层以及硬件层。
应用层可以包括一系列应用程序包。
应用程序包可以包括如图2所示的相机，图库，日历，电话，地图，导航，WLAN，蓝牙，音乐，视频，信息等应用程序。
应用框架层为应用层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图2所示，应用框架层可以包括相机访问接口，窗口管理器。
相机访问接口用于建立相机应用分别与相机管理和相机设备的通信通道。相机访问接口可以包括Java本地接口(java native interface，JNI接口)。通过使用Java接口书写程序，可以使代码在不同的平台上方便移植。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
硬件抽象层可以包含相机硬件抽象层、相机算法库和相机服务层(camera service level，CSL)。相机硬件抽象层包括至少一个相机设备。相机算法库中包含与相机硬件抽象层的相机设备对应的图像算法。相机设备可以包括传感器、图像信号处理器、数字信号处理器、图像处理器。相机服务层用于硬件抽象层与内核驱动层之间的数据交互。相机服务层为硬件抽象层与内核驱动层之间的交互提供接口。
内核驱动层是硬件和软件之间的层。内核驱动层至少包含相机设备驱动，数字信号处理器驱动，图像处理器驱动。内核驱动层可以采用视频设备的内核驱动框架(Video for linux2，V4L2)，以便于对诸如传感器之类的相机设备进行读写。
硬件层包括至少一个传感器、图像信号处理器、数字信号处理器、图像处理器。
下面结合用户触发相机应用以拍照的示例，示例性说明电子设备100软件以及硬件的工作流程。图3为可能的实现方式中用户触发相机应用以拍照的示意图。图4为可能的实现方式中相机应用被触发后图像显示的流程示意图。用户可以通过对电子设备100上相机应用的控件进行触摸单击操作来触发相机应用，用户也可以通过将电子设备100的当前应用切换到相机应用的操作来触发相机应用。
在可能的实现方式中，当如图3所示的电子设备100的触摸传感器接收到用户的触摸单击操作，相应的硬件中断被发给内核驱动层。内核驱动层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核驱动层。应用框架层从内核驱动层获取原始输入事件，识别出该输入事件所对应的控件为相机应用图标的控件。如图4所示，电子设备100的相机应用(APP)调用应用框架层的相机访问接口， 通过HAL层执行第一线程，获取相机参数和第一资源。第一资源包括创建的HAL执行符。第一线程如相机应用的启动线程(open线程)。相机应用获取到相机参数后，调用HAL层的配流线程对相机应用进行数据流配置。配流线程执行(也称为配流)：创建相机应用对应的第一通道(如session handle)，创建相机应用的各节点。第一通道也称为相机设备驱动通道。第一通道用于管理相机应用与至少一个相机设备之间的接口。相机用例包括多摄模式或单摄模式。创建相机应用的各节点包括创建各节点的线程(pipeline)。节点例如传感器初始化节点、图像算法配置节点、传感器启流节点等。传感器初始化包括通过调用内核驱动层给相机应用对应的传感器上电，向传感器写入初始化寄存器数据、校准寄存器数据。传感器启流包括通过调用内核驱动层给相机应用对应的传感器写入配流寄存器数据后，启动相机设备驱动，通过传感器采集场景图像、输出场景图像。相机应用的数据流配置完成后，电子设备100的相机应用向相机硬件抽象层下发预览请求和轮转缓冲区(轮转buffer)。下发预览请求和轮转缓冲区(轮转buffer)称为repeating。电子设备100的相机硬件抽象层调用相机算法库中的图像算法，对传感器输出的场景图像进行处理后，将处理后的图像写入轮转buffer。相机应用将轮转buffer中的图像预览显示到相机应用的界面(user interface，UI)。用户即可对场景图像进行拍照或录像操作。拍照或录像操作表征用于对界面显示的图像进行保存。
然而，由于配流线程对应的流程较多，且各流程的执行需要较多的I/O资源和CPU资源，若电子设备的内存小，和/或电子设备的CPU资源被其他后台应用大量占用，则相机应用的数据流配置需要耗费较多时间。因此，当用户进行触发相机应用的操作后，相机应用的界面需要耗费较多时间才能显示出场景图像，将导致用户无法抓拍到想要抓拍的目标场景图像。
为了提高相机应用在不同电子设备上的通用性，一些可能的相机应用系统架构采用如图4所示的执行第一线程--配流--repeating--(预览)显示图像的串行执行方式实现相机应用触发后的图像显示。一旦串行执行的多个线程中任一线程耗时延长，则相机应用被触发后图像显示的耗时延长。由于配流线程对应的流程中，传感器初始化的节点创建晚于配流线程执行开始时间，且初始化寄存器数据和校准寄存器数据的数据量大，使得传感器初始化的节点创建耗时较长，进而使得配流线程的执行耗时较长。此外，传感器初始化的节点创建与配流线程对应的流程中其他节点创建存在内存和CPU资源竞争关系，在电子设备的内存小和/或电子设备的CPU资源被其他后台应用大量占用的情况下，配流线程的执行耗时会延长。
有鉴于此，本申请实施例提出一种相机应用配置方法，该相机应用配置方法应用于电子设备，通过将传感器初始化节点创建提前，使传感器初始化节点创建开始时间早于配流线程执行开始时间，一方面实现传感器初始化的节点创建提前完成，另一方面可以有效利用配流线程开始执行之前的电子设备内存和电子设备CPU资源，使得传感器初始化节点创建能快速完成，减弱传感器初始化节点创建对配流线程对应的其他节点所需内存和CPU资源的竞争力度，使得配流线程对应其他流程在执行过程中获取到充裕的内存和CPU资源，减少了传感器初始化节点创建和配流线程执行的耗时，使得相机应用被触发后，相机应用的界面快速显示出图像。
下面结合部分实施例对本申请提供的相机应用配置方法进行说明。
图5为本申请实施例提供的用户触发相机应用以拍照的示意图。图6为本申请实施例提供的相机应用被触发后图像显示的流程示意图。如图5和图6所示，当电子设备100的触摸传感器接收到用户的触摸单击操作，相应的硬件中断被发给内核驱动层。内核驱动层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核驱动层。应用框架层从内核驱动层获取原始输入事件，识别出该输入事件所对应的控件为相机应用图标的控件。第一时刻，电子设备100的相机应用(APP)调用应用框架层的相机访问接口，通过HAL层执行第一线程获取相机参数和第一资源。第一资源包括所创建的HAL执行符。第一线程如相机应用的启动线程(open线程)。相机应用获取到相机参数后的第二时刻，电子设备100的相机应用通过第二线程对相机应用对应的传感器进行传感器初始化。传感器初始化与相机参数有关。第一线程对应流程执行完成之后的第三时刻，相机应用通过第三线程对相机应用进行数据流配置。数据流配置也称为配流。相机应用通过第三线程对相机应用进行数据流配置包括初始化相机设备驱动通道(session handle)、创建诸如图像算法配置节点之类的相机应用对应的其他节点、启流。数据流包括相机应用的各功能对应的数据流。数据流配置与相机参数以及传感器初始化中得到的参数有关，第三时刻晚于第二时刻。第二线程的执行开始时间早于第三线程的执行开始时间，使得相机应用对应传感器的传感器初始化可以提前完成。通过第三线程对相机应用进行数据流配置时，节省了执行传感器初始化流程的时间和资源，减少了对相机应用的数据流配置耗时。相机应用的数据流配置完成后，电子设备100的相机应用向相机硬件抽象层下发预览请求和轮转缓冲区(轮转buffer)。下发预览请求和轮转缓冲区(轮转buffer)也称为repeating。电子设备100的相机硬件抽象层调用相机算法库中的图像算法，对传感器输出的场景图像进行处理后，将处理后的图像写入轮转buffer。电子设备100的相机应用将轮转buffer中的图像预览显示到相机应用的界面(user interface，UI)。用户即可对场景图像进行拍照或录像操作。拍照或录像操作表征用于对界面显示的图像进行保存。
示例性地，对于从相机应用被触发到相机应用的界面显示出场景图像的耗时而言，图3所示可能的实现方式中相机应用对应的耗时为(T+730)毫秒，而图5所示本申请实施例提供的相机应用对应的耗时为(T+600)毫秒。假设图5所示相机应用的界面显示的场景图像为用户待抓拍的目标场景图像。若用户的电子设备100上的相机应用为图3所示的相机应用，则用户将无法抓拍到图5所示的目标场景图像。
在本申请实施例中，用户将电子设备100的当前应用切换到相机应用后，相机应用的界面显示出场景图像的实现原理和技术效果，与本申请实施例中示出的用户对相机应用进行触摸单击操作后相机应用的界面显示出场景图像的实现原理和技术效果类似，本实施例此处不再赘述。
本申请实施例提供的相机应用配置方法使相机应用对应传感器的传感器初始化提前完成，加速了对相机应用的数据流配置，可以减少从相机应用被触发，到相机应用的界面显示出场景图像的耗时。若用户的电子设备上的相机应用采用本申请实施例提供的相机应用配置方法，用户触发相机应用后，相机应用的界面将及时显示出用户想抓拍的目标场景图像，使得用户完成对目标场景图像的抓拍。
图7为本申请实施例提供的传感器初始化节点提前的相机应用对应架构图。如图9所示，该架构包括相机应用(APP)、应用架构(framework)和内核驱动层(kernel sensor driver)。 其中，应用架构包括：相机应用API、硬件环境区、用户端(sensor customer)和传感器节点。通过相机应用API可以触发相机应用和关闭相机应用。硬件环境区部分可以初始化全局参数、初始化线程管理器、提供传感器上电流程入口、提供线程数据同步延迟(delay)机制、提供传感器初始化流程全局状态机的配置/获取入口、提供流程退出、清楚数据和回收内存流程。用户端可以创建第二线程，创建第二线程同步锁，创建第一通道，获取第二通道，进行流程开关、逻辑标识等参数配置，释放诸如第二线程的异步线程以及CSL资源。传感器节点可以采用第一通道和第二通道，跳过第一寄存器数据写入流程，进行第一通道和第二通道相关资源释放。内核驱动层可以为传感器上电或下电，也可以进行第二寄存器数据的写入。
下面结合图8、图9、图10、图11对本申请提供的相机应用配置方法进行详细说明，同时对图7所示的架构图进行详细说明。图8为本申请实施例提供的相机应用配置方法流程图一。图9为本申请实施例提供的相机应用配置方法流程图二。图10为本申请实施例提供的多摄模式示意图。图11为本申请实施例提供的单摄模式示意图。图8所示实施例的执行主体可以是图1所示实施例中的电子设备100。如图8所示，该方法包括：
S101、第一时刻，通过第一线程获取相机参数。
在本申请实施例中，电子设备100响应于触发相机应用的指令，电子设备100在第一时刻，调用第一线程，通过第一线程获取相机参数和创建HAL执行符。第一线程如open线程。第一线程的执行由HAL层提供服务。相机参数中包括传感器标识。
S102、第二时刻，通过第二线程对传感器进行传感器初始化。传感器包括相机应用对应的传感器。传感器初始化与相机参数有关。
在本申请实施例中，电子设备100在获取到相机参数后的第二时刻，通过第二线程对相机应用对应的传感器进行传感器初始化。
示例性地，电子设备100通过第二线程对相机应用对应的传感器进行传感器初始化之前，电子设备100检测相机应用的逻辑标识(logical id)。逻辑标识包括第一模式的标识或第二模式的标识。第一模式对应m个传感器，第二模式对应一个传感器，m为大于0的整数。第一模式例如图10所示的多摄模式。第二模式例如图11所示的单摄模式。逻辑标识可以用于确定相机应用的诸如单摄模式或多摄模式这样的实例(user case)。
可选地，电子设备100若检测出相机应用的逻辑标识为第一模式的标识，则创建相机应用对应的第一通道，并创建m个第二线程。第一通道可用于管理相机应用与m个传感器之间的接口。电子设备100通过m个第二线程分别对m个传感器进行传感器初始化。对于m个第二线程的任一个第二线程，通过任一个第二线程执行：申请资源缓存，从第一通道中得到m个传感器中任一个传感器的第二通道，以及向任一个传感器写入第一寄存器数据。第二通道用于为任一个传感器上电。
示例性地，电子设备100采用CSL服务的会话控制句柄(session handle)的方式，创建相机应用与相机设备之间的第一通道。例如，电子设备100通过CSL open函数创建包含传感器(sensor)在内的相机设备的驱动节点的会话控制句柄(session handle)，得到第一通道，以便通过第一通道得到驱动传感器的第二通道。电子设备100通过第一通道实现相机应用与硬件抽象层的通信。假设m＝2，相机应用与m个传感器之间的接口可以如图10所示的devices handle 1和devices handle 2。可选地，所创建的m各第二线程，可以分 别与m个传感器的传感器标识(camera id)对应。
可选地，电子设备100通过m个第二线程分别对m个传感器进行传感器初始化可以包括：电子设备100通过第一函数(如register job family函数)注册回调函数(如sensor advance power thread job callback函数)到第二线程，配置线程优先级和参数传递buffer，通过启动函数(如post job函数)启动第二线程。
可选地，电子设备100创建每一个第二线程的同时，创建对应的同步锁。同步锁用于对第二线程对应的执行对象进行锁定，以得到精确的执行结果或洁净的执行结果数据。示例性地，假设相机应用对应如图10所示的2个传感器，电子设备100创建了2个第二线程，分别为第二线程1和第二线程2。电子设备100通过第二线程1和第二线程2分别对如图10所示的传感器1和传感器2进行传感器初始化。若第二线程1和第二线程2在执行各自对应的流程h时，执行对象均是H，且第二线程1对应的流程h先于第二线程2对应的流程h，则电子设备100在第二线程1对应的流程h时，采用第二线程1的同步锁将对象H锁定。电子设备100执行完第二线程1对应的流程h后对对象H进行解锁，以便电子设备100在第二线程2对应的流程h时锁定对象H。这样，使得并行执行的多个第二线程不会同时抢用同一个执行对象，使得每个第二线程对应的执行结果均是精准的，获得的每个第二线程的执行结果数据均是洁净的。
电子设备100执行从第一通道中得到m个传感器中任一个传感器的第二通道，例如：电子设备100通过第一通道(或session handle)，调取m个传感器中任一个传感器的控制接口(devices handle)，得到m个传感器中任一个传感器的第二通道。任一个传感器的控制接口如图10所示的devices handle 1或devices handle 2。其中，调取m个传感器中任一个传感器的控制接口(devices handle)可以是电子设备100通过获取函数(如CSL acquire device函数)创建或调取的。
示例性地，资源缓存包括包缓存区。电子设备100执行申请资源缓存，包括电子设备100通过相机服务层(CSL)向内核驱动层申请包缓存区(packet buffer)。packet buffer可以是ION buffer。packet buffer用于数据传输。ION buffer可以为硬件抽象层和内核驱动层提供一个公共的内存地址。内核驱动层可以直接使用ION buffer内的数据进行后续流程的执行，而无需再互相拷贝，这样可以提高数据传递的效率。
示例性地，电子设备100执行向任一个传感器写入第一寄存器数据的流程可以包括如下步骤S1021-S1024：
S1021、电子设备100从相机应用获取第一寄存器数据。
示例性地，第一寄存器数据可以包含传感器对应的初始化寄存器数据。第一寄存器数据还可以包含传感器对应的校准寄存器数据。校准寄存器例如PDC类寄存器，SPC类寄存器，QSC类寄存器。
S1022、电子设备100将第一寄存器数据封装后得到的第一封装包存储于包缓冲区，实现将第一寄存器数据从应用层传输到内核驱动层。
S1023、电子设备100解析从包缓存区获取的第一封装包，得到解析后的第一寄存器数据。例如，电子设备100在内核驱动层的包缓存区获取第一封装包。电子设备100在内核驱动层对第一封装包进行解析，得到解析后的第一寄存器数据。
S1024、电子设备100将解析后的第一寄存器数据写入任一个传感器。例如，电子设 备100通过I2C总线(inter-integrated circuit)将内核驱动层解析后的第一寄存器数据写入任一个传感器。
示例性地，若第一寄存器数据包含初始化寄存器数据和校准寄存器数据，电子设备100可以通过调用第二函数(如load sensor init config cmd函数)，将解析后的初始化寄存器数据写入传感器，通过调用第三函数(如load sensor config cmds函数)，将解析后的校准寄存器数据写入传感器。
可选地，步骤S1021可以在电子设备100执行申请资源缓存流程之前进行，以便电子设备100申请与第一寄存器数据量大小相匹配的包缓冲区，实现对电子设备100的内存资源的合理有效利用。
可选地，电子设备100若检测出相机应用的逻辑标识为第二模式的标识，则创建一个第二线程。由于逻辑标识为第二模式的标识的相机应用对应一个传感器，所以，电子设备100通过一个第二线程对这一个传感器进行传感器初始化。示例性地，电子设备100通过一个第二线程执行：申请资源缓存，创建第一通道，从第一通道中得到一个传感器的第二通道，以及向这一个传感器写入第一寄存器数据。第二通道用于为这一个传感器上电。
示例性地，资源缓存包括包缓存区。电子设备100执行申请资源缓存，包括电子设备100向内核驱动层申请这一传感器对应的包缓存区(packet buffer)。packet buffer可以是ION buffer。packet buffer用于数据传输。
示例性地，电子设备100采用创建CSL服务的会话控制句柄(session handle)的方式，创建相机应用与相机设备之间的第一通道(如图11所示的第一通道)。
电子设备100执行从第一通道中得到这一个传感器的第二通道例如：电子设备100通过第一通道，调取这一个传感器的控制接口(如图11所示的devices handle 1)，得到这一个传感器的第二通道。
示例性地，电子设备100执行向这一个传感器写入第一寄存器数据的流程可以包括如下步骤S1025-S1028：
S1025、电子设备100从相机应用获取第一寄存器数据。
可选地，第一寄存器数据可以包含这一个传感器对应的初始化寄存器数据。第一寄存器数据还可以包含这一个传感器对应的校准寄存器数据。校准寄存器例如相位对焦校准(phase detection correction，PDC)类寄存器，灵敏度相位校准(sensitivity phase correction，SPC)类寄存器，四合一灵敏度校准(quad sensitivity correction，QSC)类寄存器。
S1026、电子设备100将第一寄存器数据封装后得到的第一封装包存储于包缓冲区，实现将第一寄存器数据从应用层传输到内核驱动层。
S1027、电子设备100解析从包缓存区获取的第一封装包，得到解析后的第一寄存器数据。例如，电子设备100在内核驱动层的包缓存区获取第一封装包。电子设备100在内核驱动层对第一封装包进行解析，得到解析后的第一寄存器数据。
S1028、电子设备100将解析后的第一寄存器数据写入这一个传感器。例如，电子设备100通过I2C总线(inter-integrated circuit)将内核驱动层解析后的第一寄存器数据写入这一个传感器。
可选地，步骤S1025可以在电子设备100执行申请资源缓存流程之前进行，以便电子设备100申请与这一个传感器所对应的第一寄存器数据量大小相匹配的包缓冲区，实现对 电子设备100的内存资源的合理有效利用。
可选地，电子设备100通过第二线程对传感器进行传感器初始化，还包括：电子设备100配置传感器上电流程的全局状态机参数，配置第一寄存器数据写入传感器状态的全局状态机参数，以便于将传感器上电状态机和第一寄存器数据写入传感器完成状态机存入相机应用的全局变量中。状态机包括状态、转换、动作构成的有向图。
可选地，电子设备100还创建用于管理相机应用的全局变量的全局变量维护接口。电子设备100将第一通道、第二线程、第二通道和第二线程对应的流程状态机维护在全局变量中，以便电子设备100按步骤S103对相机应用进行数据流配置时对第一通道、第二线程、第二通道和第二线程对应的流程状态机进行调用，以减少电子设备100对相机应用进行数据流配置的耗时。流程状态机包括流程的流程状态、流程转换、流程动作构成的有向图。第二线程对应的流程状态机包括传感器上电状态机和第一寄存器数据写入传感器完成状态机。
示例性地，全局变量维护接口还提供传感器上电流程入口，提供线程数据同步延迟(delay)机制，提供传感器初始化流程全局状态机的配置/获取入口，还提供流程退出，清除数据和回收内存流程的功能。通过调用全局变量维护接口，电子设备100可以对相机应用的硬件环境(hw enviroment)进行全局类初始化，例如电子设备100可以进行如下全局参数初始化：初始化所有传感器上电状态机参数，传感器指针地址创建，传感器硬件信息资源链表创建与管理，创建异步线程管理器，通过电子设备100的操作系统提供的线程管理器类。通过调用全局变量维护接口，如接口类1，可以返回传感器硬件信息指针地址。通过调用全局变量维护接口，如接口类2，可以提供线程管理器，以对类似第二线程这样的异步线程进行创建和管理。可选地，线程管理器可维护的最大线程数量p可以根据m进行静态配置或动态配置，例如按p≥m，对p进行静态配置或动态配置。通过调用全局变量维护接口，电子设备100也可以维护异步线程数据执行状态机以及CSL handle，例如，调用接口类3，维护传感器上电流程参数状态机；调用接口类4，维护传感器初始化流程执行状态机；调用接口类5，维护单摄模式和多摄模式的CSL句柄(handle)，也就可以维护单摄模式和多摄模式的CSL服务的第一通道和第二通道。这样，可以使得电子设备100在执行步骤S103时，可以便捷、精准、高效地对第一通道、第二线程、第二通道和第二线程对应的流程状态机进行调用。
S103、第三时刻，通过第三线程对相机应用进行数据流配置；数据流包括相机应用的各功能对应的数据流；数据流配置与相机参数以及传感器初始化中得到的参数有关，第三时刻晚于第二时刻。
在本申请实施例中，电子设备100在执行完第一线程对应的流程后的第三时刻，通过第三线程对相机应用进行数据流配置。第三时刻晚于第二时刻，而第三时刻早于第二线程执行完成的时间点，使得第三线程可以与第二线程并行执行。
示例性地，电子设备100可以按如下S1031-S1034所示方式，通过第三线程对相机应用进行数据流配置：
S1031、电子设备100初始化第一通道，创建相机应用对应的各节点的线程。各节点的线程如realtime pipeline。节点包括用于初始化相机应用对应的传感器的节点。节点还包括相机应用的各个实例(usercase)的节点，如实例相关的图像算法的节点。图像算法的 节点的线程也与第一通道有关。实例可以包括如图5所示的夜景、人像、动态照片等实例。
S1032、电子设备100若检测到全局变量中包含用于初始化传感器的节点的线程，则从全局变量中获取第一通道。
在本申请实施例中，电子设备100需检测各节点的线程。电子设备100若检测到全局变量中包含用于初始化传感器的节点的线程，如第二线程，表征全局变量中也包含了第一通道，则电子设备100可以从全局变量中获取第一通道。
可选地，电子设备100若没有检测到全局变量中包含用于初始化传感器的节点的线程，则电子设备100间隔10ms再检测一次全局变量中是否包含用于初始化传感器的节点的线程。电子设备100从第一次检测全局变量中是否包含用于初始化传感器的节点的线程开始计时，若3s后电子设备100仍没有检测到全局变量中包含用于初始化传感器的节点的线程，则电子设备100触发相机应用的进程保护机制，以终止相机应用当前正在执行的流程以及后续待执行的流程。3s是预先设置的超时时间。这样，可以使得获取的第一通道的参数不是空数据，使得第二线程被CPU及时调度到，可以实现获取第一通道与获取第二通道的流程同步。
可选地，电子设备100也可以直接检测全局变量中是否包含第一通道，电子设备100若检测到全局变量中包含第一通道，则从全局变量中获取第一通道。
电子设备100从全局变量中获取第一通道后，可以执行S1033所示的步骤。
可选地，电子设备100若没有检测到全局变量中包含第一通道，则电子设备100间隔10ms再检测一次全局变量中是否包含第一通道。电子设备100从第一次检测全局变量中是否包含第一通道开始计时，若3s后电子设备100仍没有检测到全局变量中包含第一通道，则电子设备100触发相机应用的进程保护机制，以终止相机应用当前正在执行的流程以及后续待执行的流程。
S1033、电子设备100若检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，则从全局变量中获取各传感器的第二通道。
在本申请实施例中，电子设备100若检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，电子设备100从全局变量中获取到传感器上电状态机和第一寄存器数据写入传感器完成状态机。电子设备100从全局变量中获取到传感器上电状态机和第一寄存器数据写入传感器完成状态机，即可确定相机应用对应的传感器已经完成上电，且相机应用对应的传感器已经完成传感器初始化，则电子设备100从全局变量中获取各传感器的第二通道，以直接执行后续步骤，如S1034所示的步骤。
可选地，电子设备100若没有检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，则电子设备100间隔10ms再检测一次全局变量中是否包含传感器上电状态机和第一寄存器数据写入传感器完成状态机。
可选地，若电子设备100从第一次检测全局变量中是否包含传感器上电状态机和第一寄存器数据写入传感器完成状态机开始计时，5s后仍没有检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，则电子设备100触发相机应用的进程保护机制，以终止相机应用当前正在执行的流程以及后续待执行的流程。5s可以为预设的最大等待时间。
S1034、电子设备100跳过将第一寄存器数据写入传感器的流程，通过各传感器的第 二通道将第二寄存器数据分别写入相机应用对应的各传感器。第二寄存器数据包括根据数据流大小选择的配流寄存器数据。例如，电子设备100通过调用第三函数(如load sensor config cmds函数)，将第二寄存器数据分别写入相机应用对应的各传感器。
可选地，响应于切换目标传感器的第k数据输出模式到第v数据输出模式的指令，电子设备100可以冲洗(flush)目标传感器的第k数据输出模式对应的数据输出线程(pipeline)的数据，重新配置第v数据输出模式的对应的数据输出线程(pipeline)的数据。例如，电子设备100可以重新向目标传感器写入第二寄存器数据，以实现对第v数据输出模式的对应的数据输出线程(pipeline)的数据的配置，且目标传感器不会下电。
在本申请实施例中，若步骤S1034检测到全局变量中包含传感器上电状态机和第一寄存器数据写入传感器完成状态机，电子设备100无需再对各传感器进行传感器初始化，电子设备100直接跳过将第一寄存器数据写入传感器的流程，通过相机应用的各传感器的第二通道将第二寄存器数据分别写入相机应用对应的各传感器，实现各传感器启流。
这样，电子设备100在执行步骤S1034时，节省了向传感器写入第一寄存器数据的时间，此外，步骤S1034无需向传感器写入第一寄存器数据，由步骤S1031创建的相机应用对应的其他节点的线程的执行若与步骤S1034并行，则其他节点的线程的执行将获得更充裕的内存和CPU资源，也进一步使得步骤S103能快速完成。
各传感器启流后，各传感器可以采集场景图像，并将采集的场景图像传输到电子设备100的HAL层。响应于相机应用从应用层发送到HAL层的预览请求和轮转buffer，电子设备100调用HAL层的相机算法库中的图像算法，在HAL层对相机应用对应的传感器输出的场景图像进行处理后，将处理后的图像写入轮转buffer。相机应用将轮转buffer中的图像预览显示到相机应用的界面。
进一步地，在电子设备100通过相机应用的各传感器的第二通道将第二寄存器数据分别写入相机应用对应的各传感器之后，电子设备100可以调用全局变量维护接口，注销全局变量中的第二线程，以释放出更多内存。
可选地，若电子设备100在执行步骤S102时创建第二线程失败，则电子设备100终止执行步骤S102，电子设备100可以继续执行S103所示的步骤，实现对相机应用对应传感器的传感器初始化和对相机应用的数据流配置。这样，相机应用的界面仍可以显示出传感器采集输出的场景图像。
可选地，第二线程还包括线程开关。电子设备100也可以在电子设备100的可用内存和CPU资源总量大于资源阈值时关闭线程开关。如图9所示，电子设备100若检测出线程开关为开，则可以执行步骤102。相应地，电子设备100若检测出线程开关为关，这样电子设备100在执行完步骤S101后可以跳过步骤S102，执行步骤S103。示例性地，电子设备100若检测出线程开关的标识为开启标识，则表征电子设备100检测出线程开关为开；相应地，电子设备100若检测出线程开关的标识为关闭标识，则表征电子设备100检测出线程开关为关。开启标识如标识1，关闭标识如标识0。
可选地，在电子设备100通过第三线程对相机应用进行数据流配置之后，电子设备100还可以响应关闭相机应用的指令，检测相机应用的逻辑标识，获取传感器相关指针。若逻辑标识为第一模式的标识，则电子设备100基于所获取的传感器相关指针，释放第一通道，初始化传感器上电状态机，释放传感器的第二通道并注销第二线程；若逻辑标识为第二模 式的标识，则电子设备100初始化传感器上电状态机，释放传感器的第二通道并注销第二线程。这样，可以提高相机应用与其他应用的适配性能，使得其他应用触发相机应用后第二线程能被正常创建和执行，进而使得相机应用被其他应用触发后，能快速完成相机应用对应传感器的初始化和相机应用的数据流配置。
可选地，在电子设备100通过第三线程对相机应用进行数据流配置之后，电子设备100响应切换退出相机应用的指令，摧毁相机应用对应的各节点的线程和第一通道，并摧毁第二通道。若第一通道和第二通道是CSL服务的第一通道和第二通道，这样，相机应用退出后，所有的CSL服务的第一通道和第二通道均会被释放，以使得相机应用退出后，电子设备100拥有更多可用的CPU资源以及更多可用的输入/输出(input/output，I/O)资源。
本申请实施例提供的相机应用配置方法使相机应用对应传感器的传感器初始化提前完成，加速了对相机应用的数据流配置，可以减少从相机应用被触发，到相机应用的界面显示出场景图像的耗时。若用户的电子设备上的相机应用采用本申请实施例提供的相机应用配置方法，用户触发相机应用后，相机应用的界面将及时显示出用户想抓拍的目标场景图像，使得用户完成对目标场景图像的抓拍，提升用户对相机应用的使用体验感。此外，电子设备若采用本申请实施例提供的相机应用配置方法，相机应用被触发后，即使第二线程无法创建，该相机应用的界面仍然可以预览显示出传感器采集后输出的场景图像。
本申请实施例提供一种电子设备，该终端设备包括：包括：处理器和存储器；存储器存储计算机执行指令；处理器执行存储器存储的计算机执行指令，使得电子设备执行上述方法。
本申请实施例提供一种芯片。图12为本申请实施例提供的芯片的硬件结构示意图。芯片包括一个或两个以上(包括两个)处理器81、通信线路82、通信接口83和存储器84。处理器81用于调用存储器中的计算机程序，以执行上述实施例中的技术方案。其实现原理和技术效果与上述相关实施例类似，此处不再赘述。
本申请实施例还提供了一种计算机可读存储介质。计算机可读存储介质存储有计算机程序。计算机程序被处理器执行时实现上述方法。上述实施例中描述的方法可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。如果在软件中实现，则功能可以作为一个或多个指令或代码存储在计算机可读介质上或者在计算机可读介质上传输。计算机可读介质可以包括计算机存储介质和通信介质，还可以包括任何可以将计算机程序从一个地方传送到另一个地方的介质。存储介质可以是可由计算机访问的任何目标介质。
一种可能的实现方式中，计算机可读介质可以包括RAM，ROM，只读光盘(compact disc read-only memory，CD-ROM)或其它光盘存储器，磁盘存储器或其它磁存储设备，或目标于承载的任何其它介质或以指令或数据结构的形式存储所需的程序代码，并且可由计算机访问。而且，任何连接被适当地称为计算机可读介质。例如，如果使用同轴电缆，光纤电缆，双绞线，数字用户线(Digital Subscriber Line，DSL)或无线技术(如红外，无线电和微波)从网站，服务器或其它远程源传输软件，则同轴电缆，光纤电缆，双绞线，DSL或诸如红外，无线电和微波之类的无线技术包括在介质的定义中。如本文所使用的磁盘和光盘包括光盘，激光盘，光盘，数字通用光盘(Digital Versatile Disc，DVD)，软盘和蓝光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光光学地再现数据。上述的组合也应包括在计算机可读介质的范围内。
本申请实施例提供一种计算机程序产品，计算机程序产品包括计算机程序，当计算机程序被运行时，使得计算机执行上述方法。
本申请实施例是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程设备的处理单元以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理单元执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
以上的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本发明的保护范围之内。

Claims (17)

  1. 一种相机应用配置方法，其特征在于，应用于电子设备，所述方法包括：
    第一时刻，通过第一线程获取相机参数；
    第二时刻，通过第二线程对传感器进行传感器初始化；所述传感器包括相机应用对应的传感器；所述传感器初始化与所述相机参数有关；
    第三时刻，通过第三线程对所述相机应用进行数据流配置；所述数据流包括所述相机应用的各功能对应的数据流；所述数据流配置与所述相机参数以及所述传感器初始化中得到的参数有关，所述第三时刻晚于所述第二时刻。
  2. 根据权利要求1所述的方法，其特征在于，所述通过第二线程对传感器进行传感器初始化之前，还包括：
    检测所述相机应用的逻辑标识，所述逻辑标识包括第一模式的标识或第二模式的标识；所述第一模式对应m个传感器，所述第二模式对应一个传感器，m为大于0的整数；
    若所述逻辑标识为第一模式的标识，则创建所述相机应用对应的第一通道，并创建m个所述第二线程；
    所述通过第二线程对传感器进行传感器初始化，包括：通过所述m个第二线程分别对所述m个传感器进行传感器初始化；其中，所述第一通道用于管理所述相机应用与所述m个传感器之间的接口。
  3. 根据权利要求2所述的方法，其特征在于，所述通过所述m个第二线程分别对所述m个传感器进行传感器初始化，包括：
    对于所述m个第二线程的任一个第二线程，通过所述任一个第二线程执行：申请资源缓存，从所述第一通道中得到所述m个传感器中任一个传感器的第二通道，以及向所述任一个传感器写入第一寄存器数据；所述第二通道用于为所述任一个传感器上电。
  4. 根据权利要求3所述的方法，其特征在于，所述资源缓存包括包缓存区；
    所述向所述任一个传感器写入第一寄存器数据，包括：
    从所述相机应用获取第一寄存器数据；
    将所述第一寄存器数据封装后得到的第一封装包存储于所述包缓冲区；
    解析从所述包缓存区获取的第一封装包，得到解析后的第一寄存器数据；
    将所述解析后的第一寄存器数据写入所述任一个传感器。
  5. 根据权利要求2所述的方法，其特征在于，还包括：
    若所述逻辑标识为第二模式的标识，则创建一个第二线程；
    所述通过第二线程对传感器进行传感器初始化，包括：通过所述一个第二线程对所述一个传感器进行传感器初始化。
  6. 根据权利要求5所述的方法，其特征在于，所述通过所述一个第二线程对所述一个传感器进行传感器初始化，包括：
    通过所述一个第二线程执行：申请资源缓存，创建所述第一通道，从所述第一通道中得到所述一个传感器的第二通道，以及向所述一个传感器写入第一寄存器数据；所述第二通道用于为所述一个传感器上电。
  7. 根据权利要求6所述的方法，其特征在于，所述资源缓存包括包缓存区；
    所述向所述一个传感器写入第一寄存器数据，包括：
    从所述相机应用获取第一寄存器数据；
    将所述第一寄存器数据封装后得到的第一封装包存储于所述包缓冲区；
    解析从所述包缓存区获取的第一封装包，得到解析后的第一寄存器数据；
    将所述解析后的第一寄存器数据写入所述一个传感器。
  8. 根据权利要求1-7任一项所述的方法，其特征在于，所述第二线程包括对应的同步锁；所述同步锁用于对所述第二线程对应的执行对象进行锁定。
  9. 根据权利要求3、4、6或7所述的方法，其特征在于，所述方法还包括：
    创建用于管理所述相机应用的全局变量的全局变量维护接口；
    将所述第一通道、所述第二线程、所述第二通道和所述第二线程对应的流程状态机维护在所述全局变量中；所述流程状态机包括流程的流程状态、流程转换、流程动作构成的有向图，所述第二线程对应的流程状态机包括传感器上电状态机和第一寄存器数据写入所述传感器完成状态机。
  10. 根据权利要求9所述的方法，其特征在于，所述通过第三线程对所述相机应用进行数据流配置，包括：
    初始化所述第一通道，创建所述相机应用对应的各节点的线程；所述节点包括用于初始化所述传感器的节点；
    若检测到所述全局变量中包含用于初始化所述传感器的节点的线程，则从所述全局变量中获取所述第一通道；
    若检测到所述全局变量中包含所述传感器上电状态机和第一寄存器数据写入所述传感器完成状态机，则从所述全局变量中获取各所述传感器的第二通道；
    跳过将所述第一寄存器数据写入所述传感器的流程，通过各所述传感器的第二通道将第二寄存器数据分别写入各所述传感器；所述第二寄存器数据包括根据所述数据流大小选择的配流寄存器数据。
  11. 根据权利要求10所述的方法，其特征在于，在所述通过各所述传感器的第二通道将第二寄存器数据分别写入各所述传感器之后，还包括：
    注销所述全局变量中的所述第二线程。
  12. 根据权利要求11所述的方法，其特征在于，在所述通过第三线程对所述相机应用进行数据流配置之后，还包括：
    响应关闭所述相机应用的指令，检测所述相机应用的逻辑标识；所述逻辑标识包括第一模式的标识或第二模式的标识，所述第一模式对应m个传感器，所述第二模式对应一个传感器，m为大于0的整数；
    若所述逻辑标识为第一模式的标识，则释放所述第一通道，初始化所述传感器上电状态机，释放所述传感器的第二通道并注销所述第二线程；
    若所述逻辑标识为第二模式的标识，则初始化所述传感器上电状态机，释放所述传感器的第二通道并注销所述第二线程。
  13. 根据权利要求11所述的方法，其特征在于，在所述通过第三线程对所述相机应用进行数据流配置之后，还包括：
    响应切换退出所述相机应用的指令，摧毁所述相机应用对应的各节点的线程和所述第一通道；并摧毁所述第二通道。
  14. 一种电子设备，其特征在于，包括：处理器和存储器；
    所述存储器存储计算机执行指令；
    所述处理器执行所述存储器存储的计算机执行指令，使得所述电子设备执行如权利要求1-13中任一项所述的方法。
  15. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-13任一项所述的方法。
  16. 一种计算机程序产品，其特征在于，包括计算机程序，当所述计算机程序被运行时，使得计算机执行如权利要求1-13任一项所述的方法。
  17. 一种芯片，所述芯片包括处理器，处理器用于调用存储器中的计算机程序，以执行如权利要求1-13任一项所述的方法。
PCT/CN2023/134117 2023-02-27 2023-11-24 相机应用配置方法、设备及存储介质 Ceased WO2024179072A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310227589.8A CN117149294B (zh) 2023-02-27 2023-02-27 相机应用配置方法、设备及存储介质
CN202310227589.8 2023-02-27

Publications (2)

Publication Number Publication Date
WO2024179072A1 true WO2024179072A1 (zh) 2024-09-06
WO2024179072A9 WO2024179072A9 (zh) 2024-10-24

Family

ID=88908755

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/134117 Ceased WO2024179072A1 (zh) 2023-02-27 2023-11-24 相机应用配置方法、设备及存储介质

Country Status (2)

Country Link
CN (1) CN117149294B (zh)
WO (1) WO2024179072A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119012002A (zh) * 2024-10-25 2024-11-22 荣耀终端有限公司 相机控制方法、设备、芯片系统、存储介质及程序产品

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105120112A (zh) * 2015-09-25 2015-12-02 维沃移动通信有限公司 一种移动终端的相机启动方法及移动终端
CN111510627A (zh) * 2020-04-23 2020-08-07 Oppo广东移动通信有限公司 相机的启动方法和装置、终端和可读存储介质
CN113032030A (zh) * 2021-03-29 2021-06-25 北京小米移动软件有限公司 相机启动方法、装置、终端设备和存储介质
CN113971048A (zh) * 2021-10-29 2022-01-25 Oppo广东移动通信有限公司 应用程序的启动方法及装置、存储介质及电子设备
CN114064129A (zh) * 2020-08-10 2022-02-18 北京小米移动软件有限公司 一种相机启动方法和装置、电子设备
US20220201201A1 (en) * 2019-09-12 2022-06-23 Huawei Technologies Co., Ltd. Callback stream processing method and device

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8130275B2 (en) * 2008-06-13 2012-03-06 Nintendo Co., Ltd. Information-processing apparatus, and storage medium storing a photographing application launch program executed by information-processing apparatus
KR20120036209A (ko) * 2010-10-07 2012-04-17 삼성전자주식회사 컴퓨팅 시스템에서 부트 이미지의 크기를 최적화하는 부팅이미지 생성방법
CN102768680A (zh) * 2012-06-26 2012-11-07 广东欧珀移动通信有限公司 一种相机的快速搜索方法
KR101959359B1 (ko) * 2012-11-06 2019-03-18 에이치피프린팅코리아 유한회사 빠른 부팅을 위한 부트 이미지를 업데이트하는 방법 및 이를 수행하는 화상형성장치
CN104239336B (zh) * 2013-06-19 2018-03-16 华为技术有限公司 一种图像筛选方法、装置及终端
CN107222677A (zh) * 2017-05-27 2017-09-29 成都通甲优博科技有限责任公司 多摄像头同时打开的方法及装置
JP7139584B2 (ja) * 2017-08-17 2022-09-21 ソニーグループ株式会社 情報処理装置、情報処理方法、プログラム、および情報処理システム
CN108198124B (zh) * 2017-12-27 2023-04-25 上海联影医疗科技股份有限公司 医学图像处理方法、装置、计算机设备和存储介质
CN108989680B (zh) * 2018-08-03 2020-08-07 珠海全志科技股份有限公司 摄像进程启动方法、计算机装置及计算机可读存储介质
CN111104180B (zh) * 2019-12-11 2021-08-03 惠州Tcl移动通信有限公司 应用程序优化方法、装置、存储介质及电子设备
CN113010300B (zh) * 2019-12-20 2024-07-23 北京字节跳动网络技术有限公司 图像效果的刷新方法、装置、电子设备及计算机可读存储介质
CN111447370B (zh) * 2020-05-19 2021-07-27 Oppo广东移动通信有限公司 摄像头访问方法、访问装置、终端设备及可读存储介质
US11341025B2 (en) * 2020-05-27 2022-05-24 Red Hat Inc. Dynamic tuning of computing devices using application log data
CN111935663B (zh) * 2020-07-17 2022-01-25 腾讯科技（深圳）有限公司 传感器数据流的处理方法、装置、介质及电子设备
CN112616024A (zh) * 2020-12-31 2021-04-06 深兰科技（上海）有限公司 Usb摄像头数据获取方法、装置、电子设备和存储介质
CN121442182A (zh) * 2021-06-28 2026-01-30 北京小米移动软件有限公司 一种控制摄像头模组的方法、装置及可读存储介质
CN113596345B (zh) * 2021-08-09 2023-01-17 荣耀终端有限公司 参数的调整方法、显示的控制方法、电子设备及介质
CN113965693B (zh) * 2021-08-12 2022-12-13 荣耀终端有限公司 一种视频拍摄方法、设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105120112A (zh) * 2015-09-25 2015-12-02 维沃移动通信有限公司 一种移动终端的相机启动方法及移动终端
US20220201201A1 (en) * 2019-09-12 2022-06-23 Huawei Technologies Co., Ltd. Callback stream processing method and device
CN111510627A (zh) * 2020-04-23 2020-08-07 Oppo广东移动通信有限公司 相机的启动方法和装置、终端和可读存储介质
CN114064129A (zh) * 2020-08-10 2022-02-18 北京小米移动软件有限公司 一种相机启动方法和装置、电子设备
CN113032030A (zh) * 2021-03-29 2021-06-25 北京小米移动软件有限公司 相机启动方法、装置、终端设备和存储介质
CN113971048A (zh) * 2021-10-29 2022-01-25 Oppo广东移动通信有限公司 应用程序的启动方法及装置、存储介质及电子设备

Also Published As

Publication number Publication date
WO2024179072A9 (zh) 2024-10-24
CN117149294B (zh) 2024-07-23
CN117149294A (zh) 2023-12-01

Similar Documents

Publication Publication Date Title
US20230419570A1 (en) Image Processing Method and Electronic Device
CN113254120B (zh) 数据处理方法和相关装置
WO2020259452A1 (zh) 一种移动终端的全屏显示方法及设备
CN113722058B (zh) 一种资源调用方法及电子设备
CN111104209B (zh) 一种处理任务的方法及相关设备
US20240264882A1 (en) Application running method and related device
CN115079886B (zh) 二维码识别方法、电子设备以及存储介质
WO2023030099A1 (zh) 跨设备交互的方法、装置、投屏系统及终端
WO2020155875A1 (zh) 电子设备的显示方法、图形用户界面及电子设备
WO2022105445A1 (zh) 基于浏览器的应用投屏方法及相关装置
CN115083400B (zh) 语音助手唤醒方法及装置
CN111555825A (zh) 射频资源分配方法及装置
CN114461589B (zh) 读取压缩文件的方法、文件系统及电子设备
WO2023020528A1 (zh) 一种投屏方法、设备、存储介质及计算机程序产品
WO2024066834A1 (zh) Vsync信号的控制方法、电子设备、存储介质及芯片
WO2024051634A1 (zh) 一种投屏显示的方法、系统以及电子设备
WO2024159950A1 (zh) 一种显示方法、装置、电子设备及存储介质
WO2024179072A9 (zh) 相机应用配置方法、设备及存储介质
CN111475363A (zh) 卡死识别方法及电子设备
WO2025086648A1 (zh) 一种请求调度方法和设备
WO2024152910A1 (zh) 一种录像方法以及电子设备
US12367006B2 (en) Display method, electronic device, and system
CN113542315B (zh) 通信框架、业务事件处理方法及装置
CN115079981A (zh) 设备控制方法及相关装置
CN115269167A (zh) 资源调度方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23925022

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23925022

Country of ref document: EP

Kind code of ref document: A1