CN116074628A - Data processing method and electronic equipment - Google Patents
Data processing method and electronic equipment Download PDFInfo
- Publication number
- CN116074628A CN116074628A CN202210975685.6A CN202210975685A CN116074628A CN 116074628 A CN116074628 A CN 116074628A CN 202210975685 A CN202210975685 A CN 202210975685A CN 116074628 A CN116074628 A CN 116074628A
- Authority
- CN
- China
- Prior art keywords
- position information
- time
- data packet
- time point
- ois
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Studio Devices (AREA)
Abstract
The application provides a data processing method and electronic equipment, and relates to the technical field of terminals. The problem of the not good anti-shake effect of shooting image is solved. The specific scheme is as follows: when the system time is a first time point, acquiring a first data packet; the first data packet includes a plurality of location information and a first time point. The first time point corresponds to first position information with the latest acquisition time in the plurality of position information; determining a second point in time when the first number of location information in the first data packet is different from the first theoretical number; the first theoretical quantity is the quantity of position information acquired by the electronic equipment between the third time point and the first time point; and correcting the acquisition time corresponding to the first position information to a second time point. The acquisition time of the obtained position information is ensured to be accurate, so that the anti-shake processing effect of the follow-up shooting image is improved.
Description
The present application claims priority from the national intellectual property agency, application number 202210601063.7, chinese patent application entitled "multi-packet data timestamp conversion method due to DX platform OIS device read latency", filed 5/30 of 2022, the entire contents of which are incorporated herein by reference.
Technical Field
The present disclosure relates to the field of terminal technologies, and in particular, to a data processing method and an electronic device.
Background
When a user holds an electronic device (such as a mobile phone) to shoot, the shot image data can have image blurring due to factors such as hand shake and the like. In the related art, the influence of the hand shake of the user on the captured image data can be solved by using an optical image stabilization (optical Image stabilization, OIS) technique and an electronic image stabilization (electronics image Stabilization, abbreviated as "EIS") technique. However, whatever technique is employed, lens position information acquired by OIS firmware is required.
Typically, before reading lens position information from OIS firmware, a current system time point, which may be referred to as a system time stamp, needs to be written into OIS firmware. In this way, the OIS firmware can correlate the latest acquired lens position information with the system time point, and the system time point is used for indicating the acquisition time of the lens position information, so that the subsequent processing is convenient. However, in the process of writing in the system time point, since the time delay inevitably exists, a large error exists between the system time point corresponding to the lens position information and the actual acquisition time point, and the error directly affects the anti-shake effect of shooting.
Disclosure of Invention
The embodiment of the application provides a data processing method and electronic equipment, which are used for solving the problem of larger error between a system time point corresponding to lens position information and an actual acquisition time point and improving the image anti-shake effect of an OIS technology and an EIS technology.
In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:
in a first aspect, an embodiment of the present application provides a data processing method, which is applied to an electronic device, where the electronic device includes a lens, and the method includes: when the system time is a first time point, acquiring a first data packet; the first data packet includes a plurality of location information and the first time point. The plurality of position information are a plurality of hall data indicating positions of the lens, which are continuously collected by the electronic device, that is, the plurality of position information may indicate a change condition of the positions of the lens of the electronic device in a period of time. In the first packet, the first time point corresponds to the first position information having the latest acquisition time among the plurality of position information, and indicates the acquisition time of the first position information.
And under the condition that the first quantity of the position information in the first data packet is the same as the first theoretical quantity, calculating a second time point corresponding to the first position information, and correcting the acquisition time corresponding to the first position information to be the second time point.
The first theoretical number is the number of position information that can be acquired by the electronic device between a third time point and a first time point, where the third time point is the acquisition time corresponding to the position information with the latest acquisition time in a second data packet, and the second data packet is the data packet acquired next time.
In the above embodiment, by the number of the position information in the data packet, whether the position information in the data packet matches with the time point carried by the data packet is analyzed. And under the condition of determining mismatch, if the first number of the first data packets is different from the first theoretical number, timely correcting the acquisition time of the position information. Therefore, when the image anti-shake processing is carried out subsequently, the position information with accurate acquisition time is provided, the processing effect can be improved, and further, the picture quality of the electronic equipment is improved.
In some possible embodiments, the electronic device includes an optical image stabilization OIS firmware, and the acquiring the first data packet includes: transmitting first information to the OIS firmware, where the first information includes the first time point, the first information indicates the OIS firmware to associate the first time point with the first location information, and the first location information is location information that is acquired by the OIS firmware recently; sending second information to the OIS firmware, wherein the second information indicates the OIS firmware to package the first position information and second position information into the first data packet, and the second position information is position information stored in the OIS firmware and acquired earlier than the first position information; and receiving the first data packet sent by the OIS firmware.
It can be understood that, in the process of obtaining the first data packet, the electronic device needs to send first information to the OIS firmware, so as to write the current system time point, such as the first time point, into the OIS firmware. Then, the first data packet is read from the OIS firmware. If the delay of the first information transmitted to the OIS firmware is within a reasonable range, the corresponding first number is the same as the first theoretical number, and at this time, the first time point in the first data packet may accurately indicate the actual acquisition time of the first location information. Otherwise, if the delay of the first information transmitted to the OIS firmware does not belong to a reasonable range, the corresponding first quantity and the first theoretical quantity are different, and a larger error exists between the first time point carried in the first data packet and the actual acquisition time of the first position information.
In summary, in the above embodiment, the electronic device determines whether the first information has a problem of long transmission delay by using the number of location information in the first data packet. When the problem of overlong transmission time delay exists, that is, when the actual acquisition time of the first position information is greatly different from the first time point, the acquisition time corresponding to the first position information is corrected in time, so that the obtained position information can be used for subsequent OIS processing or EIS processing, and the effect of improving the image dithering problem is improved.
In some possible embodiments, the method further comprises: after the first data packet is acquired, the OIS firmware is instructed to delete the first location information and the second location information.
In some possible embodiments, the determining the second point in time includes: when the first number is larger than the first theoretical number, increasing the product between a first difference value and a first time on the basis of the first time point to obtain the second time point; the first difference is a difference between the first number and a first theoretical number; the first time is a time interval for the electronic device to collect position information; subtracting the product between a second difference value and the first time length on the basis of the first time point when the first number is smaller than the first theoretical number to obtain the second time point; the second difference is a difference between the first theoretical amount and the first amount.
In some possible embodiments, the first data packet further includes a plurality of second location information, the second location information having a collection time earlier than the first location information, and after correcting the collection time corresponding to the first location information to the second time point, the method further includes: and according to the second time point and the first time length, sequentially determining the acquisition time corresponding to each piece of second position information according to the acquisition sequence of the plurality of pieces of second position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information.
In some possible embodiments, the method further comprises: when the system time is the third time point, acquiring a second data packet, wherein the second data packet comprises a plurality of third position information and fourth position information, the fourth position information is associated with a third time point and is used for indicating the acquisition time of the fourth position information, and the acquisition time of the fourth position information is earlier than that of the third position information; when the second data packet meets a second condition, according to the third time point and the first time length, according to the acquisition sequence of the plurality of third position information, sequentially determining the acquisition time corresponding to each third position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information, the second condition is that the second number of the position information in the second data packet is the same as the corresponding second theoretical number, the second theoretical number is the number of the position information acquired by the electronic equipment between a fourth time point and the third time point, and the fourth time point is the system time of the data packet acquired last time.
In some possible embodiments, before determining that the first data packet meets the first condition, the method further comprises: determining a first time interval between the third point in time and the first point in time; and determining the first theoretical quantity according to the first time interval and the first duration, wherein the first duration is the time interval for the electronic equipment to acquire the position information.
In a second aspect, embodiments of the present application provide an electronic device including one or more processors and memory; the memory is coupled to the processor, the memory for storing computer program code comprising computer instructions that, when executed by the one or more processors, operate to:
when the system time is a first time point, acquiring a first data packet; the first data packet comprises a plurality of pieces of position information and the first time point, the plurality of pieces of position information are a plurality of pieces of Hall data which are continuously collected and indicate the positions of lenses, and in the first data packet, the first time point corresponds to first position information with the latest collection time in the plurality of pieces of position information and is used for indicating the collection time of the first position information;
determining a second time point when the first data packet does not meet a first condition; the first condition includes that a first number of position information in the first data packet is the same as a first theoretical number, the first theoretical number is a number of position information acquired by the electronic device between a third time point and the first time point, the third time point is an acquisition time corresponding to position information with latest acquisition time in a second data packet, and the second data packet is a data packet acquired next to the first data packet;
And correcting the acquisition time corresponding to the first position information to the second time point.
In some possible embodiments, the electronic device comprises an optically stabilized OIS firmware, the one or more processors to: transmitting first information to the OIS firmware, where the first information includes the first time point, the first information indicates the OIS firmware to associate the first time point with the first location information, and the first location information is location information that is acquired by the OIS firmware recently; sending second information to the OIS firmware, wherein the second information indicates the OIS firmware to package the first position information and second position information into the first data packet, and the second position information is position information stored in the OIS firmware and acquired earlier than the first position information; and receiving the first data packet sent by the OIS firmware.
In some possible embodiments, the one or more processors are further configured to: after the first data packet is acquired, the OIS firmware is instructed to delete the first location information and the second location information.
In some possible embodiments, the one or more processors are further configured to: when the first number is larger than the first theoretical number, increasing the product between a first difference value and a first time on the basis of the first time point to obtain the second time point; the first difference is a difference between the first number and a first theoretical number; the first time is a time interval for the electronic device to collect position information;
Subtracting the product between a second difference value and the first time length on the basis of the first time point when the first number is smaller than the first theoretical number to obtain the second time point; the second difference is a difference between the first theoretical amount and the first amount.
In some possible embodiments, the first data packet further includes a plurality of second location information, the second location information having a time of acquisition that is earlier than the first location information, and the one or more processors are further configured to, after correcting the time of acquisition corresponding to the first location information to the second time point: and according to the second time point and the first time length, sequentially determining the acquisition time corresponding to each piece of second position information according to the acquisition sequence of the plurality of pieces of second position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information.
In some possible embodiments, the one or more processors are further configured to: when the system time is the third time point, acquiring a second data packet, wherein the second data packet comprises a plurality of third position information and fourth position information, the fourth position information is associated with a third time point and is used for indicating the acquisition time of the fourth position information, and the acquisition time of the fourth position information is earlier than that of the third position information; when the second data packet meets a second condition, according to the third time point and the first time length, according to the acquisition sequence of the plurality of third position information, sequentially determining the acquisition time corresponding to each third position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information, the second condition is that the second number of the position information in the second data packet is the same as the corresponding second theoretical number, the second theoretical number is the number of the position information acquired by the electronic equipment between a fourth time point and the third time point, and the fourth time point is the system time of the data packet acquired last time.
In some possible embodiments, the one or more processors are further configured to: determining a first time interval between the third point in time and the first point in time; and determining the first theoretical quantity according to the first time interval and the first duration, wherein the first duration is the time interval for the electronic equipment to acquire the position information.
In a third aspect, embodiments of the present application provide a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and possible embodiments thereof.
In a fourth aspect, the present application provides a computer program product for, when run on an electronic device as described above, causing the electronic device to perform the method as described in the first aspect and possible embodiments thereof.
It will be appreciated that the electronic device, the computer storage medium and the computer program product described above are applied to the corresponding methods provided above, and thus, the advantages achieved by the electronic device, the computer storage medium and the computer program product are referred to the advantages of the corresponding methods provided above, and are not repeated herein.
Drawings
FIG. 1 is one of exemplary diagrams of reading position information provided in an embodiment of the present application;
FIG. 2 is a second exemplary diagram of reading position information according to an embodiment of the present application;
fig. 3 is a schematic hardware structure of an electronic device according to an embodiment of the present application;
fig. 4 is a schematic diagram of a software and hardware structure of an electronic device according to an embodiment of the present application;
FIG. 5 is one of the flowcharts of the steps of the data processing method according to the embodiment of the present application;
FIG. 6 is a second flowchart illustrating steps of a data processing method according to an embodiment of the present disclosure;
FIG. 7 is a third flowchart illustrating steps of a data processing method according to an embodiment of the present disclosure;
fig. 8 is an exemplary diagram of a data processing procedure in OIS driving provided in an embodiment of the present application;
fig. 9 is a schematic diagram of a system on chip according to an embodiment of the present application.
Detailed Description
The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.
With the progress of technology, users have also demanded more and more electronic devices (e.g., mobile phones) having an image capturing function. For example, when a user holds an electronic device for photographing (e.g., photographing or video recording), hand tremble is easily generated. When a user holds the electronic equipment, hand shake occurs, and the body of the electronic equipment and a lens module arranged on the body are driven to shake, so that a shot image frame is subjected to shake blur, which is also called image blur. Clearly, shooting anti-shake has become a fundamental requirement for mobile phones by users.
In some embodiments, the effects of user hand tremble on captured image data are addressed using optical image stabilization (optical Image stabilization, OIS) techniques and electronic image stabilization (electronics image Stabilization, abbreviated as "EIS") techniques.
The OIS technology can correct the 'optical axis deviation' by moving the position of the lens in the lens module, the principle is that the OIS firmware in the lens module detects the tiny movement, calculates the displacement amount to be compensated, and compensates the displacement amount by the compensation lens group, namely, according to the shaking direction and the displacement amount of the lens module, thereby effectively overcoming the image blurring caused by the vibration of the mobile phone.
Illustratively, OIS firmware in the lens module may include: hardware modules such as a gyroscope, a Hall sensor, an OIS motor and the like. The OIS firmware may cooperate to implement adjustment of the lens position and detection of the lens position during the shooting exposure. For example, when the mobile phone moves during exposure (for example, the mobile phone is shaken by hand shake of a user), the gyroscope can acquire the movement information of the mobile phone. The motion information may indicate a position of a lens in the OIS motor moving lens module. The data collected by the hall sensor may also be referred to as hall data, where the hall data may indicate a real-time position of a corresponding lens. Thus, the lens is continuously pushed according to the latest position of the lens and new motion data acquired by the gyroscope, and closed-loop control can be continuously realized, so that the problem of optical axis deviation is corrected in real time.
The EIS technology is a method of combining a digital image processing technology with an electronic device, and is used to remove image disturbance introduced in a video image sequence due to irregular random movement of a lens module, and stabilize the processing procedure of the image sequence, so as to eliminate or reduce movement between frames of the image sequence caused by shaking of the lens module, thereby obtaining a stable and clear image.
For example, an EIS technique is an anti-shake process based on the content of an image, and recognizes the motion condition of the image according to the content of the front and rear image frames, and after the image is configured and aligned, the image is properly cut, stretched, deformed, etc., to produce a relatively stable image sequence.
Still another EIS technique is to perform anti-shake processing based on hall data, calculate the motion condition between frames and the motion condition inside one frame of image from hall data during exposure of each frame of image, perform configuration of the image, perform appropriate clipping, stretching, deformation, and the like, and then produce a relatively stable image sequence. The algorithm is fast and low in power consumption.
The specific implementation process of the EIS technology may refer to the related technology, and will not be described herein.
In some embodiments, the electronic device may employ OIS technology and EIS technology simultaneously, so as to eliminate the influence caused by hand shake during the shooting process. For example, the first stage: the influence of the optical axis offset on the individual image frames is eliminated by using OIS technology. And a second stage: and (3) carrying out algorithm processing by using an EIS technology, and eliminating motion between frames of the image sequence. In this process, real-time position information of the lens, such as hall data collected by OIS firmware, is used. The accuracy of the hall data directly affects the final anti-shake effect.
It can be seen that at any stage, real-time position information of the lens in the lens module is required. That is, not only the position information of the lens but also the acquisition time point of the position information, which belongs to the time point on the system time axis, is required. The above-mentioned position information may be collected by OIS firmware, for example, hall data collected by hall sensors in OIS firmware. After the OIS firmware collects the location information, the collected location information may be stored, waiting for the location information to be read. For example, an OIS driver corresponding to OIS firmware may read the above-described location information from the OIS firmware.
In some embodiments, OIS firmware corresponding to any lens module may collect the position information in real time. The OIS firmware corresponding to a lens module is described below as an example. It will be appreciated that each time OIS firmware collects a frame of location information, the location information is given a corresponding native timestamp. The original time stamps corresponding to the position information of each frame are different, and the acquisition sequence of the position information of each frame can be represented. Of course, there is no direct link between the marked native timestamp and the system time of the electronic device. Obviously, the position information corresponding to the lens at each system time point cannot be determined only according to the native time stamp corresponding to each position information. Therefore, when the OIS process or the EIS process is performed later, the real-time position change of the lens cannot be determined, and the image anti-motion effect is further affected.
In some embodiments, OIS firmware may store the collected location information. Before the OIS driver reads the location information, the current system time (e.g., referred to as time point a) needs to be written into OIS firmware. In this way, OIS firmware associates the time point a with the last acquired location information (e.g., referred to as location information 1) as a system time tag corresponding to location information 1. Then, the position information 1 and a plurality of position information 2 whose acquisition time is earlier than the position information 1 in the OIS firmware are read out. In addition, after the position information 1 and the position information 2 are read out, the OIS firmware may clear the position information 1 and the plurality of position information 2 that are originally stored.
Specifically, as shown in fig. 1: the system time axis is shown in fig. 1. Each frame of position information collected by the OIS firmware corresponds to a native timestamp, and the native timestamp is not related to the system time, but the collection time of each frame of position information corresponds to a system time point on the system time axis, and the OIS firmware can collect one frame of position information every 2us and store the collected position information. The OIS driver sends instruction information a, e.g., AP qdimer Write, to the OIS firmware before reading the location information that the OIS firmware has acquired. The instruction information a includes a current corresponding system time point, such as a time point a. Of course, there is a certain time interval between the OIS driver issuing the instruction information a and the OIS firmware receiving the instruction information. If the OIS firmware receives the instruction information a at the time point b, the time interval is a time interval between the time point b and the time point a.
After the OIS firmware receives the instruction information a, the time point a is associated with the latest acquired frame of position information, and is used as a system time tag of the position information. Since the position information acquired most recently before the time point b is the position information 1, the time point a is associated with the position information 1.
Thereafter, the OIS firmware may encapsulate the location information 1 with the system time stamp and the location information 2 with the native time stamp before the location information 1 into a data packet, and send the data packet to the OIS driver. In addition, after the position information 1 and the position information 2 are sent out, the OIS firmware may delete the position information 1 and the position information 2.
Of course, the above is an ideal implementation procedure, and delay often occurs in the process of transferring instruction information by using the bus under the condition that the electronic device is operating normally. That is, as shown in fig. 2, the OIS drives the instruction information a sent at the time point a, but the OIS firmware receives the instruction information a at the time point c, and a frame of position information, such as the position information 3, is newly acquired before the time point c. Thus, OIS firmware may erroneously associate time point a with location information 3, rather than location information 1. The position information is corresponding to the dislocation of the system time, so that the effect of the follow-up image anti-shake processing is affected.
In order to improve the above problems, embodiments of the present application provide a data processing method. The method comprises the following steps: and evaluating whether a dislocation exists between the system time tag and the associated position information in the read data packet. If there is a misalignment problem, i.e. the system time stamp cannot accurately refer to the acquisition time of the position information, the value of the system time stamp needs to be calibrated. And then, determining a system time point corresponding to the position information of each frame in the data packet according to the calibrated system time tag. Of course, if there is no misalignment problem, the system time point of each frame position information is determined according to the system time stamp.
In addition, the above method can be applied to any electronic device having a photographing function. For example, the electronic device may be a tablet computer, a smart watch, a laptop, a handheld computer, a notebook, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR) \virtual reality (VR) device, or the like, which is not particularly limited in the specific form of the electronic device according to the embodiments of the present application.
Fig. 3 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application. As shown in fig. 3, the electronic device 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (universal serial bus, USB) interface 130, charge management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headset interface 170D, sensor module 180, keys 190, motor 191, indicator 192, camera 193, display 194, and subscriber identity module (subscriber identification module, SIM) card interface 195, etc.
The sensor module 180 may include a pressure sensor, a gyroscope sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
It is to be understood that the structure illustrated in the present embodiment does not constitute a specific limitation on the electronic apparatus 100. In other embodiments, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.
The controller may be a neural hub and command center of the electronic device 100. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.
A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.
In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.
It should be understood that the connection relationship between the modules illustrated in this embodiment is only illustrative, and does not limit the structure of the electronic device 100. In other embodiments, the electronic device 100 may also employ different interfaces in the above embodiments, or a combination of interfaces.
The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. The display panel may employ a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (flex), a mini, a Micro-OLED, a quantum dot light-emitting diode (quantum dot light emitting diodes, QLED), or the like.
The electronic device 100 may implement photographing functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
The ISP is used to process data fed back by the camera 193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera 193.
The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, electronic device 100 may include N cameras 193, N being a positive integer greater than 1.
The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to fourier transform the frequency bin energy, or the like.
Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: dynamic picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
The NPU is a neural-network (NN) computing processor, and can rapidly process input information by referencing a biological neural network structure, for example, referencing a transmission mode between human brain neurons, and can also continuously perform self-learning. Applications such as intelligent awareness of the electronic device 100 may be implemented through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, etc.
The above is an example of the hardware configuration of the electronic apparatus having the photographing function. The electronic device can run a DX software platform, and a software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of the application, taking an Android system with a layered architecture as an example, a software structure of an electronic device is illustrated.
Fig. 4 is a block diagram of the software and hardware structure of the electronic device according to the embodiment of the application.
The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Zhuoyun row (Android run) and system libraries, and a kernel layer, respectively.
The application layer may include a series of application packages.
As shown in fig. 4, APP (application) for calls, browsers, cameras, gallery, calendar, map, music, video, navigation, etc. may be installed in the application layer.
The application framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions.
As shown in fig. 4, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a camera service, and the like.
Wherein, the window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.
The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.
The view system described above may be used to build a display interface for an application. Each display interface may be composed of one or more controls. In general, controls may include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets (widgets), and the like.
The telephone manager is used for providing communication functions of the electronic equipment. Such as the management of call status (including on, hung-up, etc.).
The resource manager provides various resources, such as localization strings, icons, pictures, layout files, video files, and the like, to the application program.
The notification manager enables the application to display notification information in a status bar, can be used for conveying notification type messages, and can automatically disappear after a short stay without user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is presented in a status bar, a prompt tone is emitted, vibration is generated, and an indicator light blinks.
The camera service is used for providing functions such as photographing or image capturing.
As shown in fig. 4, the system library may include a plurality of functional modules. For example: layer integrators (surfeflingers), media Libraries (Media Libraries), two-dimensional/three-dimensional graphics processing Libraries, such as three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), etc. SurfaceFlinger is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications. Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc. The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like. The 2D graphics engine is a drawing engine for 2D drawing.
Android run time includes a core library and virtual machines. Android run time is responsible for scheduling and management of the Android system.
The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.
The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.
The kernel layer is a layer between hardware and software. The kernel layer at least includes OIS driver, camera driver, EIS module, etc., which is not limited in this embodiment of the present application. In this embodiment of the present application, the OIS driver includes a time tag correction module. The time tag correction module is used for correcting the system time tag of the read position information.
Various software drivers in the kernel layer can interact with a hardware layer of the electronic device to instruct the hardware device of the hardware layer to execute corresponding functions. As shown in fig. 4, the hardware layer includes a camera sensor and OIS firmware. The camera sensor and OIS firmware shown in fig. 4 may be the same as a lens module, which may also include other hardware such as lenses. In addition, the electronic device may include a plurality of lens modules.
Thus, upon receiving an operation indicating that the camera application is running, the application layer of the electronic device may enable camera driving and OIS driving through the camera service in the application framework layer. After the camera drive is started, the camera sensor may be instructed to collect and transmit image data back to the camera application. After the OIS driver is started, OIS firmware may be instructed to begin collecting motion information, position information, etc. In addition, the OIS driver may also read the location information from OIS firmware. For example, the location information is read from OIS firmware once every time period 1, e.g., 10 microseconds (us). In addition, the time tag correction module in OIS driving may be used to process the read position information. After the processing of the time tag correction module, the read position information of each frame can be determined, and on a system time axis, the corresponding accurate system time point can be also called as a system time stamp. Thus, the OIS driver can achieve ideal image anti-shake effect no matter the OIS driver performs image anti-shake processing by using the position information or the EIS module obtains the position information from the OIS driver and performs image anti-shake processing.
The implementation details of the data processing method provided in the embodiment of the present application will be described below with reference to the accompanying drawings, taking a lens module in an electronic device as an example.
In an embodiment of the present application, as shown in fig. 5, the above data processing method may include the following steps:
s101, the electronic equipment acquires the data packet a.
The data packet a may include a plurality of pieces of location information, and the plurality of pieces of location information include the location information a. It can be appreciated that the plurality of position information may be hall data collected by OIS firmware in the lens module, and may be used to indicate a position change condition of a lens in the lens module. The position information a is one frame of hall data having the latest acquisition time in the packet a.
In some embodiments, an OIS driver in the electronic device may instruct OIS firmware to initiate acquisition of location information. Thus, OIS firmware may periodically acquire location information at pre-configured acquisition time intervals. For example, if the acquisition time interval of the OIS firmware is preset to be 2us, after the OIS firmware starts to acquire the position information, the OIS firmware may acquire position information, such as hall data, every 2 us.
In addition, after the OIS firmware starts collecting the position information, the OIS driver may perform reading the position information from the OIS firmware once every time period 1. Correspondingly, the OIS firmware may package the plurality of location information into one data packet and pass it to the OIS driver.
Wherein the above-mentioned duration 1 may also be referred to as a read time interval. For example, the duration 1 may be a fixed value, such as 10us, and the OIS driver indicates 10us after the OIS firmware starts the acquisition, and the position information may be read from the OIS firmware for the first time, where the first time the position information is the position information acquired by the OIS firmware in the 10 us. After the first time of reading the position information, the OIS driver may read the position information from the OIS firmware for a second time by 10us, where the read position information is the position information collected by the OIS firmware in the second 10us, and so on.
In other examples, the duration 1 may not be a fixed value, but a variable value. Thus, the OIS driver indicates a time period 1 (e.g., 10 us) after the OIS firmware initiates the acquisition, and the first time the position information is read from the OIS firmware, the first time the OIS firmware acquires within the 10 us. After the first reading of the location information, the OIS driver may read the location information from the OIS firmware a second time for a period of time 1 (e.g., 8 us), the read location information being the location information collected by the OIS firmware within 8us, and so on.
The data packet a mentioned in the embodiment of the present application may be the position information read from the OIS firmware by the OIS driver for the ith time, that is, the position information collected by the OIS firmware in the ith 10 us. Wherein i is a positive integer greater than zero.
As one implementation, the manner in which the electronic device obtains the data packet a is as shown in fig. 6:
a101, OIS driver sends instruction information 1 to OIS firmware.
The instruction information 1 includes the current system time, for example, time point 1. The current system time refers to the system time of creating the instruction information 1 by the OIS driver. The instruction information 1 is used to indicate that the time point 1 is associated with the latest acquired position information (e.g., position information a).
A102, the OIS firmware associates the time point 1 with the position information a in response to the instruction information 1.
In some embodiments, the location information a may be the last frame location information collected by OIS firmware before receiving the instruction information 1.
A103, the OIS driver sends instruction information 2 to the OIS firmware.
Wherein the instruction information 2 indicates the read data packet a. The data packet a includes position information a associated with the time point 1 and position information whose acquisition time precedes the position information a.
In some embodiments, a103 may be located after a101, but there is no necessary order with a 102.
A104, OIS firmware determines the data packet a.
In some embodiments, OIS firmware needs to package location information a and the location information that has been collected and whose collection time is before location information a into data packet a before a 104. For example, the OIS firmware may determine the location information associated with the system time tag, determine other location information whose acquisition time is before the location information, and finally package the determined location information, thereby obtaining the data packet a. For example, packet a may be as shown in table 1 below:
TABLE 1
| Position information | Native time stamp | System time tag | |
| Position information a | Time stamp 5 | Time point 1 | |
| Position information | Time stamp | 4 | |
| Position information c | Timestamp 3 | ||
| Position information d | Timestamp 2 | ||
| Position information e | Timestamp 1 |
As shown in table 1 above, the plurality of pieces of location information in the data packet a are location information e, location information d, location information c, location information b, and location information a, respectively, and the plurality of pieces of location information have their own time stamps of time stamp 1, time stamp 2, time stamp 3, time stamp 4, and time stamp 5, respectively. It will be appreciated that the native timestamp is a timestamp given by OIS firmware to each frame of location information, and the plurality of location information is continuously collected location information, so that the timestamp 1, timestamp 2, timestamp 3, timestamp 4, and timestamp 5 are five consecutive timestamps. Of the above-described time stamps, the time stamp 5 has a marking time later than the time stamp 4, the time stamp 3, the time stamp 2, and the time stamp 1, that is, the time stamp 5 may indicate that the position information a is the position information of which the acquisition time is the latest among the above-described plurality of position information. The location information a has a system time stamp, e.g., time point 1.
In some embodiments, after a104, OIS firmware may delete the location information corresponding to packet a. Thus, the next time the position information is read from OIS firmware, no repeated position information is read.
A105, OIS firmware sends the data packet a to OIS driver.
Thus, after the OIS driver obtains the data packet a, a system time point corresponding to the position information of each frame in the data packet a, that is, a system time corresponding to the time when the position information is acquired, may be determined. The implementation process of determining the system time point corresponding to the position information of each frame may refer to the description in the subsequent embodiment, which is not repeated herein. In the following embodiment, description will be made taking an example in which the system time point corresponding to the position information a is the time point 1. Thus, the above-described time point 1 is also referred to as a fourth time point.
S102, the electronic device acquires a data packet b, wherein the data packet b comprises position information f.
The data packet b is the data read from the OIS firmware by the OIS driver i+1st time. Similar to the data packet a, the data packet b also includes a plurality of pieces of position information, and the acquisition time of the plurality of pieces of position information in the data packet b is later than that of the position information in the data packet a. The position information f in the packet b is the latest acquisition time among the plurality of pieces of position information.
In some embodiments, the implementation process of S102 described above may refer to S101. Illustratively, the OIS driver first sends instruction information 3 containing the current system time (e.g., time point 2), which is also referred to as a third time point, to the OIS firmware. OIS firmware associates time point 2 with the latest acquired frame of position information (e.g., position information f) in response to instruction information 3. Wherein the acquisition time of the location information f is no later than the OIS driver issues the instruction information 3, i.e. no later than the time point 2. The above-described position information f is also referred to as fourth position information.
Then, the OIS driver sends instruction information 4 indicating to read the location information to the OIS firmware, and in response to the instruction information 4, the OIS firmware may package the location information f associated with the time point 2 and the location information (e.g., referred to as third location information) stored and acquired before the location information f into a data packet b, and transmit the data packet b to the OIS driver. It will be appreciated that after the data packet a is sent out, OIS firmware may clear the location information corresponding to the data packet a, so there is no duplicate data between the data packet b and the data packet a. In addition, the position information (e.g., position information j) with the earliest acquisition time in the data packet b is the next position information acquired by the OIS firmware after the position information a. For example, packet b may be as shown in table 2 below:
TABLE 2
| Position information | Native time stamp | System time tag |
| Position information f | Time stamp 10 | Time point 2 |
| Position information g | Time stamp 9 | |
| Position information h | Time stamp 8 | |
| Position information i | Timestamp 7 | |
| Position information j | Time stamp 6 |
As shown in table 2 above, the plurality of pieces of location information in the data packet b are the location information j, the location information i, the location information h, the location information g, and the location information f, and the plurality of pieces of location information have the original time stamps of the time stamp 6, the time stamp 7, the time stamp 8, the time stamp 9, and the time stamp 10, respectively. It will be appreciated that the plurality of location information is location information continuously collected by OIS firmware, and thus, the time stamp 6, the time stamp 7, the time stamp 8, the time stamp 9, and the time stamp 10 are also a plurality of continuous time stamps. In addition, the marking time of the timestamp 6 is earlier than the timestamp 7, the timestamp 8, the timestamp 9 and the timestamp 10, and the timestamp 6 is the next adjacent timestamp of the timestamp 5, that is, if all the position information collected by the OIS firmware is ordered according to the collection sequence, the position information j is arranged next to the position information a. In addition, the time stamp 10 marks later than the time stamp 6, the time stamp 7, the time stamp 8, and the time stamp 9, that is, the time stamp 10 may indicate that the position information f is the position information having the latest acquisition time among the plurality of position information. The location information f has a system time stamp, e.g. time point 2.
S103, the electronic device determines that the position information f in the data packet b is matched with the time point 2 according to the data packet a.
In some embodiments, the OIS driver may determine a read time interval for the OIS driver to read data from the OIS firmware based on the system time point (e.g., time point 1) of the location information a and the time point 2 parsed from the data packet b. For example, the difference between the time point 1 and the time point 2 is taken as the reading time interval. In addition, in a scenario where the reading time interval of the OIS drive is a fixed value, the OIS drive can directly read the configured reading time interval without acquiring the difference between the time point 1 and the time point 2.
After the OIS driver determines the read time interval between data packet a and data packet b and the acquisition time interval of OIS firmware, a theoretical amount of location information (also referred to as a second theoretical amount) in data packet b may be determined. The theoretical number refers to the number of position information that OIS firmware can theoretically collect in a reading time interval. For example, the theoretical amount of location information in the data packet b may be determined from the quotient between the reading time interval and the acquisition time interval. For example, the reading time interval is 10us, the acquisition time interval is 2us, and the theoretical number of position information in the data packet b is 5.
After the theoretical amount is obtained, the actual amount of position information in the data packet b, also called the second amount, is obtained. It will be appreciated that since the acquisition time of the positional information f in the data packet b is not later than the time point 2, the actual amount of positional information in the data packet b is equal to the theoretical amount. For example, as shown in table 2, the actual number of the data packets b is also 5, and then the actual number of the position information in the data packets b is consistent with the theoretical number, which indicates that the time delay between sending the instruction information 3 by the OIS driver and receiving the instruction information 3 by the OIS firmware is not large, and in this time delay, the OIS firmware does not acquire the position information newly, so that the time point 2 may indicate the system time point corresponding to the position information f, that is, the time point 2 matches the position information f. In addition, when the actual number of the position information in the data packet b and the theoretical number are identical, it may be determined that the data packet b satisfies the second condition.
In addition, it should be noted that if the time delay between the OIS driver sending the instruction information 3 and the OIS firmware receiving the instruction information 3 is relatively large, and the OIS firmware collects the new position information during the time delay, then the association between the time point 2 and the new position information will occur, and it is obvious that the system time point corresponding to the new position information should be later than the time point 2, that is, the time point 2 does not match with the new position information. Correspondingly, the actual amount of location information in the data packet b will be greater than the theoretical amount.
S104, the electronic equipment determines a system time point corresponding to the position information in the data packet b according to the time point 2 and the acquisition time interval.
It will be appreciated that after the OIS driver determines that the location information f matches the time point 2, the system time point corresponding to the location information f may be determined as the time point 2. In addition, under the condition that the position information in the data packet b is arranged according to the acquisition sequence, the interval of the system time points corresponding to any group of adjacent position information is equal to the acquisition time interval. By utilizing the characteristic, the OIS driver can sequentially determine the system time points of other position information in the data packet b.
For example, when the time point 2 is 0, 0 minutes, 0 seconds, and 20 microseconds, and the acquisition time interval is 2us, the system time point for determining the position information in the data packet b is as follows in table 3:
TABLE 3 Table 3
In other embodiments, after S104, as shown in fig. 7, the method further includes:
s105, the electronic device acquires a data packet c, where the data packet c includes the location information k.
The data packet c (also referred to as the first data packet) is the data read from the OIS firmware by the OIS driver i+2th time. Similar to the data packet a and the data packet b, the data packet c also comprises a plurality of position information, and the acquisition time of the plurality of position information in the data packet c is later than that of the position information in the data packet b. The position information k in the packet c is the latest acquisition time among the plurality of position information.
In some embodiments, the implementation process of S105 described above may also refer to S101. Illustratively, the OIS driver first sends instruction information 5, also referred to as first information, containing the current system time (e.g., time point 3) to the OIS firmware. The above-mentioned time point 3 is also referred to as a first time point, and OIS firmware associates the time point 3 with the latest acquired one-frame position information (e.g., position information k) in response to the instruction information 5. The above-described positional information k may also be referred to as first positional information. However, due to the longer delay, the acquisition time of the position information k is later than the point in time when the OIS drive issues the instruction information 5.
Then, the OIS driver sends instruction information 6, also called second information, indicating to read the location information to the OIS firmware, and in response to the instruction information 6, the OIS firmware may package the location information k associated with the time point 3 and the location information (such as the second location information) stored and acquired before the location information k into a data packet c, and transmit the data packet c to the OIS driver. It will be appreciated that after the data packets b and a are sent out, OIS firmware may clear the location information corresponding to the data packets a and b, so there is no duplicate data between the data packet c and the data packets a and b. In addition, the position information (e.g., position information p) with the earliest acquisition time in the data packet c is the next position information acquired by the OIS firmware after the position information f. For example, packet c may be as shown in table 4 below:
TABLE 4 Table 4
| Position information | Native time stamp | System time tag |
| Position information k | Time stamp 16 | Time point 3 |
| Position information l | Time stamp 15 | |
| Position information m | Time stamp 14 | |
| Position information n | Time stamp 13 | |
| Position information o | Time stamp 12 | |
| Position information p | Time stamp 11 |
As shown in table 4 above, the plurality of pieces of position information in the packet c are the position information p, the position information o, the position information n, the position information m, the position information l, and the position information k, respectively, and the raw time stamps of the plurality of pieces of position information are the time stamp 11, the time stamp 12, the time stamp 13, the time stamp 14, the time stamp 15, and the time stamp 16, respectively. It will be appreciated that the plurality of location information is location information continuously collected by OIS firmware, and thus, the time stamp 11, the time stamp 12, the time stamp 13, the time stamp 14, the time stamp 15, and the time stamp 16 are also a plurality of continuous time stamps. In addition, the marking time of the timestamp 11 is earlier than the timestamp 12, the timestamp 13, the timestamp 14, the timestamp 15 and the timestamp 16, and the timestamp 11 is the next adjacent timestamp of the timestamp 10, that is, if all the location information acquired by the OIS firmware is ordered in the acquisition order, the above location information p is ordered in the next adjacent bit of the location information f. In addition, the time stamp 16 marks later than other time stamps, that is, the time stamp 16 may indicate that the position information k is the position information of which the acquisition time is the latest among the plurality of position information. The location information k has a system time stamp, e.g. time point 3.
And S106, the electronic equipment determines that the position information k in the data packet c is not matched with the time point 3 according to the data packet b.
In some embodiments, the process of determining whether the position information k in the data packet c matches the time point 3 may refer to S103 described above, and will not be described herein. Of course, since the acquisition time of the position information k is later than the point in time when the OIS driver issues the instruction information 5, that is, later than the point in time 3, the theoretical amount of position information in the data packet c is smaller than the actual amount. In other embodiments, the theoretical number may be greater than the actual number. In summary, the theoretical number and the actual number of the location information in the data packet c are the same, which may be called that the data packet c satisfies the first condition, and otherwise, the first condition is not satisfied.
It will be appreciated that the theoretical amount is the amount of position information that the OIS firmware can theoretically collect between the system time point (e.g., time point 2) and time point 3 (i.e., the reading time interval between the data packet c and the data packet b) of the position information f, such as the first theoretical amount. The system time point of the above-mentioned position information f is the acquisition time of the position information f, which may also be referred to as a third time point. As for the data packet c, the data packet b is the data packet collected last time, and may be referred to as the second data packet corresponding to the data packet c.
Of course, the position information k in the data packet c is the data collected after the time point 3, and does not belong to the position information collected by the OIS firmware in the reading time interval. Thus, the theoretical number of data packets c (corresponding to the first theoretical number) will be smaller than the actual number (e.g., referred to as the first number).
In case it is determined that the theoretical number of data packets c would be smaller than the actual number, the OIS driver may determine that the location information k does not match the point in time 3.
S107, the electronic device determines a system time point corresponding to the position information of each frame in the data packet c according to the time point 3, the acquisition time interval, the actual number and the theoretical number.
In some embodiments, the OIS firmware may first determine the system time point corresponding to the location information k according to the time point 3, the acquisition time interval, the actual number, and the theoretical number. For example, a difference between the actual number and the theoretical number may be obtained, such as referred to as difference 1 or as a first difference. Then, the product between the difference 1 and the acquisition time interval (also referred to as the first time period) is determined as the adjustment amount. And adding the adjustment amount to the time point 3 to obtain a system time point corresponding to the position information k, which is also called a second time point. For example, if the time point 3 is 0, 0 minutes, 0 seconds and 30 microseconds, the acquisition time interval is 2us, the actual number is 6, and the theoretical number is 5, then it can be determined that the system time point corresponding to the position information k is 0, 0 minutes, 0 seconds and 32 microseconds.
In addition, if the actual number is smaller than the theoretical number, a difference between the theoretical number and the actual number may be obtained, such as a second difference. Then, the product of the second difference and the acquisition time interval (also called the first time period) is determined as the adjustment amount. The above adjustment amount is subtracted from the time point 3 to obtain a system time point corresponding to the position information k, which is also referred to as a second time point.
After determining the system time point corresponding to the position information k, the system time point of other position information in the data packet c is determined by using the acquisition time interval. For example, table 5 below shows:
TABLE 5
In addition, the system time points corresponding to the position information in the packet a mentioned in the foregoing embodiment may refer to S103 to S104, and S106 to S107. In this process, if the data packet a is the first read location information, the corresponding reading time interval may be determined according to the time point a and the time point when the OIS firmware starts the location information acquisition. If the data packet a is not the first read location information, the corresponding read time interval may be determined from the time point 4 corresponding to the time point a and the data packet d. The data packet d includes the i-1 th time of position information read from the OIS firmware. The time point 4 is a system time point corresponding to the position information with the latest acquisition time in the data packet d.
In some embodiments, OIS drivers are used during each round of data processing, as shown in fig. 8: the current system time is obtained and assigned to the first variable (time_to_fw), and then the value of the first variable is sent to OIS firmware, that is, a101 in the above embodiment. Thereafter, the OIS firmware reads the data packet 1, and the latest position information (e.g., referred to as position information q) acquired in the data packet 1 is associated with the system time stamp 1. Then, the system time stamp 1 is obtained from the data packet 1 and assigned to the second variable (time_in_fw). At this time, the value of the second variable is the same as the value of the first variable.
The number of frames of the position information in the data packet 1 is acquired and assigned to a third variable (samples). Thereafter, a system time point of the position information (e.g., position information r) with the latest acquisition time is acquired from the data packet read next and last, e.g., referred to as data packet 2, and assigned to the fourth variable (pre_time_in_fw). Then, the theoretical amount of position information in the data packet 1 is determined using the time_in_fw, the pre_time_in_fw, and the acquisition time interval, and assigned to the fifth variable (cal_samples). Then, the sequence tag of the position information q in the packet 1 is assigned to the sixth variable (last_valid_index). It can be understood that each position information in the data packet 1 corresponds to a sequence tag, and the size of the sequence tag indicates the sequence of collection in the position information. The sequence tag corresponding to the position information q may be a value of samples minus 1. The value of the sequence flag of the position information of the earliest acquisition time in the packet 1 is 0. The difference of the sequential labels of the adjacent two position information is 1.
When the value of last_valid_index is a sequential tag of the position information q, a system time point of the position information q is determined. The determination process may refer to the foregoing embodiments, and will not be described herein. After determining the system time point of the position information q, the system time point is assigned to an mth seventh variable (timetables m), where m is the total number of position information in the data packet 1.
After that, the value of last_valid_index is decremented by 1. Thus, after the last_valid_index update, the system time point of the position information indicated by the last_valid_index is determined, for example, the acquisition time interval is subtracted on the basis of the timesamples [ m ]. The obtained system time point is assigned to the m-1 th seventh variable (timestamps [ m-1 ]).
The value of last_valid_index is decremented by 1. Thus, after the last_valid_index update, the system time point of the position information indicated by the last_valid_index is determined, for example, the acquisition time interval is subtracted on the basis of the timesamples [ m-1 ]. The obtained system time point is assigned to the m-2 th seventh variable (timestamps [ m-2 ]). The loop is then repeated until the value of last_valid_index is decremented to 0 and the system time point of the location information indicated by the sequence tag "0" is determined.
In some embodiments, the above-described method may be performed by the time tag correction module of FIG. 4.
The embodiment of the application also provides an electronic device, which may include: a memory and one or more processors. The memory is coupled to the processor. The memory is for storing computer program code, the computer program code comprising computer instructions. The computer instructions, when executed by the processor, cause the electronic device to perform the steps performed by the handset in the embodiments described above. Of course, the electronic device includes, but is not limited to, the memory and the one or more processors described above.
The embodiment of the application also provides a chip system, which can be applied to the terminal equipment in the previous embodiment. As shown in fig. 9, the system-on-chip includes at least one processor 2201 and at least one interface circuit 2202. The processor 2201 may be a processor in an electronic device as described above. The processor 2201 and the interface circuit 2202 may be interconnected by wires. The processor 2201 may receive and execute computer instructions from the memory of the electronic device described above through the interface circuit 2202. The computer instructions, when executed by the processor 2201, cause the electronic device to perform the steps performed by the handset in the embodiments described above. Of course, the chip system may also include other discrete devices, which are not specifically limited in this embodiment of the present application.
In some embodiments, it will be clearly understood by those skilled in the art from the foregoing description of the embodiments, for convenience and brevity of description, only the division of the above functional modules is illustrated, and in practical application, the above functional allocation may be implemented by different functional modules, that is, the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above. The specific working processes of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which are not described herein.
The functional units in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic or optical disk, and the like.
The foregoing is merely a specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A data processing method, applied to an electronic device, the electronic device including a lens, the method comprising:
when the system time is a first time point, acquiring a first data packet; the first data packet comprises a plurality of pieces of position information and the first time point, the plurality of pieces of position information are a plurality of pieces of Hall data which are continuously collected and indicate the positions of lenses, and in the first data packet, the first time point corresponds to first position information with the latest collection time in the plurality of pieces of position information and is used for indicating the collection time of the first position information;
determining a second time point when the first data packet does not meet a first condition; the first condition includes that a first number of position information in the first data packet is the same as a first theoretical number, the first theoretical number is a number of position information acquired by the electronic device between a third time point and the first time point, the third time point is an acquisition time corresponding to position information with latest acquisition time in a second data packet, and the second data packet is a data packet acquired next to the first data packet;
And correcting the acquisition time corresponding to the first position information to the second time point.
2. The method of claim 1, wherein the electronic device comprises an optically stabilized OIS firmware, and wherein the acquiring the first data packet comprises:
transmitting first information to the OIS firmware, where the first information includes the first time point, the first information indicates the OIS firmware to associate the first time point with the first location information, and the first location information is location information that is acquired by the OIS firmware recently;
sending second information to the OIS firmware, wherein the second information indicates the OIS firmware to package the first position information and second position information into the first data packet, and the second position information is position information stored in the OIS firmware and acquired earlier than the first position information;
and receiving the first data packet sent by the OIS firmware.
3. The method according to claim 2, wherein the method further comprises:
after the first data packet is acquired, the OIS firmware is instructed to delete the first location information and the second location information.
4. The method according to claim 1 or 2, wherein said determining a second point in time comprises:
When the first number is larger than the first theoretical number, increasing the product between a first difference value and a first time on the basis of the first time point to obtain the second time point; the first difference is a difference between the first number and a first theoretical number; the first time is a time interval for the electronic device to collect position information;
subtracting the product between a second difference value and the first time length on the basis of the first time point when the first number is smaller than the first theoretical number to obtain the second time point; the second difference is a difference between the first theoretical amount and the first amount.
5. A method according to any of claims 1-3, wherein the first data packet further comprises a plurality of second location information, the second location information having a time of acquisition that is earlier than the first location information, the method further comprising, after correcting the time of acquisition corresponding to the first location information to the second point in time:
and according to the second time point and the first time length, sequentially determining the acquisition time corresponding to each piece of second position information according to the acquisition sequence of the plurality of pieces of second position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information.
6. The method according to claim 1, wherein the method further comprises:
when the system time is the third time point, acquiring a second data packet, wherein the second data packet comprises a plurality of third position information and fourth position information, the fourth position information is associated with a third time point and is used for indicating the acquisition time of the fourth position information, and the acquisition time of the fourth position information is earlier than that of the third position information;
when the second data packet meets a second condition, according to the third time point and the first time length, according to the acquisition sequence of the plurality of third position information, sequentially determining the acquisition time corresponding to each third position information, wherein the first time length is the time interval of the electronic equipment for acquiring the position information, the second condition is that the second number of the position information in the second data packet is the same as the corresponding second theoretical number, the second theoretical number is the number of the position information acquired by the electronic equipment between a fourth time point and the third time point, and the fourth time point is the system time of the data packet acquired last time.
7. The method of claim 1, wherein prior to determining that the first data packet satisfies the first condition, the method further comprises:
Determining a first time interval between the third point in time and the first point in time;
and determining the first theoretical quantity according to the first time interval and the first duration, wherein the first duration is the time interval for the electronic equipment to acquire the position information.
8. An electronic device comprising one or more processors and memory; the memory being coupled to a processor, the memory being for storing computer program code comprising computer instructions which, when executed by one or more processors, are for performing the method of any of claims 1-7.
9. A computer storage medium comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the method of any of claims 1-7.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when run on a computer, causes the computer to perform the method according to any of claims 1-7.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210601063 | 2022-05-30 | ||
| CN2022106010637 | 2022-05-30 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116074628A true CN116074628A (en) | 2023-05-05 |
| CN116074628B CN116074628B (en) | 2023-10-20 |
Family
ID=86172175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210975685.6A Active CN116074628B (en) | 2022-05-30 | 2022-08-12 | Data processing method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116074628B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108737735A (en) * | 2018-06-15 | 2018-11-02 | Oppo广东移动通信有限公司 | Method for correcting image, electronic equipment and computer readable storage medium |
| WO2021190354A1 (en) * | 2020-03-26 | 2021-09-30 | 华为技术有限公司 | Method of collecting and tracing call chain and electronic device |
| CN113556464A (en) * | 2021-05-24 | 2021-10-26 | 维沃移动通信有限公司 | Shooting method and device and electronic equipment |
| CN113965488A (en) * | 2021-09-10 | 2022-01-21 | 北京百度网讯科技有限公司 | Method and device for delayed acquisition of data packets, electronic device and storage medium |
| CN113994658A (en) * | 2019-06-25 | 2022-01-28 | 高通股份有限公司 | Combined electronic and optical image stabilization |
| CN114024991A (en) * | 2022-01-07 | 2022-02-08 | 深圳比特微电子科技有限公司 | Data acquisition equipment, data acquisition system and electronic image stabilization equipment |
-
2022
- 2022-08-12 CN CN202210975685.6A patent/CN116074628B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108737735A (en) * | 2018-06-15 | 2018-11-02 | Oppo广东移动通信有限公司 | Method for correcting image, electronic equipment and computer readable storage medium |
| CN113994658A (en) * | 2019-06-25 | 2022-01-28 | 高通股份有限公司 | Combined electronic and optical image stabilization |
| WO2021190354A1 (en) * | 2020-03-26 | 2021-09-30 | 华为技术有限公司 | Method of collecting and tracing call chain and electronic device |
| CN113556464A (en) * | 2021-05-24 | 2021-10-26 | 维沃移动通信有限公司 | Shooting method and device and electronic equipment |
| CN113965488A (en) * | 2021-09-10 | 2022-01-21 | 北京百度网讯科技有限公司 | Method and device for delayed acquisition of data packets, electronic device and storage medium |
| CN114024991A (en) * | 2022-01-07 | 2022-02-08 | 深圳比特微电子科技有限公司 | Data acquisition equipment, data acquisition system and electronic image stabilization equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116074628B (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115473957B (en) | Image processing method and electronic equipment | |
| CN113572993B (en) | Video processing method and mobile terminal | |
| EP4160596A1 (en) | Video synthesis method and apparatus, electronic device, and storage medium | |
| CN111400605A (en) | Recommendation method and device based on eyeball tracking | |
| CN116320783B (en) | Method for capturing images in video and electronic equipment | |
| CN113660408A (en) | Anti-shake method and device for video shooting | |
| WO2023035921A1 (en) | Method for image snapshot in video recording, and electronic device | |
| CN113536866A (en) | Character tracking display method and electronic equipment | |
| CN115150542B (en) | Video anti-shake method and related equipment | |
| CN116306733B (en) | Method for amplifying two-dimensional code and electronic equipment | |
| CN111381996B (en) | Memory exception handling method and device | |
| CN116074628B (en) | Data processing method and electronic equipment | |
| WO2022121988A1 (en) | Display synchronization method, electronic device, and readable storage medium | |
| CN115802148B (en) | Method for acquiring image and electronic equipment | |
| CN114945019A (en) | Data transmission method, device and storage medium | |
| CN114630152A (en) | Parameter transmission method and device for image processor and storage medium | |
| CN117690177B (en) | Face focusing method, face focusing device, electronic equipment and storage medium | |
| CN116700578B (en) | Layer synthesis method, electronic device and storage medium | |
| CN114630153B (en) | Parameter transmission method and device for application processor and storage medium | |
| CN116266159B (en) | Page fault exception handling method and electronic equipment | |
| CN115776532B (en) | Method for capturing images in video and electronic equipment | |
| CN115460343B (en) | Image processing method, device and storage medium | |
| WO2022206600A1 (en) | Screen projection method and system, and related apparatus | |
| US20240069845A1 (en) | Focus synchronization method and electronic device | |
| US20230058296A1 (en) | Video text tracking method and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |