WO2022032611A1

WO2022032611A1 - Automatic focus control accounting for lens movement during image capture

Info

Publication number: WO2022032611A1
Application number: PCT/CN2020/109066
Authority: WO
Inventors: Jintao XU; Xiaocheng Wang; Mingchen Gao; Yaoyao HOU
Original assignee: Qualcomm Incorporated
Priority date: 2020-08-14
Filing date: 2020-08-14
Publication date: 2022-02-17

Abstract

A motor moves a lens of an image capture device toward a target lens position. An image sensor of the image capture device captures a first image frame. A lens position sensor measures whether the lens was still in motion during capture of the first image frame. If the lens was not in motion during capture of the first image frame, then the first image frame was captured with the lens at the target lens position, and the first image frame is used to determine a focus indicator corresponding to the target lens position. If the lens was in motion during capture of the first image frame, a second image frame is captured after the first image frame, and the second image frame is used (instead of the first image frame) to determine the focus indicator corresponding to the target lens position. Autofocus is performed using the focus indicator.

Description

AUTOMATIC FOCUS CONTROL ACCOUNTING FOR LENS MOVEMENT DURING IMAGE CAPTURE

FIELD

This application is related to image processing. More specifically, this application relates to systems and methods of performing automatic focus control that accounts for lens movement during capture of images.

BACKGROUND

Cameras are devices that capture images of a scene when light from the scene reaches an image sensor of the camera. Cameras generally include one or more lenses through which light travels before the light reaches the image sensor of the camera to capture the image. These lenses bend light that they receive from the scene to focus the light onto the image sensor. If the light is focused precisely on the image sensor, the scene appears sharp and in focus. If the light is not focused precisely on the image sensor, the scene appears blurry and out of focus.

Adjusting the focus of a camera is generally achieved by moving a lens of the camera either closer to or farther from the image sensor. Image data captured while the lens is at one lens position can be compared to image data captured while the lens is at another lens position to determine which lens position provides better focus on a scene. If the lens is still moving while image data is captured for a particular lens position, however, focus may be inaccurately determined for that lens position, which can lead to an inaccurate determination of which lens position provides the better focus on a scene.

SUMMARY

Systems, apparatuses, methods, and computer-readable media are described herein for performing automatic focus control techniques that detect and account for lens movement. In some examples, an image capture and processing device detects when image frames captured during an automatic focus control operation are captured during lens movement, and automatically skips those images for the purposes of checking focus at a corresponding lens position, instead using a subsequently captured image frame for checking focus at the corresponding lens position.

In one example, a method of automatic focus control is provided. The method includes actuating a motor to move a lens from a first position to a second position. The method includes receiving a first image frame captured by an image sensor. The method includes receiving a measurement from a lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame. The method includes determining, based on the measurement, that the lens is in motion during the capture time window. The method includes receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor. The method includes determining a focus indicator associated with the second position. The focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window.

In another example, an apparatus for automatic focus control is provided. The apparatus includes motor connector coupled to a motor configured to move a lens, an image sensor connector coupled to an image sensor, and a lens position sensor connector coupled to a lens position sensor. The apparatus includes one or more processors as well as one or more non-transitory storage media storing instructions. Execution of the instructions by the one or more processors causes the one or more processors to perform operations. The operations include actuating the motor using the motor connector to move the lens from a first position to a second position. The operations also include receiving, using the image sensor connector, a first image frame captured by the image sensor. The operations also include receiving, using the lens position sensor connector, a measurement from the lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame. The operations also include determining, based on the measurement, that the lens is in motion during the capture time window. The operations also include receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor. The operations also include determining a focus indicator associated with the second position. The focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window

In another example, a non-transitory computer-readable medium is provided that has stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: actuate a motor to move a lens from a first position to a second position; receive a first image frame captured by an image sensor; receive a measurement from a lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame; determine, based on the measurement, that the lens is in motion during the capture time window; receive a second image frame captured by the image sensor after the first image frame is captured by the image sensor; and determine a focus indicator associated with the second position, wherein the focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window.

In another example, an apparatus for automatic focus control is provided. The apparatus includes: means for actuating a motor to move a lens from a first position to a second position; means for receiving a first image frame captured by an image sensor; means for receiving a measurement from a lens position sensor during a capture time window within which the image sensor captures the first image frame; means for determining, based on the measurement, that the lens is in motion during the capture time window; means for receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor; and means for determining a focus indicator associated with the second position, wherein the focus indicator is determined using the second image frame rather than the first image frame in response to determining that the lens is in motion during the capture time window.

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: determining a measured position of the lens based on the measurement, wherein determining that the lens is in motion during the capture time window includes determining that the measured position has not reached the second position during the capture time window.

In some aspects, determining that the lens is in motion within the capture time window includes determining, based on the measurement, that the motor is actuated during the capture time window. In some aspects, the lens position sensor includes a Hall effect sensor and the measurement identifies a Hall feedback of the motor, and wherein determining that the lens is in motion during the capture time window is based on the Hall feedback of the motor.

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: performing a contrast detection autofocus (CDAF) procedure based on the focus indicator, wherein the focus indicator includes a contrast value. In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: performing a phase detection autofocus (PDAF) procedure based on the focus indicator, wherein the focus indicator includes a phase difference.

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: actuating the motor to move the lens from the second position to a third position automatically in response to determining the focus indicator associated with the second position. In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: delaying a movement of the lens from the second position to a third position from a first scheduled time after capture of the first image frame to a second scheduled time after capture of the second image frame in response to determining that the lens is in motion during the capture time window.

In some aspects, the motor is a voice coil motor (VCM) . In some aspects, the motor is a closed-loop voice coil motor (VCM) . In some aspects, the motor is an open-loop voice coil motor (VCM) .

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: setting a lens position value associated with the first image frame to be equal to a previous position value in response to determining that the lens is in motion during the capture time window, the previous position value associated with a previous image frame captured by the image sensor before capture of the first image frame.

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: receiving a second measurement from the lens position sensor during a second capture window within which the image sensor captures the second image frame; and determining, based on the second measurement, that the lens is stationary during the second capture window, wherein the focus indicator is determined using the second image frame also in response to determining that the lens is stationary during the second capture window.

In some aspects, the first image frame and the second image frame are captured consecutively by the image sensor.

In some aspects, the methods, apparatuses, and computer-readable medium described above further comprise: packing the first image frame together with a first lens position into a first container, the first lens position determined based on the measurement from the lens position sensor; packing the second image frame together with a second lens position into a second container, the second lens position determined based on a second measurement from the lens position sensor that is measured during a second capture window within which the image sensor captures the second image frame; unpacking first information from the first container, wherein determining that the lens is in motion during the capture time window is based on the first information from the first container; and unpacking second information from the second container, wherein determining the focus indicator associated with the second position is based on the second information from the second container.

In some aspects, the apparatus comprises a camera, a mobile device (e.g., a mobile telephone or so-called “smart phone” or other mobile device) , a wearable device, a wireless communication device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device) , a personal computer, a laptop computer, a server computer, or other device. In some aspects, the apparatus includes a camera or multiple cameras for capturing one or more images.

In some aspects, the apparatus further includes a display for displaying one or more images, notifications, and/or other displayable data. For example, the apparatus may include the display configured to display the second image frame. The apparatus may include the display configured to display an image captured by the image sensor after capture of the second image frame and after an autofocus procedure is performed using the focus indicator.

In some aspects, the apparatus further includes the motor, the image sensor, the lens position sensor, and the lens. In some aspects, the lens position sensor connector is the motor connector.

This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all drawings, and each claim.

The foregoing, together with other features and embodiments, will become more apparent upon referring to the following specification, claims, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present application are described in detail below with reference to the following figures:

FIG. 1 is a block diagram illustrating an architecture of an image capture and processing device;

FIG. 2 is a conceptual diagram illustrating different positions of a lens of an image capture device providing differences in focus;

FIG. 3 is a contrast detection autofocus (CDAF) graph illustrating the effects of lens movements on a focus value associated with image contrast;

FIG. 4 is a contrast detection autofocus (CDAF) table illustrating the effects of lens movements on a focus value associated with image contrast;

FIG. 5 is a phase detection autofocus (PDAF) graph illustrating the effects of lens movements on a phase difference value;

FIG. 6 is a phase detection autofocus (PDAF) table illustrating the effects of lens movements on a phase difference value;

FIG. 7 is a block diagram illustrating an image capture and processing device performing a hybrid autofocus operation accounting for lens position;

FIG. 8 is a flow diagram illustrating operations for packing lens position data with image data;

FIG. 9 is a flow diagram illustrating operations for unpacking lens position data from image data;

FIG. 10 is a flow diagram illustrating operations for automatic focus control; and

FIG. 11 is a diagram illustrating an example of a system for implementing certain aspects of the present technology.

DETAILED DESCRIPTION

Certain aspects and embodiments of this disclosure are provided below. Some of these aspects and embodiments may be applied independently and some of them may be applied in combination as would be apparent to those of skill in the art. In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be restrictive.

The ensuing description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the application as set forth in the appended claims.

An image capture device (e.g., a camera) is a device that receives light and captures image frames, such as still images or video frames, using an image sensor. The terms “image, ” “image frame, ” and “frame” are used interchangeably herein. An image capture device typically includes at least one lens that receives light from a scene and bends the light toward an image sensor of the image capture device. The light received by the lens passes through an aperture controlled by one or more control mechanisms and is received by the image sensor. The one or more control mechanisms can control exposure, focus, and/or zoom based on information from the image sensor and/or based on information from an image processor (e.g., a host or application process and/or an image signal processor) . In some examples, the one or more control mechanisms include a motor or other control mechanism that moves a lens of an image capture device to a target lens position.

As described in more detail below, systems and techniques are described herein for performing automatic focus control process, also referred to as an autofocus process, that is both fast and accurate. During an autofocus process, an image capture device can move its lens from one end of a range of lens positions to another. The image capture device briefly stops the lens at each of a number of lens positions along the range before directing the lens to continue moving along the range. During each brief stop of the lens, the image capture device captures an image frame and determines a focus indicator corresponding to the stopped lens position using the captured image frame. Once the image capture device determines a focus indicator for all of the lens positions at all of the stops across the range, the image capture device identifies a single lens position with the best focus for the photographed scene based on the focus indicators at each of the lens positions.

It is desirable for autofocus processes to be performed quickly so that the scene does not change during the autofocus process. One way to optimize the speed of an autofocus process is to capture only one image frame during each stop of the lens. However, an image capture device sometimes captures an image frame before the lens completely stops moving and before the lens reaches a target lens position for that stop of the lens. A focus indicator that is determined using an image frame captured while the lens is still moving is inaccurate, since the actual lens position during capture of the image frame does not match the target lens position for the stop of the lens. An image frame captured while the lens is still moving can also suffer from distortion caused by the lens movement. This focus indicator inaccuracy can occur multiple times during an autofocus process, and can result in an inaccurate autofocus process in which the image capture device ultimately identifies the wrong lens position as having the best focus for the photographed scene.

The accuracy of the autofocus process can be improved by capturing two or more image frames during each stop of the lens, and using one of the later-captured image frames to determine the focus indicator for that stop of the lens. However, capturing multiple image frames during every stop of the lens slows down the autofocus process significantly. The systems and techniques described below achieve both speed and accuracy in autofocus by adding a lens position sensor to the image capture device and by using the lens position sensor during the autofocus process. For instance, the image capture device can capture a first image frame and can determine, using the lens position sensor, whether the lens was still in motion during a capture time window within which the first image frame was captured. If the image capture device determines that the lens is stationary (no longer in motion) during the capture time window, the image capture device knows that the lens has reached a target lens position and uses the first image frame to determine a focus indicator corresponding to a target lens position before moving the lens again. On the other hand, if the image capture device determines that the lens is still in motion during the capture time window, the image capture knows that the lens has not yet reached the target lens position. The image capture device then captures a second image frame after capturing the first image frame but before moving the lens further past the target lens position, and uses the second image frame to determine the focus indicator corresponding to the target lens position. This focus indicator, determined using the first image frame or the second image frame depending on the lens position sensor measurement, can then be used to perform an autofocus process to determine a lens position with the best focus for the photographed scene.

FIG. 1 is a block diagram illustrating an architecture of an image capture and processing system 100. The image capture and processing system 100 includes various components that are used to capture and process images of scenes (e.g., an image of a scene 110) . The image capture and processing system 100 can capture standalone images (or photographs) and/or can capture videos that include multiple images (or video frames) in a particular sequence. A lens 115 of the system 100 faces a scene 110 and receives light from the scene 110. The lens 115 bends the light toward the image sensor 130. The light received by the lens 115 passes through an aperture controlled by one or more control mechanisms 120 and is received by an image sensor 130.

The one or more control mechanisms 120 may control exposure, focus, and/or zoom based on information from the image sensor 130 and/or based on information from the image processor 150. The one or more control mechanisms 120 may include multiple mechanisms and components; for instance, the control mechanisms 120 may include one or more exposure control mechanisms 125A, one or more focus control mechanisms 125B, and/or one or more zoom control mechanisms 125C. The one or more control mechanisms 120 may also include additional control mechanisms besides those that are illustrated, such as control mechanisms controlling analog gain, flash, HDR, depth of field, and/or other image capture properties.

The focus control mechanism 125B of the control mechanisms 120 can obtain a focus setting. In some examples, focus control mechanism 125B store the focus setting in a memory register. Based on the focus setting, the focus control mechanism 125B can adjust the position of the lens 115 relative to the position of the image sensor 130. For example, based on the focus setting, the focus control mechanism 125B can move the lens 115 closer to the image sensor 130 or farther from the image sensor 130 by actuating a motor or servo (or other lens mechanism) , thereby adjusting focus. In some cases, additional lenses may be included in the system 100, such as one or more microlenses over each photodiode of the image sensor 130, which each bend the light received from the lens 115 toward the corresponding photodiode before the light reaches the photodiode. The focus setting may be determined via contrast detection autofocus (CDAF) , phase detection autofocus (PDAF) , hybrid autofocus (HAF) , or some combination thereof. The focus setting may be determined using the control mechanism 120, the image sensor 130, and/or the image processor 150. The focus setting may be referred to as an image capture setting and/or an image processing setting.

The exposure control mechanism 125A of the control mechanisms 120 can obtain an exposure setting. In some cases, the exposure control mechanism 125A stores the exposure setting in a memory register. Based on this exposure setting, the exposure control mechanism 125A can control a size of the aperture (e.g., aperture size or f/stop) , a duration of time for which the aperture is open (e.g., exposure time or shutter speed) , a sensitivity of the image sensor 130 (e.g., ISO speed or film speed) , analog gain applied by the image sensor 130, or any combination thereof. The exposure setting may be referred to as an image capture setting and/or an image processing setting.

The zoom control mechanism 125C of the control mechanisms 120 can obtain a zoom setting. In some examples, the zoom control mechanism 125C stores the zoom setting in a memory register. Based on the zoom setting, the zoom control mechanism 125C can control a focal length of an assembly of lens elements (lens assembly) that includes the lens 115 and one or more additional lenses. For example, the zoom control mechanism 125C can control the focal length of the lens assembly by actuating one or more motors or servos (or other lens mechanism) to move one or more of the lenses relative to one another. The zoom setting may be referred to as an image capture setting and/or an image processing setting. In some examples, the lens assembly may include a parfocal zoom lens or a varifocal zoom lens. In some examples, the lens assembly may include a focusing lens (which can be lens 115 in some cases) that receives the light from the scene 110 first, with the light then passing through an afocal zoom system between the focusing lens (e.g., lens 115) and the image sensor 130 before the light reaches the image sensor 130. The afocal zoom system may, in some cases, include two positive (e.g., converging, convex) lenses of equal or similar focal length (e.g., within a threshold difference of one another) with a negative (e.g., diverging, concave) lens between them. In some cases, the zoom control mechanism 125C moves one or more of the lenses in the afocal zoom system, such as the negative lens and one or both of the positive lenses.

The image sensor 130 includes one or more arrays of photodiodes or other photosensitive elements. Each photodiode measures an amount of light that eventually corresponds to a particular pixel in the image produced by the image sensor 130. In some cases, different photodiodes may be covered by different color filters, and may thus measure light matching the color of the filter covering the photodiode. For instance, Bayer color filters include red color filters, blue color filters, and green color filters, with each pixel of the image generated based on red light data from at least one photodiode covered in a red color filter, blue light data from at least one photodiode covered in a blue color filter, and green light data from at least one photodiode covered in a green color filter. Other types of color filters may use yellow, magenta, and/or cyan (also referred to as “emerald” ) color filters instead of or in addition to red, blue, and/or green color filters. Some image sensors (e.g., image sensor 130) may lack color filters altogether, and may instead use different photodiodes throughout the pixel array (in some cases vertically stacked) . The different photodiodes throughout the pixel array can have different spectral sensitivity curves, therefore responding to different wavelengths of light. Monochrome image sensors may also lack color filters and therefore lack color depth.

In some cases, the image sensor 130 may alternately or additionally include opaque and/or reflective masks that block light from reaching certain photodiodes, or portions of certain photodiodes, at certain times and/or from certain angles, which may be used for phase detection autofocus (PDAF) . The image sensor 130 may also include an analog gain amplifier to amplify the analog signals output by the photodiodes and/or an analog to digital converter (ADC) to convert the analog signals output of the photodiodes (and/or amplified by the analog gain amplifier) into digital signals. In some cases, certain components or functions discussed with respect to one or more of the control mechanisms 120 may be included instead or additionally in the image sensor 130. The image sensor 130 may be a charge-coupled device (CCD) sensor, an electron-multiplying CCD (EMCCD) sensor, an active-pixel sensor (APS) , a complimentary metal-oxide semiconductor (CMOS) , an N-type metal-oxide semiconductor (NMOS) , a hybrid CCD/CMOS sensor (e.g., sCMOS) , or some other combination thereof.

The image processor 150 may include one or more processors, such as one or more image signal processors (ISPs) (including ISP 154) , one or more host processors (including host processor 152) , and/or one or more of any other type of processor 1110 discussed with respect to the computing device 1100. The host processor 152 can be a digital signal processor (DSP) and/or other type of processor. In some implementations, the image processor 150 is a single integrated circuit or chip (e.g., referred to as a system-on-chip or SoC) that includes the host processor 152 and the ISP 154. In some cases, the chip can also include one or more input/output ports (e.g., input/output (I/O) ports 156) , central processing units (CPUs) , graphics processing units (GPUs) , broadband modems (e.g., 3G, 4G or LTE, 5G, etc. ) , memory, connectivity components (e.g., Bluetooth ^TM, Global Positioning System (GPS) , etc. ) , any combination thereof, and/or other components. The I/O ports 156 can include any suitable input/output ports or interface according to one or more protocol or specification, such as an Inter-Integrated Circuit 2 (I2C) interface, an Inter-Integrated Circuit 3 (I3C) interface, a Serial Peripheral Interface (SPI) interface, a serial General Purpose Input/Output (GPIO) interface, a Mobile Industry Processor Interface (MIPI) (such as a MIPI CSI-2 physical (PHY) layer port or interface, an Advanced High-performance Bus (AHB) bus, any combination thereof, and/or other input/output port. In one illustrative example, the host processor 152 can communicate with the image sensor 130 using an I2C port, and the ISP 154 can communicate with the image sensor 130 using an MIPI port.

The image processor 150 may perform a number of tasks, such as de-mosaicing, color space conversion, image frame downsampling, pixel interpolation, automatic exposure (AE) control, automatic gain control (AGC) , CDAF, PDAF, automatic white balance, merging of image frames to form an HDR image, image recognition, object recognition, feature recognition, receipt of inputs, managing outputs, managing memory, or some combination thereof. The image processor 150 may store image frames and/or processed images in random access memory (RAM) 140/1020, read-only memory (ROM) 145/1025, a cache, a memory unit, another storage device, or some combination thereof.

Various input/output (I/O) devices 160 may be connected to the image processor 150. The I/O devices 160 can include a display screen, a keyboard, a keypad, a touchscreen, a trackpad, a touch-sensitive surface, a printer, any other output devices 1035, any other input devices 1045, or some combination thereof. In some cases, a caption may be input into the image processing device 105B through a physical keyboard or keypad of the I/O devices 160, or through a virtual keyboard or keypad of a touchscreen of the I/O devices 160. The I/O 160 may include one or more ports, jacks, or other connectors that enable a wired connection between the system 100 and one or more peripheral devices, over which the system 100 may receive data from the one or more peripheral device and/or transmit data to the one or more peripheral devices. The I/O 160 may include one or more wireless transceivers that enable a wireless connection between the system 100 and one or more peripheral devices, over which the system 100 may receive data from the one or more peripheral device and/or transmit data to the one or more peripheral devices. The peripheral devices may include any of the previously-discussed types of I/O devices 160 and may themselves be considered I/O devices 160 once they are coupled to the ports, jacks, wireless transceivers, or other wired and/or wireless connectors.

In some cases, the image capture and processing system 100 may be a single device. In some cases, the image capture and processing system 100 may be two or more separate devices, including an image capture device 105A (e.g., a camera) and an image processing device 105B (e.g., a computing device coupled to the camera) . In some implementations, the image capture device 105A and the image processing device 105B may be coupled together, for example via one or more wires, cables, or other electrical connectors, and/or wirelessly via one or more wireless transceivers. In some implementations, the image capture device 105A and the image processing device 105B may be disconnected from one another.

As shown in FIG. 1, a vertical dashed line divides the image capture and processing system 100 of FIG. 1 into two portions that represent the image capture device 105A and the image processing device 105B, respectively. The image capture device 105A includes the lens 115, control mechanisms 120, and the image sensor 130. The image processing device 105B includes the image processor 150 (including the ISP 154 and the host processor 152) , the RAM 140, the ROM 145, and the I/O 160. In some cases, certain components illustrated in the image capture device 105A, such as the ISP 154 and/or the host processor 152, may be included in the image capture device 105A.

The image capture and processing system 100 can include an electronic device, such as a mobile or stationary telephone handset (e.g., smartphone, cellular telephone, or the like) , a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the image capture and processing system 100 can include one or more wireless transceivers for wireless communications, such as cellular network communications, 802.11 wi-fi communications, wireless local area network (WLAN) communications, or some combination thereof. In some implementations, the image capture device 105A and the image processing device 105B can be different devices. For instance, the image capture device 105A can include a camera device and the image processing device 105B can include a computing device, such as a mobile handset, a desktop computer, or other computing device.

While the image capture and processing system 100 is shown to include certain components, one of ordinary skill will appreciate that the image capture and processing system 100 can include more components than those shown in FIG. 1. The components of the image capture and processing system 100 can include software, hardware, or one or more combinations of software and hardware. For example, in some implementations, the components of the image capture and processing system 100 can include and/or can be implemented using electronic circuits or other electronic hardware, which can include one or more programmable electronic circuits (e.g., microprocessors, GPUs, DSPs, CPUs, and/or other suitable electronic circuits) , and/or can include and/or be implemented using computer software, firmware, or any combination thereof, to perform the various operations described herein. The software and/or firmware can include one or more instructions stored on a computer-readable storage medium and executable by one or more processors of the electronic device implementing the image capture and processing system 100.

FIG. 2 is a conceptual diagram illustrating different positions of a lens of an image capture device. The different lens positions provide differences in focus. The conceptual diagram of FIG. 2 illustrates an image capture device 205, which may be the image capture device 105A, the image capture and processing system 100, the image capture and processing device 700 of FIG. 7, the computing device 1100 of FIG. 11, or some combination thereof. The image capture device 205 includes an image sensor 210, a lens 260 within a lens guide 270, and a motor 250. The motor 250 is an example of one of the one or more focus control mechanisms 125B of FIG. 1. Actuating the motor 250 moves the lens 260 along an axis perpendicular to the image sensor 210 and within the lens guide 270. The lens guide 270 may help restrict movement of the lens 260 to the axis perpendicular to the image sensor 210, and may include one or more lens barrels, one or more guide rails, one or more guide bars, or some combination thereof. The lens 260 is illustrated as a convex lens, but in some cases may instead be a concave lens or any other type of lens. The motor 250 can include any type of motor configured to move the lens 260. In some examples, the motor 250 may be a voice coil motor (VCM) . The motor may be an open loop voice coil motor (VCM) or a closed-loop voice coil motor (VCM) .

The image capture device 205 is illustrated in FIG. 2 capturing one or more images (e.g., one or more still images, one or more video frames of a video, etc. ) a subject 280. The subject 280 is illustrated as a human being, but may be any type of living being, object, scene, or combination thereof. The subject 280 is illustrated at two different positions, including a near subject position 225 and a far subject position 220. The near subject position 225 is closer to the image capture device 205 than the far subject position 220. Similarly, the lens 260 in FIG. 2is illustrated at two positions, including a near-focus lens position 230 and a far-focus lens position 240. While the lens 260 is at the near-focus lens position 230, the subject 280 in the near subject position 225 is in focus. While the lens 260 is in the far-focus lens position 240, the subject 280 in the far subject position 220 is in focus. The near focal length 235 is the distance between the image sensor 210 and the near-focus position 230. The far focal length 245 is the distance between the image sensor 210 and the far-focus lens position 240. The near focal length 235 exceeds the far focal length 245. A depiction of the subject 280 is illustrated upside-down beside the image sensor 210 to indicate capture of an image of the subject 280 by the image sensor 210 of the image capture device 205.

A subject 280 located between the near subject position 225 and the far subject position 220 would be in focus while the lens 260 is in a position between the near-focus lens position 230 and the far-focus lens position 240. To find a lens positon that provides focus for such a subject 280 using a contrast detection autofocus (CDAF) procedure, the motor 250 may move the lens 260, stepwise, to multiple positions along the axis perpendicular to the image sensor 210 between the near-focus lens position 230 and the far-focus lens position 240. For instance, the motor 250 may move the lens 260 from the near-focus lens position 230 to the far-focus lens position 240, or from the far-focus lens position 240 to the near-focus lens position 230. At each step, the image sensor 210 captures an image frame of the subject 280. A focus value is determined for each image frame based on a contrast between the a depiction of the subject 280 and depiction of a background in that image frame. The lens position corresponding to the image frame with the highest focus value is the lens position that provides focus for the subject 280.

To find a lens positon that provides focus for a subject 280 located between the near subject position 225 and the far subject position 220 using a phase detection autofocus (PDAF) procedure, an image frame of the subject 280 is captured. A phase difference is determined based on data from one or more phase detection (PD) photodiodes of the image sensor 210. Based on the sign (negative of positive) of the phase difference, the image capture device 205 determines whether to move the lens 260 closer to the image sensor 210 or further from the image sensor 210 to provide focus for the subject 280. Based on the magnitude (absolute value) of the phase difference, the image capture device 205 determines a distance to move the lens 260 in the determined direction to provide focus for the subject 280. In some cases, the above-described PDAF procedure is performed a number of times, to increase accuracy. In some cases, CDAF is performed after PDAF within a range of positions around the lens position that PDAF determines to provide focus for the subject 280 to further enhance the focus provided to the subject 280. This combine usage of PDAF and CDAF may be referred to as a hybrid autofocus (HAF) procedure.

CDAF focus values and PDAF phase differences may both be referred to as focus indicators, since CDAF focus values and PDAF phase differences both represent indicators as to how well an image captured while the lens 260 is in a particular lens positon is focused on the subject 280. If a CDAF, PDAF, and/or HAF autofocus function attempts to step through different lens positions and capture corresponding image frames too quickly, however, some of the image frames may be captured while the lens is moving. Focus indicators determined using an image frame that is captured while the lens 260 is moving may be inaccurate, as shown in FIGs. 3-6. Inaccurate focus indicators may result in inaccurate autofocus.

Focus values are determined during CDAF procedures and may be referred to as contrast values or contrast focus values. Phase differences are determined during PDAF procedures and may be referred to as phase difference values or phase difference focus values.

FIG. 3 is a contrast detection autofocus (CDAF) graph 300 illustrating the effects of lens movements on a focus value associated with image contrast. The CDAF graph 300 includes a curve 305 plotted on a plane. The horizontal axis of the plane represents lens positions 310, which range from 899 to 419 in the CDAF graph 300. The units for the lens positions 310 can be expressed in terms of lens movement steps, which represent how much electric current is provided to an actuator and/or driver for a motor 250 and/or other mechanism that moves the lens. Lens movement steps can be expressed in units of electric current, such as amperes (A) , milliamps (mA) , microamps (μA) , or nanoamps (nA) . For example, the

values

899 and 419 along the horizontal axis of the CDAF graph 300 may represent 899 μA and 419 μA, respectively. The units for the lens positions 310 can be micrometers (μm) . In some cases, the lens movement steps can be directly related to the lens position in micrometers (μm) ; for instance, every two lens movement steps (2 μA) can correspond to a lns movement over a distance of 1 μm. On the right-hand side of the horizontal axis, lens position value 419 represents an example of the near-focus lens position 240 from FIG. 2. On the left-hand side of the horizontal axis, lens position value 899 represents an example of the far-focus lens position 230 from FIG. 2.

The vertical axis of the plane represents focus values (FV) 315, which range from 0 to 160000000 in the CDAF graph 300. The numerical values for the FV 315 can express a contrast value, a sharpness value, or some combination thereof. In one example, the numerical values for the FV 315 can express a sum of sharpness values for all valid pixels in a region of interest (ROI) of the corresponding image frame. The sharpness and/or contrast values for these pixels in the ROI may be obtained from the image sensor 130, from a hardware filter, from an image processor 150, or some combination thereof. The hardware filter may include a high-pass filter, a low-pass filter, a band-stop filter, or some combination thereof. The FV can be determined in the Bayer color domain, the RGB color domain, the YUV domain, or some combination thereof. While the maximum FV 315 in the CDAF graph 300 is 160000000, this is just an example. Different images of different scenes may have different focus values Thus, the max focus values as graphed can vary for different scenes. For example, the max FV for an image of a finely-detailed scene can be larger than for an image of a less-detailed scene.

A lens of an image capture device 105A is moved through a range of positions between far-focus lens position value 899 and near-focus lens position value 419 during a CDAF procedure. In one example, the lens is moved from far-focus lens position value 899 to near-focus lens position value 419, represented from left to right along the horizontal axis representing lens position 310. In some examples, the movements are stepwise, in that the lens is moved from a first position to a second position, is stationary for a period, and then is moved from the second position to a third position, and so forth. Image frames are captured while the lens is at each of the lens positions, and the focus value (FV) 315 is determined based on these image frames and graphed as the curve 305 by determining a contrast between a subject and a background in the image frame. In particular, the curve 305 includes three FV values at each lens position at which the lens was stopped, indicating that three image frames were captured at each lens position at which the lens was stopped. Due to the stepwise movements of the lens, the curve 305 visually includes steps. The shape of the curve 305 is approximately that of a normal distribution, also referred to as a bell curve, with steps. The maximum focus value (FV) in the curve 305 is approximately 150000000, and occurs approximately at lens position 670. Thus, once the CDAF process is complete, the image capture device 105A actuates its motor to return the lens to lens position 670, and captures and stores an in-focus image while the lens is at lens position 670.

As noted above, in the CDAF process, the lens is moved from a first position to a second position, and stops (is stationary) at the second position for a period of time. The lens is moved from the second position to a third position. In some cases, the image sensor 130 captures an image frame from which to determine a FV 315 while the lens position is stationary at the second position. However, the motion of the lens and the capture of image data by the image sensor 130 are not always perfectly synchronized. In some cases, the image frame is captured while the lens is in motion, for example while the lens is still finishing the motion from the first position to the second position, or while the lens has just started the motion from the second positon to the third position. An image frame captured while the lens is in motion can produce inaccurate focus values. Inaccurate focus values due to capture of image frames while the lens is in motion are visible in the curve 305 at artificial peak 320 and artificial peak 325, which appear as small peaks at the edge of a step. For instance, artificial peak 320 may occur due to the lens still moving from a position N ₁ to a position N ₂ during capture of the earliest image frame corresponding to the position N ₂. Artificial peak 325 may similarly occur due to the lens still moving from a position N ₃ to a position N ₄ during capture of the earliest image frame corresponding to the position N ₄. For instance, the actual lens position during capture of that earliest image frame may still be closer to lens position N ₃ rather than the target les position N ₄, resulting in artificial peak 325. Inaccurate focus values due to capture of image frames while the lens is in motion may also take the form of artificial dips, such as artificial dip 330, which appears as a small dip at the edge of a step rather than a small peak at the edge of a step. Artificial dip 330 may occur due to the lens still moving from a position N ₅ to a position N ₆ during capture of the earliest image frame corresponding to the position N ₆.

An image frame can be captured while the lens is in motion for a number of reasons. For instance, in some cases, the motor that moves the lens may be a VCM, which can in some cases suffer from hysteresis behavior that causes movement of the motor to continue for a short time after the current application actuating the movement stops. The hysteresis behavior means that, in some cases, a time at which current application actuating the movement stops is not an accurate and precise indication of when lens movement stops. The hysteresis behavior may, in some cases, cause a lens moving from a starting position to a target position to overshoot past the target position briefly before settling back to the target position. This can cause the artificial peaks and/or artificial dips discussed above. Additionally, in some cases, the image capture device 105A, image processing device 105B, and/or motor drivers divide a particular lens movement into multiple smaller movements. This division of a larger movement into multiple smaller movements can improve lens movement accuracy and precision, but can also cause the larger lens movement to be performed more slowly than it might otherwise be performed. This, too, can cause artificial peaks and/or artificial dips discussed above. In some cases, image capture device 105A and/or image processing device 105B can be overworked and/or underpowered, causing image processing and/or autofocus calculations (e.g., CDAF calculations, PDAF calculations, HAF calculations, lens movement calculations, or some combination thereof) to be performed more slowly, which in turn delays lens movement. Such delays in lens movement can also cause the artificial peaks and/or artificial dips discussed above. The

operations

700, 800, 900, and/or 1000 may provide technical improvements to image capture devices 105A and/or image processing devices 105B by compensating for effects (e.g., artificial peaks and/or artificial dips) of capture of image frames while a lens is still in motion caused by at least any combination of the above reasons.

FIG. 4 is a contrast detection autofocus (CDAF) table 400 illustrating the effects of lens movements on a focus value associated with image contrast. The CDAF table 400 includes a timestamp 405 column, a frame index 410 column, a lens position 415 column, and a focus value 420 column. The CDAF table 400 identifies, in the focus value 420 column, focus values determined from image frames captured by the image sensor. The focus value for each image frame is determined based on a contrast between a subject and a background in the image frame. The CDAF table 400 identifies, in the lens position 415 column, a reported lens position corresponding to each focus value in the CDAF table 400. The CDAF table 400 identifies, in the frame index 410 column, a frame index corresponding to the image frame from which each focus value is determined in the CDAF table 400. The CDAF table 400 identifies, in the timestamp 405 column, a timestamp corresponding to capture of the image frame from which each focus value is determined in the CDAF table 400.

Each reported lens position in the lens position 415 column may refer to a target lens position that the lens is expected to be at during capture of an image frame from which the corresponding focus value is determined (or within a capture time window within which the image frame is captured) , assuming that all lens movements are completed prior to completion of capture of the image frame. Each reported lens position in the lens position 415 column may alternately refer to a lens position that is determined based on a lens position measurement measured by a lens position sensor during capture of an image frame (or within a capture time window within which the image frame is captured) from which the corresponding focus value is determined. This may be referred to as a measured lens position, or a measured position. In some cases, the reported lens position may be a combination of the target lens position and the measured lens position, such as an average of the target lens position and the measured lens position.

The CDAF table 400 includes three focus values at each of five reported lens positions, including

lens positions

727, 723, 719, 715, and 711. The earliest instance of each reported lens position, and the corresponding focus value, are both highlighted with black-outlined rectangles in the CDAF table 400. The highlighting emphasizes that the earliest focus value at each reported lens position is, in several cases, artificially higher (artificial peak) or artificially lower (artificial dip) than the latter two focus values at each reported lens position. For instance, the earliest focus value at lens position 727 is 114405616, which is higher than the latter two focus values at lens position 727 (117864832 and 117551160) . The earliest focus value at lens position 719 is 122937568, which is lower than the latter two focus values at lens position 719 (125848400 and 125501712) . Such artificial peaks and artificial dips may result from the lens still being in motion during capture of the earliest of the three image frames corresponding to a particular reported lens position.

As shown, the CDAF table 400 includes three consecutive focus values determined based on three separate image frames captured at each reported lens position. In some cases, less than three focus values (e.g., 1 or 2) are determined based on less than three separate image frames (e.g., 1 or 2) consecutively captured at each reported lens position during CDAF. A user experience is generally improved when autofocus occurs as quickly as possible so that the user captures a desired image before the photographed scene changes. To speed up CDAF, some image capture devices (e.g., image capture device 105A) only capture one or two image frames per reported lens position, and therefore only determine one or two focus values per reported lens position. While capturing one or two image frames per reported lens position may be faster than capturing three image frames per reported lens position, it may result in a less accurate focus value determination. An inaccurate focus value determination may ultimately result in an inaccurate focus setting as a result of a CDAF autofocus process. For instance, the image capture device 105A may interpret an artificial peak to be a false maximum focus value in a CDAF focus value curve, causing the image capture device 105A to move the lens to a position corresponding to the false maximum focus value that does not actually provide optimal focus. Similarly, the image capture device 105A may fail to identify a lens position that corresponds to an actual maximum focus value if the image capture device 105A only determines a single focus value for that lens position and that focus value is artificially lowered due to an artificial dip. If more additional image frames were captured at the lens position corresponding to the actual maximum focus value, then the image capture device 105A may be able to identify the actual maximum focus value despite the artificial dip in the earliest focus value corresponding to that lens position.

Techniques can be performed to avoid potential inaccuracies due to image frame capture while the lens is still in motion. For example, one technique is to skip or ignore the focus value for the earliest image frame captured corresponding to a particular reported lens position for CDAF, and to instead use one or more subsequent image frames captured at the reported lens position for CDAF. However, always skipping the earliest image frame captured corresponding to a particular reported lens position for CDAF means that two or more image frames need to be captured at each reported lens position, which can result in slow CDAF performance and therefore a negative user experience. While the earliest image frame captured corresponding to a particular reported lens position is sometimes inaccurate due to the lens still being in motion during capture of the image frame, this is not always the case. Thus, not all such frames should be skipped, and skipping some such frames provides no improvement to CDAF accuracy.

Optimizing CDAF to perform with high speed and accuracy involves capturing as few image frames as feasible at each reported lens position, and skipping only those image frames that an image capture device 105A determines to have been captured while the lens is still in motion. Such optimizations may be achieved through operations illustrated in, and discussed with respect to, FIGs. 7-10.

FIG. 5 is a phase detection autofocus (PDAF) graph 500 graph 500 illustrating the effects of lens movements on a phase difference value. The PDAF graph 500 includes a curve 505 plotted on a plane. The horizontal axis of the plane represents lens positions 510, which range from 899 to 411 in the PDAF graph 500. On the right-hand side of the horizontal axis, lens position value 411 represents an example of the near-focus lens position 240 from FIG. 2. On the left-hand side of the horizontal axis, lens position value 899 represents an example of the far-focus lens position 230 from FIG. 2. The vertical axis of the plane represents phase differences (PD) 515, which range from approximately -8 to 10, with the arrow representing the horizontal axis illustrated at a phase difference of 0. Phase difference may be identified in a distance of shift between PD pixels corresponding to opposing directions, such as a PD left pixel and a PD right pixel, or a PD top pixel and a PD bottom pixel. This distance may be expressed in terms of pixels. For instance, the distance may be the width of a pixel, or the width of a pixel multiplied by a multiplier. The multiplier may be negative or positive, and have an absolute value such as 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, a value greater than 5, a value between any two of the previously-recited values. The distance may be expressed in terms of millimeters (mm) , micrometers (μm) , or nanometers (nm) . For instance, if the size of a pixel on the image sensor 130 is 1.0 μm by 1.0 μm, and the distance corresponding to the phase difference is the width of a pixel, then the distance is 1.0 μm. In some cases, the phrase difference may alternately be expressed in units of radians or degrees.

A lens of an image capture device 105A is moved through a range of positions between 899 and 411. In one example, the lens is moved from the far-focus lens position value 899 to the near-focus lens position value 411, represented from left to right along the horizontal axis representing lens position 510. As in the CDAF graph 300, the movements in the PDAF graph 500 can be stepwise, in that the lens is moved from a first position to a second position, is stationary for a period, and then is moved from the second position to a third position, and so forth. During PDAF, data from phase detection photodiodes, or phase detection pixels, of an image sensor 130 are used to determine a phase difference between light hitting the image sensor 130 from different angles. PD frames –that is, image frames with PD photodiode data –are captured by the image sensor 130 while the lens is at each of the lens positions, and the phase difference is determined based on these PD frames and graphed as the curve 505. In particular, the curve 505 includes three PD values at each reported lens position at which the lens was stopped, indicating that three PD frames were captured at each reported lens position at which the lens was stopped. Due to the stepwise movements of the lens, the curve 505 visually includes steps. The shape of the curve 505 is approximately linear, with steps. PD is zero at approximately lens position 670. Thus, once the PDAF process is complete, the image capture device 105A actuates its motor to return the lens to lens position 670, and captures and stores an in-focus image while the lens is at lens position 670.

Like the curve 305 of the CDAF graph 300 includes some inaccuracies in focus value 315, the curve 505 of the PDAF graph 500 includes some inaccuracies in phase difference 515, such as artificial peaks and artificial dips. For example, the curve 505 of the PDAF graph 500 includes an artificial peak 520 in phase difference 515 at approximately lens position 619 and an artificial peak 525 in phase difference 515 at approximately lens position 773. These inaccuracies in phase difference 515 may be phase differences determined from PD frames captured while the lens is still in motion. For instance, artificial peak 520 may occur due to the lens still moving from a position N ₇ to a position N ₈ during capture of the earliest PD frame corresponding to the position N ₈. Artificial peak 525 may occur due to the lens still moving from a position N ₉ to a position N ₁₀ during capture of the earliest PD frame corresponding to the position N ₁₀.

Some image sensors (e.g., image sensor 130) that include PD photodiodes are able to capture an image frame and a PD frame simultaneously, for instance by using a pixel interpolation algorithm to correct any pixel data gaps in an image frame and/or pixel data inaccuracies in an image frame that might be caused by PD photodiodes and/or PD pixels.

FIG. 6 is a phase detection autofocus (PDAF) table 600 illustrating the effects of lens movements on a phase difference value. The PDAF table 600 includes a timestamp 605 column, a frame index 610 column, a lens position 615 column, and a phase difference 620 column. The PDAF table 600 identifies, in the phase difference 620 column, phase differences determined from PD frames captured by the image sensor. The phase difference for each PD frame is determined based on a difference between a first phase of light received by one or more PD photodiodes from a first angle and a second phase of light received by one or more PD photodiodes from a second angle. The PDAF table 600 identifies, in the lens position 615 column, a reported lens position corresponding to each phase difference in the PDAF table 600. The PDAF table 600 identifies, in the frame index 610 column, a frame index corresponding to the PD frame from which each phase difference is determined in the PDAF table 600. The PDAF table 600 identifies, in the timestamp 605 column, a timestamp corresponding to capture of the PD frame from which each phase difference is determined in the PDAF table 600.

Each reported lens position in the lens position 615 column may refer to a target lens position that the lens is expected to be at during capture of a PD frame (or within a capture time window within which the PD frame is captured) from which the corresponding phase difference is determined, assuming that all lens movements are completed prior to completion of capture of the PD frame. Each reported lens position in the lens position 615 column may alternately refer to a lens position that is determined based on a lens position measurement measured by a lens position sensor during capture of a PD frame (or within a capture time window within which the PD frame is captured) from which the corresponding phase difference is determined. In some cases, the reported lens position may be a combination of the target lens position and the measured lens position, such as an average of the target lens position and the measured lens position.

The PDAF table 600 includes three phase differences at each of five reported lens positions, including

lens positions

751, 747, 743, 739, and 735. The earliest instance of each reported lens position, and the corresponding phase difference, are both highlighted with black-outlined rectangles in the PDAF table 600. The highlighting emphasizes that the earliest phase difference at each reported lens position is, in several cases, artificially higher (artificial peak) or artificially lower (artificial dip) than the latter two phase differences at each reported lens position. For instance, the earliest phase difference at lens position 747 is -5.02, which is lower than the latter two phase differences at lens position 747 (-4.71 and -4.74) . Similarly, the earliest phase difference at lens position 735 is -4.07, which is lower than the latter two phase differences at lens position 735 (-3.92 and -3.96) . Such artificial dips may result from the lens still being in motion during capture of the earliest of the three PD frames corresponding to a particular reported lens position. Artificial peaks may also result from these circumstances.

As shown, the PDAF table 600 includes three consecutive phase differences determined based on three separate PD frames captured at each reported lens position. In some cases, less than three focus values (e.g., 1 or 2) are determined based on less than three separate image frames (e.g., 1 or 2) consecutively captured at each reported lens position during PDAF. A user experience is generally improved when autofocus occurs as quickly as possible so that the user captures a desired image before the photographed scene changes. To speed up PDAF, some image capture devices (e.g., image capture device 105A) only capture one or two PD frames per reported lens position, and therefore only determine one or two phase differences per reported lens position. While capturing one or two PD frames per reported lens position may be faster than capturing three PD frames per reported lens position, it may result in a less accurate phase difference determination. An inaccurate phase difference determination may ultimately result in an inaccurate focus setting as a result of a PDAF autofocus process. For instance, the image capture device 105A may incorrectly identify a lens position as corresponding to a PD of zero due to an artificial peak, when the true PD at that lens position is actually a negative value that is less than zero. Similarly, the image capture device 105A may incorrectly identify a lens position as corresponding to a PD of zero due to an artificial dip, when the true PD at that lens position is actually a positive value that is greater than zero. The image capture device 105A may also fail to identify a lens position corresponding to an actual PD of zero due to an artificial peak or artificial dip that affects the measured PD by increasing the measured PD above zero or decreasing the measured PD below zero.

Techniques can be performed to avoid potential inaccuracies due to PD frame capture while the lens is still in motion. For example, one solution is to skip or ignore the phase difference for the earliest PD frame captured corresponding to a particular reported lens position for PDAF, and to instead use one or more subsequent PD frames captured at the reported lens position for PDAF. However, always skipping the earliest PD frame captured corresponding to a particular reported lens position for PDAF means that two or more PD frames need to be captured at each reported lens position, which can result in slow PDAF performance and therefore a negative user experience. While the earliest PD frame captured corresponding to a particular reported lens position is sometimes inaccurate due to the lens still being in motion during capture of the PD frame, this is not always the case. Thus, not all such frames should be skipped, and skipping some such frames provides no improvement to PDAF accuracy.

Optimizing PDAF to perform with high speed and accuracy involves capturing as few PD frames as feasible at each reported lens position, and skipping only those PD frames that an image capture device 105A determines to have been captured while the lens is still in motion. Such optimizations may be achieved through operations illustrated in, and discussed with respect to, FIGs. 7-10.

While the discussion of FIGs. 3-6 identifies certain determinations and actions undertaken by the image capture device 105A, it should be understood that these determinations and actions may be undertaken by an image capture device 105A, an image processing device 105B, an image capture and processing system 100, an image capture device 205, an image capture and processing device 700, a computing device 1100, or some combination thereof.

FIG. 7 is a block diagram illustrating an image capture and processing device 700 performing a hybrid autofocus operation accounting for lens position. The image capture and processing device 700 includes an image capture device 705 and an image processor 770. The image capture device 705 may be an example of an image capture device 105A, an image capture device 205, or some combination thereof. The image processor 770 may be an example of an image processor 150, an ISP 154, a host processor 152, a DSP, an image processing device 105B, a computing device 1100, or some combination thereof.

The image capture device 705 provides capture data 725 to the image processor 770. The image capture device 705 includes an image sensor 710, a lens motor 715, and a lens position sensor 720. The image sensor 710 may be an example of an image sensor 130. The capture data 725 may include image data from one or more image frames captured by the image sensor 710 and/or PD data from one or more PD frames captured by the image sensor 710. The lens motor 715 may be an example of one of the one or more focus control mechanisms 125B. The lens position sensor 720 may determine a position of the lens, for instance by determining a state of the motor 715. In some cases, the lens motor 715 may be a voice coil motor (VCM) . In some cases, the lens position sensor 720 may be a Hall effect sensor that measures a Hall measurement measuring a magnetic field associated with the lens motor 715. A Hall effect sensor may also be referred to as a Hall sensor, a magnetic field sensor, a magnetism sensor, a magnetic sensor, a magnet sensor, or some combination thereof.

The lens position measurement feedback may include a measurement that may be used to identify a position of the motor (and thus the lens) and/or that may be used to used to identify whether the motor (and thus the lens) is stationary or in motion at any given time. The capture data 725 may include lens position measurement feedback identifying the lens position measurement measured by the lens position sensor 720, a lens position of the lens at a particular time determined based on the lens position measurement, a determination as to whether the lens is stationary or in motion at a particular time determined based on the lens position measurement, or some combination thereof. Where the lens position sensor 720 includes a Hall effect sensor, the lens position measurement feedback may include Hall feedback identifying the Hall measurement measured by the Hall effect sensor and/or information determined based on the Hall feedback. The capture data 725 may include lens position feedback corresponding to a lens position during capture of the image data and/or the PD data (or within a capture time window within which the image data and/or the PD data is captured) included in the capture data 725.

The capture time window may include the time of capture of the image data and/or the PD data and, in some cases, a window of time around the time of capture, such as a threshold time before and/or after the time of capture. The threshold time may include, for example, 1 nanosecond (ns) , 2 ns, 3 ns, 4 ns, 5 ns, 6 ns, 7 ns, 8 ns, 9 ns, 10 ns, a time greater than 10 ns a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 microsecond (μs) , 2 μs, 3 μs, 4 μs, 5 μs, 6 μs, 7 μs, 8 μs, 9 μs, 10 μs, a time greater than 10 μs a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 millisecond (ms) , 2 ms, 3 ms, 4 ms, 5 ms, 6 ms, 7 ms, 8 ms, 9 ms, 10 ms, a time greater than 10 ms a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 centisecond (cs) , 2 cs, 3 cs, 4 cs, 5 cs, 6 cs, 7 cs, 8 cs, 9 cs, 10 cs, a time greater than 10 cs a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 decisecond (ds) , 2 ds, 3 ds, 4 ds, 5 ds, 6 ds, 7 ds, 8 ds, 9 ds, 10 ds, a time greater than 10 ds a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 second (s) , 2 s, 3 s, 4 s, 5 s, 6 s, 7 s, 8 s, 9 s, 10 s, a time greater than 10 s a time in between any previously-mentioned time, or some combination thereof.

The image processor 770 receives the capture data 725 from the image capture device 705 and performs a packing operation 730. In the packing operation 730, the image processor 770 packs the image data and PD data together with corresponding lens position feedback corresponding to a position of the lens of the image capture device 705 during capture of the image data and/or the PD data (or within a capture time window within which the image data and/or the PD data is captured) . Packing these sets of data together in the packing operation 730 may involve combining these sets of data within a container file, archive file, wrapper file, interchange file, or some combination thereof. Packing these sets of data together in the packing operation 730 may involve multiplexing and/or interleaving these sets of data together. In some cases, the lens position feedback from the capture data 725 may be processed and/or change form during the packing operation 730. For instance, while the capture data 725 may include raw lens position sensor measurement data measured by the lens position sensor 720, the image processor 770 may, through the packing operation 730, convert the raw lens position sensor measurement data into an actual lens position. In some cases, the raw lens position sensor measurement data may also be used to determine whether or not the motor (and therefore the lens) is still moving in addition to determining lens positions. The operations 800 of FIG. 8 described below are example operations that can be included in the packing operation 730 of FIG. 7.

An example result of the packing operation 730 is illustrated as the packed datasets 735. The packed datasets 735 include a packed dataset with an image frame N packed together with corresponding lens position feedback, a packed dataset with an image frame N+1 packed together with corresponding lens position feedback, and a packed dataset with an image frame N+2 packed together with corresponding lens position feedback. The packed datasets 735 include a packed dataset with a PD frame N packed together with corresponding lens position feedback, a packed dataset with a PD frame N+1 packed together with corresponding lens position feedback, and a packed dataset with a PD frame N+2 packed together with corresponding lens position feedback. In some cases, a packed dataset with the image frame N may also include the PD frame N as well as the corresponding lens position feedback. In this way, the lens position feedback corresponding to the image frame N and the PD frame N does not need to be stored in duplicate in the packed datasets 735 as illustrated in FIG. 7. Similarly, a packed dataset with the image frame N+1 may also include the PD frame N+1 as well as the corresponding lens position feedback, and a packed dataset with the image frame N+2 may also include the PD frame N+2 as well as the corresponding lens position feedback.

In some cases, each image frame may include the corresponding PD frame, depending on how the image data and PD data are received from the image sensor. Thus, the image frame N may include the PD frame N, the image frame N+1 may include the PD frame N+1, the image frame N+2 may include the PD frame N+2, and so forth.

The image processor 770 performs an unpacking operation 740 on one or more of the packed datasets 735 at a time to unpack the lens position feedback from the image data and the PD data. Unpacking these sets of data in the unpacking operation 740 may involve extracting these sets of data from a container file, archive file, wrapper file, interchange file, or some combination thereof. Unpacking these sets of data in the unpacking operation 740 may involve demultiplexing and/or de-interleaving these sets of data from a multiplexed or interleaved data stream. In some cases, unpacking these sets of data may include identifying pointers pointing to a memory buffer in which the packed data sets are stored, and then accessing and parsing these packed data sets. The unpacked image data and corresponding lens position feedback is sent to a CDAF controller 750 of a hybrid autofocus (HAF) controller 745. The operations 900 of FIG. 9 are example operations that can be included in the unpacking operation 740 of FIG. 7. The unpacked PD data and corresponding lens position feedback is sent to a PDAF controller 755 of the HAF controller 745. The HAF controller 745 performs a lens movement control determination 760 based on the CDAF controller 750 and/or PDAF controller 755. During the lens movement control determination 760, the HAF controller 745 may identify, for example, whether the lens was stationary or in motion during of capture of the image data and/or PD data received by the CDAF controller 750 and/or the PDAF controller 755 (or within a capture time window within which the image data and/or the PD data is captured) . The HAF controller 745 may identify that the lens is ready to be moved to the next position if the lens was stationary during of capture of the image data and/or PD data received by the CDAF controller 750 and/or the PDAF controller 755 (or within a capture time window within which the image data and/or the PD data is captured) , suggesting that the image data and/or PD data are accurate. The HAF controller 745 may identify that the lens is not ready to be moved to the next position if the lens was in motion during of capture of the image data and/or PD data received by the CDAF controller 750 and/or the PDAF controller 755 (or within a capture time window within which the image data and/or the PD data is captured) , suggesting that the image data and/or PD data are inaccurate. The HAF controller 745 may send a lens movement command 765 to the image capture device 705 that identifies whether the lens should be moved. The lens movement command 765 is based on image and/or PD data and/or lens position feedback. As discussed above, the capture time window may include the time of capture of the image data and/or the PD data and, in some cases, a window of time around the time of capture, such as a threshold time before and/or after the time of capture.

In some cases, the hybrid autofocus (HAF) controller 745 may identify a distance and/or direction to move the lens during the lens movement control determination 760, and may include this information in the lens movement command 765. For instance, if the CDAF controller 750 identifies a lens position corresponding to a maximum focus value such as the peak of the curve 305 at lens position 670 of FIG. 3, the lens movement command 765 may command that the lens be moved to that lens position corresponding to the maximum focus value. In some cases, the CDAF controller 750 can identify a lens position corresponding to a maximum focus value, such as the peak of the curve 305 at lens position 670 of FIG. 3, in which case the lens movement command 765 may command that the lens be moved to that lens position corresponding to the maximum focus value. In some cases, the PDAF controller 755 can identify a direction of lens movement based on a sign (negative or positive) of a phase difference, and a distance of the lens movement based on a magnitude (absolute value) of the phase difference.

Some variants of the image capture and processing device 700 only perform CDAF, and do not perform PDAF at all. In such variants of the image capture and processing device 700, the hybrid autofocus (HAF) controller 745 of FIG. 7 may be replaced by the CDAF controller 750. Other variants of the image capture and processing device 700 only perform PDAF, and do not perform CDAF at all. In such variants of the image capture and processing device 700, the hybrid autofocus (HAF) controller 745 of FIG. 7 may be replaced by the PDAF controller 755.

The lens motor 715 may be a closed-loop VCM or an open-loop VCM. An open-loop VCM may in some cases lack the lens position sensor 720. A variant of the image capture and processing device 700 in which the lens motor 715 is an open-loop VCM and/or in which the lens position sensor 720 is missing would lack the packing operation 730 and the unpacking operation 740 discussed further herein with respect to the image processor 770. The presence of a lens position sensor 720 provides a number of technical benefits. For instance, the presence of a lens position sensor 720 allows an image capture and processing device 700 to confirm that a focus value and phase difference are accurate by determining that the image frame and PD frame from which the focus value and phase difference are determined were captured while the lens was stationary. The presence of a lens position sensor 720 allows an image capture and processing device 700 to confirm that a focus value and phase difference are inaccurate by determining that the image frame and PD frame from which the focus value and phase difference are determined were captured while the lens was in motion. These technical benefits are explored further in the example of the packing operation 730 in the operations 800 of FIG. 8 described below and in the example of the unpacking operation 740 in the operations 900 of FIG. 9 described below.

FIG. 8 is a flow diagram illustrating operations 800 for packing lens position data with image data. The operations 800 may be performed as part of the packing operation 730 of FIG. 7. The operations 800 are described below as being performed by a device. The device may be, or may include, an image capture device 105A, an image processing device 105B, an image capture and processing system 100, an image capture device 205, an image capture and processing device 700, an image processor 150, an image processor 770, an ISP 154, a host processor 152, a DSP, an image processing device 105B, a computing device 1100, or some combination thereof.

Operation 805 includes receiving capture data 725 and parsing the capture data 725. The capture data 725 may include image frame data that includes one or more image frames captured by the image sensor 710. The capture data 725 may include PD frame data that includes one or more PD frames captured by the image sensor 710. The capture data 725 may include lens position sensor feedback that identifies measurements measured by the lens position sensor 720 during capture of each of the image frames and/or PD frames in the capture data 725 (or within a capture time window within which the image frames and/or the PD frames are captured) . The lens position sensor feedback may be parsed at operation 805 or before operation 805 to identify one or more positions of the lens. Each of the lens positions may, for example, be expressed as a focal length.

Operation 810 includes checking whether a lens position sensor 720 is present within the image capture and processing device 700. If, at operation 810, the device determines that the lens position sensor 720 is present within the image capture and processing device 700, operation 815 sets a closeloop indicator to 1 (true) and performs operation 825.

Operation 825 includes determining a lens movement status corresponding to a particular image frame and/or PD frame. Operation 830 includes determining whether or not a lens movement is complete during capture of the image frame and/or PD frame (or within a capture time window within which the image frame and/or the PD frame are captured) . Operation 830 may work by determining, based on a lens position sensor measurement captured by the lens position sensor 720 and received as part of the lens position sensor feedback of the capture data 725, whether the lens motor 715 (and therefore the lens) is stationary or in motion. The lens motor 715 and/or lens being stationary indicates that the lens movement is complete. The lens motor 715 and/or lens being in motion indicates that the lens movement is not yet complete.

If, at operation 830, the device determines that the lens movement is complete, then the device performs operation 835 by setting a lens position variable corresponding to the particular image frame and/or PD frame to the current lens position as determined based on the lens position sensor measurement captured by the lens position sensor 720. At operation 850, this lens position variable is then packed together with the image frame data and/or the PD frame data corresponding to the particular image frame and/or PD frame into a packed dataset.

If, at operation 830, the device determines that the lens movement is not complete, then the device performs operation 840 by setting a lens position variable corresponding to the particular image frame and/or PD frame to a previous lens position. At operation 850, this lens position variable is then packed together with the image frame data and/or the PD frame data corresponding to the particular image frame and/or PD frame into a packed dataset. The previous lens position may be determined based on a previous lens position sensor measurement captured by the lens position sensor 720 during capture by the image sensor 710 of the previous image frame and/or previous PD frame (or within a previous capture time window within which the previous image frame and/or previous PD frame are captured) just prior to the current image frame and/or current PD frame. The image sensor 710 thus captures the current image frame and/or current PD frame consecutively after the capture of the previous image frame and/or previous PD frame by the image sensor 710. The term “previous position value” may refer to the previous lens position sensor measurement and/or a previous lens position determined based on the previous lens position sensor measurement.

If, at operation 810, the device determines that the lens position sensor 720 is not present within the image capture and processing device 700, operation 820 sets a closeloop indicator to 0 (false) and performs operation 850. Thus, if no lens position sensor 720 is present, lens movement is assumed to be complete and have reached a target lens position that the lens was intended to reach via the lens movement. At operation 850, the target lens position is then packed together with the image frame data and/or the PD frame data corresponding to the particular image frame and/or PD frame into a packed dataset. After operation 850, the operations 800 may repeat at operation 805 with receipt of the next capture data 725 corresponding to the next image frame and/or PD frame.

Operation 850 may include packing the lens position with the image frame data and/or the PD frame data by combining these sets of data within a container file, archive file, wrapper file, interchange file, or some combination thereof. Operation 850 may include packing the lens position with the image frame data and/or the PD frame data by multiplexing and/or interleaving these sets of data together.

FIG. 9 is a flow diagram illustrating operations 900 for unpacking lens position data from image data. The operations 900 may be performed as part of the unpacking operation 740 of FIG. 7. The operations 900 are described below as being performed by a device. The device may be, or may include, an image capture device 105A, an image processing device 105B, an image capture and processing system 100, an image capture device 205, an image capture and processing device 700, an image processor 150, an image processor 770, an ISP 154, a host processor 152, a DSP, an image processing device 105B, a computing device 1100, or some combination thereof.

Operation 905 includes receiving a packed dataset corresponding to a particular image frame and/or PD frame, and parsing the packed data. The packed dataset may be the packed dataset output as during the packing operation 730 of FIG. 3, the packed dataset output during operation 850 of the operations 800 of FIG. 8, or some combination thereof. The packed dataset may include image frame data that includes an image frame captured by the image sensor 710. The packed dataset may include PD frame data that includes a PD frame captured by the image sensor 710. For instance, the packed dataset may include the image frame N and/or the PD frame N of the packed datasets 735 of FIG. 7. The packed dataset may include lens position information identifying a lens position of the lens during capture of the image frame and/or PD frame (or within a capture time window within which the image frame and/or PD frame are captured) . The lens position may be a target lens position, a measured lens position, or some combination thereof. The lens position may be expressed as a focal length.

Parsing the packed dataset in operation 905 may include extracting the lens position, the image data, and/or the PD data from a container file, an archive file, a wrapper file, an interchange file, or some combination thereof. Parsing the packed dataset in operation 905 may include demultiplexing and/or de-interleaving the lens position, the image data, and/or the PD data from a multiplexed or interleaved data stream received in operation 905.

Operation 910 includes determining whether a lens position sensor 720 is present within the image capture and processing device 700. If, at operation 910, the device determines that the lens position sensor 720 is present within the image capture and processing device 700, the device at operation 912 sets a closeloop indicator to 1 (true) and the device performs operation 920.

Operation 920 includes determining whether the measured lens position determined based on the lens position sensor 720 is equal to the lens’s target position that the lens is expected to be at. Determining that the measured lens position is equal to the lens’s target position may include determining that the lens motor 715, and therefore the lens, is stationary. If, at operation 920, the device determines that the measured lens position is equal to the lens’s target position, then the device performs operation 930. Operation 930 includes determining the focus value based on the image frame in the packed dataset and/or determining the phase difference based on the PD frame in the packed dataset. Operation 935 includes sending out a next movement command 765 to actuate the lens motor 715 and move the lens into a next position, after which the operations 900 may repeat at operation 905 with receipt of the next packed dataset corresponding to the next image frame and/or PD frame. Operation 935 may occur automatically in response to operation 930.

If, at operation 920, the device determines that the measured lens position is not equal to the lens’s target position, then the device performs operation 925. Determining that the measured lens position is not equal to the lens’s target position may include determining that the measured lens position falls within a range of positions between a prior lens position and the target position, the range of positions excluding the prior lens position and the target position. Determining that the measured lens position is not equal to the lens’s target position may include determining that the lens motor 715, and therefore the lens, is still in motion.

Operation 925 skips use of any focus value and/or phase difference associated with the current image frame and/or PD frame and gives the lens time to reach the target position. Operation 925 may include determining, based on the measured lens position being unequal to the lens’s target position, that the lens was still in motion during capture of the image frame and/or PD frame (or within a capture time window within which the image frame and/or PD frame are captured) . Operation 925 may include determining focus value and/or phase difference as in operation 930, then ignoring or discarding the determined focus value and/or phase difference. Operation 925 may include skipping the determination of focus value and/or phase difference altogether, thereby conserving computing resources.

If, at operation 910, the device determines that the lens position sensor 720 is not present within the image capture and processing device 700, the device operation 915 sets a closeloop indicator to 0 (false) and the device performs operation 930. If the lens position sensor 720 is not present, then, the device effectively assumes that the actual lens position is equal to the lens’s target position.

In some cases, the closeloop indicator may be omitted, and thus setting the closeloop indicator as in

operations

815, 820, 912, and 915 of FIGs. 8-9 may be omitted. To this effect, a “YES” determination at operation 810 of FIG. 8 may be followed by operation 825. A “NO” determination at operation 810 of FIG. 8 may be followed by operation 850. A “YES” determination at operation 910 of FIG. 9 may be followed by operation 920. A “NO” determination at operation 910 of FIG. 9 may be followed by operation 930.

FIG. 10 is a flow diagram illustrating operations 1000 for automatic focus control. The operations 900 are described below as being performed by a device. The device may be, or may include, an image capture device 105A, an image processing device 105B, an image capture and processing system 100, an image capture device 205, an image capture and processing device 700, an image processor 150, an image processor 770, an ISP 154, a host processor 152, a DSP, an image processing device 105B, a computing device 1100, or some combination thereof.

Operation 1005 includes actuating a motor to move a lens from a first position to a second position. The device may include a motor connector that is coupled to the device and to the motor. The device may actuate the motor by sending a motor actuation signal to the motor through the motor connector. In some aspects, the device may include the motor.

The motor can be, or include, a voice coil motor (VCM) and/or a voice coil actuator (VCA) . The motor can be, or include, a linear VCM. The motor can be, or include, a closed-loop VCM and/or an open-loop VCM. The motor can be, or include, an ultrasonic motor, a micromotor, a piezoelectric motor, a linear actuator, a stepper motor, a direct current (DC) motor, an alternating current (AC) motor, a brushless motor, or a combination of any of the above-recited types of motors. The motor may include a magnetic (or ferromagnetic) housing and a coil, one or more valves, dampers, hydraulic cylinders, pneumatic cylinders, screws, or some combination thereof.

Operation 1010 includes receiving a first image frame captured by an image sensor. The device may include an image sensor connector that is coupled to the device and to the image sensor. The device may receive the first image frame captured by the image sensor through the image sensor connector. In some aspects, the device may include the image sensor. The first image frame may include a first PD frame corresponding to the first image frame. The first image frame may include PD photodiode and/or PD pixel data corresponding to the first image frame. In some cases, the first image frame, in the context of the operations 1000, may refer to the first PD frame instead of referring to a frame with non-PD image data. In some cases, the first image frame, in the context of the operations 1000, may refer to both an image frame and a corresponding PD frame, for instance packed together into a container as in the packing operations 730 and/or 800.

Operation 1015 includes receiving a measurement from a lens position sensor. The measurement is measured by the lens position sensor during a capture time window within which the image sensor captures the first image frame. In some cases, the measurement is also received during the capture time window. The device may include a lens position sensor connector that is coupled to the device and to the lens position sensor. The device may receive the measurement captured by the lens position sensor through the lens position sensor connector. In some aspects, the device may include the lens position sensor. The lens position sensor connector can, in some cases, also be the motor connector, for instance where the measurement received from the lens position sensor is a measurement of an aspect of the motor, such as power going to the motor, magnetism produced by the motor, a force produced by the motor, a Hall effect produced by the motor, or some combination thereof.

The capture time window may include the time of capture of the first image frame and, in some cases, a window of time around the time of capture, such as a threshold time before the time of capture and/or after the time of capture. The threshold time may include, for example, 1 nanosecond (ns) , 2 ns, 3 ns, 4 ns, 5 ns, 6 ns, 7 ns, 8 ns, 9 ns, 10 ns, a time greater than 10 ns a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 microsecond (μs) , 2 μs, 3 μs, 4 μs, 5 μs, 6 μs, 7 μs, 8 μs, 9 μs, 10 μs, a time greater than 10 μs a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 millisecond (ms) , 2 ms, 3 ms, 4 ms, 5 ms, 6 ms, 7 ms, 8 ms, 9 ms, 10 ms, a time greater than 10 ms a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 centisecond (cs) , 2 cs, 3 cs, 4 cs, 5 cs, 6 cs, 7 cs, 8 cs, 9 cs, 10 cs, a time greater than 10 cs a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 decisecond (ds) , 2 ds, 3 ds, 4 ds, 5 ds, 6 ds, 7 ds, 8 ds, 9 ds, 10 ds, a time greater than 10 ds a time in between any previously-mentioned time, or some combination thereof. The threshold time may include, for example, 1 second (s) , 2 s, 3 s, 4 s, 5 s, 6 s, 7 s, 8 s, 9 s, 10 s, a time greater than 10 s a time in between any previously-mentioned time, or some combination thereof.

Operation 1020 includes determining, based on the measurement, that the lens is in motion during the capture time window. Operation 1020 can correspond to the “YES” determination at operation 920 of FIG. 9. Operation 1020 can alternately or additionally correspond to the “YES” determination at operation 830 of FIG. 8.

The operations 1000 can further include determining a measured position of the lens based on the measurement. Determining that the lens is in motion during the capture time window, can, in this example, include determining that the measured position has not reached, or has not settled into, the second position during the capture time window.

Determining that the lens is in motion within the capture time window can alternately or additionally include determining, based on the measurement, that the motor is actuated during the capture time window. The measurement may be a measurement of voltage at and/or around the motor, current flowing to the motor, resistance at a motor, or some combination thereof. The lens position sensor may include a voltmeter, an ammeter, an ohmmeter, a multimeter, or some combination thereof. In some cases, the lens position sensor includes a Hall effect sensor and the measurement identifies a Hall feedback of the motor. Determining that the lens is in motion during the capture time window, in such cases, can be based on the Hall feedback of the motor.

The operations 1000 can include setting a lens position value associated with the first image frame to be equal to a previous position value in response to determining that the lens is in motion during the capture time window, as in the operation 840 of the operations 800. The previous position value is associated with a previous image frame captured by the image sensor before capture of the first image frame.

Operation 1025 includes receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor. The device may receive the second image frame captured by the image sensor through the image sensor connector. The image sensor may capture the first image frame and the second image frame consecutively, in that no other image frames are captured by the image sensor in between the time of capture of the first image frame and the time of capture of the second image frame. The second image frame may include a second PD frame corresponding to the second image frame. The second image frame may include PD photodiode and/or PD pixel data corresponding to the second image frame. In some cases, the second image frame in the context of the operations 1000 may refer to the second PD frame instead of referring to a frame with non-PD image data. In some cases, the term second image frame in the context of the operations 1000 may refer to both an image frame and a corresponding PD frame, for instance packed together into a container as in the packing operations 730 and/or 800.

Operation 1030 includes determining a focus indicator associated with the second position. The focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window. The focus indicator is not determined using the first image frame in response to determining that the lens is in motion during the capture time. The lens is more likely to have reached the second position and to be stationary during capture of the second image frame. In some cases, the operations 1000 include receiving a second measurement from the lens position sensor during a second capture window within which the image sensor captures the second image frame, and determining, based on the second measurement, that the lens is stationary during the second capture window. In such cases, the focus indicator is determined using the second image frame also in response to determining that the lens is stationary during the second capture window.

The second capture time window may include the time of capture of the second image frame and, in some cases, a window of time around the time of capture, such as a threshold time before the time of capture and/or after the time of capture. The threshold time any value discussed previously with respect to the threshold used in a capture time window.

The operations 1000 can also include actuating the motor to move the lens from the second position to a third position automatically in response to receiving the second image frame and/or in response to determining the focus indicator associated with the second position.

In some cases, the a movement of the lens from the second position to a third position may initially (at the beginning of the operations 1000) be scheduled to occur at a first scheduled time. The operations 1000 can include delaying a movement of the lens from the second position to the third position from the first scheduled time to a second scheduled time in response to determining that the lens is in motion during the capture time window. The first scheduled time is after capture of the first image frame, and in some cases before capture of the second image frame. The second scheduled time is after capture of the second image frame. The first schedule time and the second scheduled time may each be scheduled as an absolute time relative to a clock or a relative time relative to an event. The event may be capture of an image frame by the image sensor, such as capture of the first image frame by the image sensor or capture of the second image frame by the image sensor.

In some cases, the focus indicator includes a contrast value, and the operations 1000 further include performing a contrast detection autofocus (CDAF) procedure based on the focus indicator. In some cases, the focus indicator includes a phase difference, and the operations 1000 further include performing a phase detection autofocus (PDAF) procedure based on the focus indicator. In some cases, the operations 1000 include performing a hybrid autofocus (HAF) procedure that includes both a PDAF procedure and a CDAF procedure. The operations 1000 can include moving the lens to a focused lens position determined using an autofocus procedure and capturing a focused image while the lens is at the focused lens position. The operations 1000 can include determining an autofocus setting using the autofocus procedure, and capturing the focused image using the autofocus setting. The autofocus procedure may be being a CDAF procedure, a PDAF procedure, a HAF procedure, or a combination thereof.

The operations 1000 can include identifying, based on a plurality of focus indicators corresponding to a plurality of lens positions, a focused lens position corresponding to a focused focus indicator of the plurality of focus indicators. The focused focus indicator represents a high-quality focus on the photographed scene, and in some cases represents the best focus on the photographed scene of the plurality of focuses on the photographed scene represented by the plurality of focus indicators. The plurality of focus indicators includes the focus indicator determined at operation 1030. The plurality of lens positions includes at least the second lens position, and in some cases also includes the first lens position referenced in operation 1005 and/or the third lens position referenced two paragraphs prior to this paragraph. The plurality of lens positions can include every lens position at which a lens stops at as the lens moves from one end of a range of lens positions to another end of the range of lens positions during an autofocus procedure. The operations 1000 can include moving the lens to the focused lens position and capturing a focused image while the lens is at the focused lens position. The plurality of focus indicators may include one or more focus indicators of a type associated with a CDAF procedure such as a contrast value, one or more focus indicators of a type associated with a PDAF procedure such as a phase difference, or a combination thereof.

The operations 1000 can also include packing the first image frame together with a first lens position into a first container. The first lens position is determined based on the measurement from the lens position sensor. The operations 1000 can include packing the second image frame together with a second lens position into a second container. The second lens position is determined based on a second measurement from the lens position sensor that is measured during a second capture window within which the image sensor captures the second image frame. These packing operations may correspond to the packing operations 730 of FIG. 7 and/or the packing operations 800 of FIG. 8. The operations 1000 can include unpacking first information from the first container. The first information may include, for instance, the first image frame and the first lens position. Determining that the lens is in motion during the capture time window in operation 1015 can be based on the first information from the first container, such as the first lens position. In some cases, determining that the lens is in motion during the capture time window in operation 1015 can instead or additionally occur during identification of the first lens position based on the measurement and/or during packing of the first lens position into the first container. The operations 1000 can include unpacking second information from the second container. The second information may include, for instance, the second image frame and the second lens position. Determining the focus indicator associated with the second position in operation 1030 can be based on the second information from the second container. These unpacking operations may correspond to the unpacking operations 740 of FIG. 7 and/or the unpacking operations 900 of FIG. 9.

In some cases, the device comprises a camera, a mobile device (e.g., a mobile telephone or so-called “smart phone” or other mobile device) , a wearable device, a wireless communication device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device) , a personal computer, a laptop computer, a server computer, another device or set of component (s) discussed herein, or some combination thereof. In some aspects, the apparatus includes a camera or multiple cameras for capturing one or more images.

The device may include one or more displays for displaying one or more images, notifications, and/or other displayable data. For example, the device can be include a display configured to display the second image frame. The display can be configured to display an image captured by the image sensor after capture of the second image frame and/or after an autofocus procedure is performed using the focus indicator, such as the focused image discussed in the above paragraphs.

In some examples, the processes described herein (e.g., processes 800, 900, 1000, and/or other process described herein) may be performed by a computing device or apparatus. In one example, the

processes

800, 900, and/or 1000 can be performed by the image capture device 105A of FIG. 1. In another example, the

processes

800, 900, and/or 1000 can be performed by the image processing device 105B of FIG. 1. The

processes

800, 900, and/or 1000 can also be performed by the image capture and processing system 100 of FIG. 1. The

processes

800, 900, and/or 1000 can be performed by a computing device with the computing system 1100 shown in FIG. 11. The computing device can include any suitable device, such as a mobile device (e.g., a mobile phone) , a wireless communication device, a desktop computing device, a tablet computing device, a wearable device (e.g., a VR headset, an AR headset, AR glasses, a network-connected watch or smartwatch, or other wearable device) , a server computer, an autonomous vehicle or computing device of an autonomous vehicle, a robotic device, a television, and/or any other computing device with the resource capabilities to perform the processes described herein, including the

processes

800, 900, and/or 1000. In some cases, the computing device or apparatus may include various components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and/or other component (s) that are configured to carry out the steps of processes described herein. In some examples, the computing device may include a display, a network interface configured to communicate and/or receive the data, any combination thereof, and/or other component (s) . The network interface may be configured to communicate and/or receive Internet Protocol (IP) based data or other type of data.

The components of the computing device can be implemented in circuitry. For example, the components can include and/or can be implemented using electronic circuits or other electronic hardware, which can include one or more programmable electronic circuits (e.g., microprocessors, graphics processing units (GPUs) , digital signal processors (DSPs) , central processing units (CPUs) , and/or other suitable electronic circuits) , and/or can include and/or be implemented using computer software, firmware, or any combination thereof, to perform the various operations described herein.

The

processes

800, 900, and/or 1000 are illustrated as logical flow diagrams, the operation of which represents a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.

Additionally, the

processes

800, 900, 1000, and/or other process described herein may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors, by hardware, or combinations thereof. As noted above, the code may be stored on a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

FIG. 11 is a diagram illustrating an example of a system for implementing certain aspects of the present technology. In particular, FIG. 11 illustrates an example of computing system 1100, which can be for example any computing device making up internal computing system, a remote computing system, a camera, or any component thereof in which the components of the system are in communication with each other using connection 1105. Connection 1105 can be a physical connection using a bus, or a direct connection into processor 1110, such as in a chipset architecture. Connection 1105 can also be a virtual connection, networked connection, or logical connection.

In some embodiments, computing system 1100 is a distributed system in which the functions described in this disclosure can be distributed within a datacenter, multiple data centers, a peer network, etc. In some embodiments, one or more of the described system components represents many such components each performing some or all of the function for which the component is described. In some embodiments, the components can be physical or virtual devices.

Example system 1100 includes at least one processing unit (CPU or processor) 1110 and connection 1105 that couples various memory units and other system components including system memory 1115, such as read-only memory (ROM) 1120 and random access memory (RAM) 1125 to processor 1110. Computing system 1100 can include a cache 1112 of high-speed memory connected directly with, in close proximity to, or integrated as part of processor 1110.

Processor 1110 can include any general purpose processor and a hardware service or software service, such as

services

1132, 1134, and 1136 stored in storage device 1130, configured to control processor 1110 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Processor 1110 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

To enable user interaction, computing system 1100 includes an input device 1145, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, etc. Computing system 1100 can also include output device 1135, which can be one or more of a number of output mechanisms. In some instances, multimodal systems can enable a user to provide multiple types of input/output to communicate with computing system 1100. Computing system 1100 can include communications interface 1140, which can generally govern and manage the user input and system output. The communication interface may perform or facilitate receipt and/or transmission wired or wireless communications using wired and/or wireless transceivers, including those making use of an audio jack/plug, a microphone jack/plug, a universal serial bus (USB) port/plug, an

port/plug, an Ethernet port/plug, a fiber optic port/plug, a proprietary wired port/plug, a

wireless signal transfer, a

low energy (BLE) wireless signal transfer, an

wireless signal transfer, a radio-frequency identification (RFID) wireless signal transfer, near-field communications (NFC) wireless signal transfer, dedicated short range communication (DSRC) wireless signal transfer, 802.11 Wi-Fi wireless signal transfer, wireless local area network (WLAN) signal transfer, Visible Light Communication (VLC) , Worldwide Interoperability for Microwave Access (WiMAX) , Infrared (IR) communication wireless signal transfer, Public Switched Telephone Network (PSTN) signal transfer, Integrated Services Digital Network (ISDN) signal transfer, 3G/4G/5G/LTE cellular data network wireless signal transfer, ad-hoc network signal transfer, radio wave signal transfer, microwave signal transfer, infrared signal transfer, visible light signal transfer, ultraviolet light signal transfer, wireless signal transfer along the electromagnetic spectrum, or some combination thereof. The communications interface 1140 may also include one or more Global Navigation Satellite System (GNSS) receivers or transceivers that are used to determine a location of the computing system 1100 based on receipt of one or more signals from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the US-based Global Positioning System (GPS) , the Russia-based Global Navigation Satellite System (GLONASS) , the China-based BeiDou Navigation Satellite System (BDS) , and the Europe-based Galileo GNSS. There is no restriction on operating on any particular hardware arrangement, and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

Storage device 1130 can be a non-volatile and/or non-transitory and/or computer-readable memory device and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, a floppy disk, a flexible disk, a hard disk, magnetic tape, a magnetic strip/stripe, any other magnetic storage medium, flash memory, memristor memory, any other solid-state memory, a compact disc read only memory (CD-ROM) optical disc, a rewritable compact disc (CD) optical disc, digital video disk (DVD) optical disc, a blu-ray disc (BDD) optical disc, a holographic optical disk, another optical medium, a secure digital (SD) card, a micro secure digital (microSD) card, a Memory

card, a smartcard chip, a EMV chip, a subscriber identity module (SIM) card, a mini/micro/nano/pico SIM card, another integrated circuit (IC) chip/card, random access memory (RAM) , static RAM (SRAM) , dynamic RAM (DRAM) , read-only memory (ROM) , programmable read-only memory (PROM) , erasable programmable read-only memory (EPROM) , electrically erasable programmable read-only memory (EEPROM) , flash EPROM (FLASHEPROM) , cache memory (L1/L2/L3/L4/L5/L#) , resistive random-access memory (RRAM/ReRAM) , phase change memory (PCM) , spin transfer torque RAM (STT-RAM) , another memory chip or cartridge, and/or a combination thereof.

The storage device 1130 can include software services, servers, services, etc., that when the code that defines such software is executed by the processor 1110, it causes the system to perform a function. In some embodiments, a hardware service that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 1110, connection 1105, output device 1135, etc., to carry out the function.

As used herein, the term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other mediums capable of storing, containing, or carrying instruction (s) and/or data. A computer-readable medium may include a non-transitory medium in which data can be stored and that does not include carrier waves and/or transitory electronic signals propagating wirelessly or over wired connections. Examples of a non-transitory medium may include, but are not limited to, a magnetic disk or tape, optical storage media such as compact disk (CD) or digital versatile disk (DVD) , flash memory, memory or memory devices. A computer-readable medium may have stored thereon code and/or machine-executable instructions that may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted using any suitable means including memory sharing, message passing, token passing, network transmission, or the like.

In some embodiments the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

Specific details are provided in the description above to provide a thorough understanding of the embodiments and examples provided herein. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For clarity of explanation, in some instances the present technology may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software. Additional components may be used other than those shown in the figures and/or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

Individual embodiments may be described above as a process or method which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed, but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function.

Processes and methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media. Such instructions can include, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or a processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions, information used, and/or information created during methods according to described examples include magnetic or optical disks, flash memory, USB devices provided with non-volatile memory, networked storage devices, and so on.

Devices implementing processes and methods according to these disclosures can include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and can take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a computer-readable or machine-readable medium. A processor (s) may perform the necessary tasks. Typical examples of form factors include laptops, smart phones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rackmount devices, standalone devices, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are example means for providing the functions described in the disclosure.

In the foregoing description, aspects of the application are described with reference to specific embodiments thereof, but those skilled in the art will recognize that the application is not limited thereto. Thus, while illustrative embodiments of the application have been described in detail herein, it is to be understood that the inventive concepts may be otherwise variously embodied and employed, and that the appended claims are intended to be construed to include such variations, except as limited by the prior art. Various features and aspects of the above-described application may be used individually or jointly. Further, embodiments can be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. The specification and drawings are, accordingly, to be regarded as illustrative rather than restrictive. For the purposes of illustration, methods were described in a particular order. It should be appreciated that in alternate embodiments, the methods may be performed in a different order than that described.

One of ordinary skill will appreciate that the less than ( “<” ) and greater than ( “>” ) symbols or terminology used herein can be replaced with less than or equal to ( “≤” ) and greater than or equal to ( “≥” ) symbols, respectively, without departing from the scope of this description.

Where components are described as being “configured to” perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits or other hardware to perform the operation, by programming programmable electronic circuits (e.g., microprocessors, or other suitable electronic circuits) to perform the operation, or any combination thereof.

The phrase “coupled to” refers to any component that is physically connected to another component either directly or indirectly, and/or any component that is in communication with another component (e.g., connected to the other component over a wired or wireless connection, and/or other suitable communication interface) either directly or indirectly.

Claim language or other language reciting “at least one of” a set and/or “one or more” of a set indicates that one member of the set or multiple members of the set (in any combination) satisfy the claim. For example, claim language reciting “at least one of A and B” means A, B, or A and B. In another example, claim language reciting “at least one of A, B, and C” means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The language “at least one of” a set and/or “one or more” of a set does not limit the set to the items listed in the set. For example, claim language reciting “at least one of A and B” can mean A, B, or A and B, and can additionally include items not listed in the set of A and B.

The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or combinations thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices such as general purposes computers, wireless communication device handsets, or integrated circuit devices having multiple uses including application in wireless communication device handsets and other devices. Any features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, performs one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise memory or data storage media, such as random access memory (RAM) such as synchronous dynamic random access memory (SDRAM) , read-only memory (ROM) , non-volatile random access memory (NVRAM) , electrically erasable programmable read-only memory (EEPROM) , FLASH memory, magnetic or optical data storage media, and the like. The techniques additionally, or alternatively, may be realized at least in part by a computer-readable communication medium that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer, such as propagated signals or waves.

The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs) , general purpose microprocessors, an application specific integrated circuits (ASICs) , field programmable logic arrays (FPGAs) , or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general purpose processor may be a microprocessor; but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Accordingly, the term “processor, ” as used herein may refer to any of the foregoing structure, any combination of the foregoing structure, or any other structure or apparatus suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured for encoding and decoding, or incorporated in a combined video encoder-decoder (CODEC) .

Claims

A method of automatic focus control, the method comprising:

actuating a motor to move a lens from a first position to a second position;

receiving a first image frame captured by an image sensor;

receiving a measurement from a lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame;

determining, based on the measurement, that the lens is in motion during the capture time window;

receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor; and

determining a focus indicator associated with the second position, wherein the focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window.
The method of claim 1, further comprising:

determining a measured position of the lens based on the measurement, wherein determining that the lens is in motion during the capture time window includes determining that the measured position has not reached the second position during the capture time window.
The method of claim 1, wherein determining that the lens is in motion within the capture time window includes determining, based on the measurement, that the motor is actuated during the capture time window.
The method of claim 1, wherein the lens position sensor includes a Hall effect sensor and the measurement identifies a Hall feedback of the motor, and wherein determining that the lens is in motion during the capture time window is based on the Hall feedback of the motor.
The method of claim 1, further comprising:

performing a contrast detection autofocus (CDAF) procedure based on the focus indicator, wherein the focus indicator includes a contrast value.
The method of claim 1, further comprising:

performing a phase detection autofocus (PDAF) procedure based on the focus indicator, wherein the focus indicator includes a phase difference.
The method of claim 1, further comprising:

actuating the motor to move the lens from the second position to a third position automatically in response to determining the focus indicator associated with the second position.
The method of claim 1, further comprising:

delaying a movement of the lens from the second position to a third position from a first scheduled time after capture of the first image frame to a second scheduled time after capture of the second image frame in response to determining that the lens is in motion during the capture time window.
The method of claim 1, wherein the motor is a voice coil motor (VCM) .
The method of claim 1, further comprising:

setting a lens position value associated with the first image frame to be equal to a previous position value in response to determining that the lens is in motion during the capture time window, the previous position value associated with a previous image frame captured by the image sensor before capture of the first image frame.
The method of claim 1, further comprising:

receiving a second measurement from the lens position sensor, the second measurement measured during a second capture window within which the image sensor captures the second image frame; and

determining, based on the second measurement, that the lens is stationary during the second capture window, wherein the focus indicator is determined using the second image frame also in response to determining that the lens is stationary during the second capture window.
The method of claim 1, wherein the first image frame and the second image frame are captured consecutively by the image sensor.
The method of claim 1, further comprising:

packing the first image frame together with a first lens position into a first container, the first lens position determined based on the measurement from the lens position sensor;

packing the second image frame together with a second lens position into a second container, the second lens position determined based on a second measurement from the lens position sensor that is measured during a second capture window within which the image sensor captures the second image frame;

unpacking first information from the first container, wherein determining that the lens is in motion during the capture time window is based on the first information from the first container; and

unpacking second information from the second container, wherein determining the focus indicator associated with the second position is based on the second information from the second container.
An apparatus for automatic focus control, the apparatus comprising:

a motor connector coupled to a motor configured to move a lens;

an image sensor connector coupled to an image sensor;

a lens position sensor connector coupled to a lens position sensor;

one or more non-transitory storage media storing instructions; and

one or more processors configured to execute the instructions, wherein execution of the instructions by the one or more processors causes the one or more processors to:

actuate the motor using the motor connector to move the lens from a first position to a second position;

receive, using the image sensor connector, a first image frame captured by the image sensor;

receive, using the lens position sensor connector, a measurement from the lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame;

determining, based on the measurement, that the lens is in motion during the capture time window;

receiving a second image frame captured by the image sensor after the first image frame is captured by the image sensor; and

determining a focus indicator associated with the second position, wherein the focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window.
The apparatus of claim 14, wherein the apparatus is a mobile device.
The apparatus of claim 14, wherein the apparatus is a wireless communication device.
The apparatus of claim 14, wherein the apparatus includes a display configured to display the second image frame.
The apparatus of claim 14, further comprising:

the motor;

the image sensor;

the lens position sensor; and

the lens.
The apparatus of claim 14, wherein the apparatus is a camera.
The apparatus of claim 14, wherein the lens position sensor connector is the motor connector.
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

determine a measured position of the lens based on the measurement, wherein determining that the lens is in motion during the capture time window includes determining that the measured position has not reached the second position during the capture time window.
The apparatus of claim 14, wherein determining that the lens is in motion during the capture time window includes determining, based on the measurement, that the motor is actuated during the capture time window.
The apparatus of claim 14, wherein the lens position sensor includes a Hall effect sensor and the measurement identifies a Hall feedback of the motor, and wherein determining that the lens is in motion during the capture time window is based on the Hall feedback of the motor.
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

perform at least one of a contrast detection autofocus (CDAF) procedure and a phase detection autofocus (PDAF) procedure based on the focus indicator, wherein the focus indicator includes at least one of a contrast value and a phase difference.
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

actuate the motor to move the lens from the second position to a third position automatically in response to determining the focus indicator associated with the second position.
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

delay a movement of the lens from the second position to a third position from a first scheduled time after capture of the first image frame to a second scheduled time after capture of the second image frame in response to determining that the lens is in motion during the capture time window.
The apparatus of claim 14, wherein the motor is a voice coil motor (VCM) .
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

set a lens position value associated with the first image frame to be equal to a previous position value in response to determining that the lens is in motion during the capture time window, the previous position value associated with a previous image frame captured by the image sensor before capture of the first image frame.
The apparatus of claim 14, wherein execution of the instructions by the one or more processors causes the one or more processors to:

receiving a second measurement from the lens position sensor, the second measurement measured during a second capture window within which the image sensor captures the second image frame; and

determining, based on the second measurement, that the lens is stationary during the second capture window, wherein the focus indicator is determined using the second image frame also in response to determining that the lens is stationary during the second capture window.
A non-transitory computer-readable medium is provided that has stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

actuate a motor to move a lens from a first position to a second position;

receive a first image frame captured by an image sensor;

receive a measurement from a lens position sensor, the measurement measured during a capture time window within which the image sensor captures the first image frame;

determine, based on the measurement, that the lens is in motion during the capture time window;

receive a second image frame captured by the image sensor after the first image frame is captured by the image sensor; and

determine a focus indicator associated with the second position, wherein the focus indicator is determined using the second image frame in response to determining that the lens is in motion during the capture time window.