CN112911135B - Focusing control method, front-end image processor and electronic equipment - Google Patents
Focusing control method, front-end image processor and electronic equipment Download PDFInfo
- Publication number
- CN112911135B CN112911135B CN202011632623.2A CN202011632623A CN112911135B CN 112911135 B CN112911135 B CN 112911135B CN 202011632623 A CN202011632623 A CN 202011632623A CN 112911135 B CN112911135 B CN 112911135B
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- definition
- open
- driving current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Abstract
The application provides a focusing control method, a front-end image processor and an electronic device, wherein an initial driving current is provided for an open-loop motor in a camera to drive a lens in the camera to an initial position; acquiring an initial position image through a camera, and acquiring a first definition of a focusing area of the camera according to the initial position image; updating the driving current provided for the open-loop motor according to the first definition, and driving the lens to an updated position; acquiring an image through the camera again to obtain an updated position image, and acquiring a second definition of the focusing area according to the updated position image; and updating the driving current provided for the open-loop motor according to the second definition, and circulating the steps until the lens is driven to the focusing position corresponding to the focusing area. Therefore, the purpose of improving the focusing efficiency of the open-loop motor is achieved by simulating the feedback process of the closed-loop motor.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a focus control method, a front-end image processor, and an electronic device.
Background
At present, users usually use electronic devices (such as digital cameras, smart phones, etc.) with cameras to capture images, so as to record things around, scenes seen, etc. anytime and anywhere. In order to improve the shooting experience of a user, not only the camera pixels of the electronic device but also the shooting speed of the electronic device need to be improved, and the shooting speed depends on the focusing speed of the camera to a certain extent. In the related art, an electronic device in which a lens in a camera is driven by an open-loop motor to focus tends to have poor focusing efficiency.
Disclosure of Invention
The embodiment of the application provides a focusing control method, a front-end image processor and an electronic device, which can improve the focusing efficiency of an open-loop motor.
The application provides a focusing control method, which comprises the following steps:
providing an initial driving current for an open-loop motor in a camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current;
acquiring an image through the camera to obtain an initial position image, and acquiring a first definition of a focusing area of the camera according to the initial position image;
updating the driving current provided for the open-loop motor according to the first definition, so that the open-loop motor drives the lens to an updated position according to the updated driving current;
acquiring an image through the camera to obtain an updated position image, and acquiring a second definition of the focusing area according to the updated position image;
and updating the driving current provided for the open-loop motor according to the second definition until the lens is driven to the focusing position corresponding to the focusing area.
The present application also provides a front-end image processor, comprising:
the central processing unit is used for providing an initial driving current for an open-loop motor in the camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current;
the image signal processing unit acquires an image through the camera to obtain an initial position image and acquires a first definition of a focusing area of the camera according to the initial position image;
the central processing unit is further used for updating the driving current provided for the open-loop motor according to the first definition, so that the open-loop motor drives the lens to an updated position according to the updated driving current;
the image signal processing unit is also used for acquiring images through the camera to obtain an updated position image and acquiring a second definition of the focusing area according to the updated position image;
the central processing unit is further configured to update the driving current provided to the open-loop motor according to the second definition until the lens is driven to a focusing position corresponding to the focusing area.
The present application further provides an electronic device, including:
the camera is used for collecting images;
the front-end image processor is used for carrying out first-time image processing on an image acquired by the camera;
the image processor is used for carrying out second image processing on the image after the first image processing;
and the application processor is used for displaying the image after the second image processing.
The present application further provides an electronic device, comprising:
a memory for storing a computer program;
an application processor for executing the computer program to perform the focus control method as provided herein.
According to the method, an initial driving current is provided for an open-loop motor in the camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current; acquiring images through a camera to obtain an initial position image, and acquiring a first definition of a focusing area of the camera according to the initial position image; updating the driving current provided for the open-loop motor according to the first definition, so that the open-loop motor drives the lens to an updated position according to the updated driving current; acquiring an image through a camera to obtain an updated position image, and acquiring a second definition of a focusing area according to the updated position image; and updating the driving current provided for the open-loop motor according to the second definition until the lens is driven to the focusing position corresponding to the focusing area. Therefore, the definition of images collected at different positions by the lens in the camera driven by the open-loop motor is utilized in the focusing control based on the open-loop motor, the driving current provided for the open-loop motor is updated, the feedback process of the closed-loop motor can be simulated, and the purpose of improving the focusing efficiency of the open-loop motor is achieved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 is a flowchart illustrating a focus control method according to an embodiment of the present application.
FIG. 2 is a graph illustrating a correlation function between a driving current of an open-loop motor and a sharpness of a focus area according to an embodiment of the present disclosure.
Fig. 3 is a graph illustrating a derivative function corresponding to the correlation function in fig. 2.
Fig. 4 is a diagram illustrating a comparison result between the focus control in the present application and the related art.
Fig. 5 is a schematic diagram of sharpness evaluation performed by a sharpness evaluation model in an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a front image processor according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Fig. 8 is another schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
It is to be appreciated that the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.
Relational terms such as first and second, and the like may be used solely to distinguish one object or operation from another object or operation without necessarily limiting the actual sequential relationship between the objects or operations.
The embodiment of the application provides a focusing control method, a front-end image processor and an electronic device, wherein an execution main body of the focusing control method can be the front-end image processor provided in the embodiment of the application, and can also be an application processor in the electronic device. The entity display form of the electronic device may be a device with processing capability configured with a processor, such as a smart phone, a tablet computer, a vehicle-mounted computer, a palm computer, a notebook computer, a server, or a desktop computer.
Referring to fig. 1, fig. 1 is a schematic flow chart of a focusing control method according to an embodiment of the present disclosure, and as shown in fig. 1, the flow of the focusing control method according to the embodiment of the present disclosure may be as follows:
at 110, an initial driving current is provided to an open loop motor in the camera, so that the open loop motor drives a lens in the camera to an initial position according to the initial driving current.
It should be noted that the camera head is composed of a plurality of parts, and mainly includes a lens, a motor, an image sensor, and the like. The lens is used for projecting an external optical signal to the image sensor; the image sensor is used for performing photoelectric conversion on the optical signal projected by the lens and converting the optical signal into an available electric signal to obtain original image data; the motor is used for driving the lens to move, so that the distance between the lens and the image sensor is adjusted to meet an imaging formula (or called lens imaging formula, Gaussian imaging formula and the like), and imaging is clear. In the method, a process of driving a lens to move by a motor to make an image clear is commonly called focusing.
In many aspects, the motor configured in the camera is usually a voice coil motor (or called as a voice coil motor), and the main principle is that in a permanent magnetic field, by changing the magnitude of the driving current of the coil in the voice coil motor, the driving current is converted into mechanical force to control the stretching position of the spring plate in the voice coil motor, so as to drive the lens to move. The voice coil motor can be functionally classified into an open-loop motor, a closed-loop motor, a mid-motor, an optical anti-vibration motor (including a translation motor, a shift shaft motor, a memory metal motor, etc.), and an optical anti-vibration closed-loop motor.
The open-loop motor drives the lens to move for multiple times in a control-response-control mode, and a focusing position (namely, the position which enables the camera to shoot the image clearest) is found in repeated movement. Due to the fact that the focusing speed of the open-loop motor is low, the open-loop motor is controlled for multiple times to drive the lens to move, focusing time is further prolonged, and the focusing efficiency of the open-loop motor is poor. Although the focusing efficiency of the open-loop motor is poor, the open-loop motor still occupies most of the market share at present due to the consideration of cost and the like. Therefore, it is necessary to improve the focusing efficiency of the camera using the open-loop motor, and accordingly, the present application provides a focusing control method suitable for a camera using the open-loop motor for focusing control.
Wherein, when triggering the focusing of the camera, an initial driving current is provided to an open loop motor in the camera. For example, focusing of the camera may be triggered according to a focusing operation of a user, or the focusing may be automatically triggered after the camera is enabled.
It should be noted that, in the embodiment of the present application, the current magnitude of the initial driving current is not particularly limited, and may be configured by a person skilled in the art according to a specific open-loop motor, for example, for an open-loop motor, the initial driving current may be configured to be a minimum driving current capable of driving the lens to generate a displacement.
As described above, the lens in the camera is driven to move to a corresponding position by the open-loop motor under the action of the driving current, and accordingly, the position to which the lens is driven by the open-loop motor under the action of the initial driving current is referred to as an initial position in the present application.
At 120, image acquisition is performed through the camera to obtain an initial position image, and a first definition of a focusing area of the camera is obtained according to the initial position image.
As described above, after the lens in the camera is driven to the initial position by the open-loop motor, image capturing is performed by the camera, and the image captured at this time is regarded as the initial position image. It will be appreciated that the initial position image may or may not be sharp. In order to measure whether the initial position image is clear or not, after the initial position image is acquired through a camera, the definition of a focusing area in the initial position image is quantitatively evaluated according to a preset definition evaluation strategy, and the definition obtained through evaluation is recorded as the first definition of the focusing area.
It should be noted that the configuration of the sharpness evaluation policy in the embodiment of the present application is not particularly limited, and may be configured by a person having ordinary skill in the art according to actual needs.
At 130, the driving current provided to the open-loop motor is updated according to the first sharpness, such that the open-loop motor drives the lens to an updated position according to the updated driving current.
After the first definition of the focusing area is obtained, further judging whether the first definition reaches a preset definition, if so, judging that the camera is focused, and the current position (initial position at the moment) of the camera is a focused position; if not, the camera is judged not to be in focus, and the position of the lens needs to be updated.
It should be noted that, the value of the preset definition may be configured as a fixed value, or may also be configured as a dynamic value, for example, when configured as a dynamic value, the maximum definition that the focusing area can achieve in the focusing process may be configured as the preset definition.
And when the camera is judged not to be in focus, further updating the driving current provided for the open-loop motor according to the first definition according to the configured current updating strategy. The configuration of the current update strategy is not particularly limited, and may be configured by those skilled in the art according to actual needs.
It is understood that when the driving current supplied to the open-loop motor is updated, the open-loop motor will drive the lens in the camera head to a new position different from the initial position, which is referred to as an updated position.
At 140, image acquisition is performed through the camera to obtain an updated position image, and a second definition of the focusing area is obtained according to the updated position image.
As described above, after the lens in the camera is driven to the update position, image capture is performed by the camera, and the image captured at this time is regarded as an update position image. As above, it is to be appreciated that the updated position image may or may not be sharp. In order to judge whether the updated position image is clear or not, after the updated position image is acquired through the camera, the definition of a focusing area in the updated position image is quantitatively evaluated according to a preset definition evaluation strategy, and the definition obtained through evaluation is recorded as the second definition of the focusing area.
At 150, the driving current provided to the open-loop motor is updated according to the second sharpness until the lens is driven to a focus position corresponding to the focus area.
After the second definition of the focusing area is obtained, whether the second definition reaches the preset definition is further judged, if yes, the camera is judged to be focused, and the current position (the updated position at the moment) of the camera is the focused position; if not, the camera is judged not to be in focus, and the position of the lens needs to be updated.
And when the camera is judged not to be focused, further updating the driving current provided for the open-loop motor according to the second definition according to the configured current updating strategy, and circulating the steps until the lens in the camera is driven to a focusing position, namely the position where the definition of a focusing area reaches the preset definition.
In the focusing control method provided by the application, an initial driving current is provided for an open-loop motor in a camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current; acquiring an image through a camera to obtain an initial position image, and acquiring a first definition of a focusing area of the camera according to the initial position image; updating the driving current provided for the open-loop motor according to the first definition, so that the open-loop motor drives the lens to an updated position according to the updated driving current; acquiring an image through a camera to obtain an updated position image, and acquiring a second definition of a focusing area according to the updated position image; and updating the driving current provided for the open-loop motor according to the second definition until the lens is driven to the focusing position corresponding to the focusing area. By the above, in the focusing control based on the open-loop motor, the definition of images collected at different positions by the lens in the camera driven by the open-loop motor is utilized, the drive current provided for the open-loop motor is updated, the feedback process of the closed-loop motor can be simulated, and the aim of improving the focusing efficiency of the open-loop motor is fulfilled.
Optionally, in an embodiment, the focus control method provided by the present application further includes:
(1) obtaining a correlation function between the driving current of the open-loop motor and the definition of a focusing area, and obtaining a derivative function of the correlation function;
updating a drive current provided to an open loop motor in accordance with a first definition, comprising:
(2) obtaining a next iteration solution which enables the value of the derivative function to be zero through a Newton iteration formula according to the derivative value corresponding to the first definition;
(3) the drive current provided to the open-loop motor is updated according to the next iterative solution.
It should be noted that, in the embodiment of the present application, the value of the preset definition is dynamically configured, that is, the preset definition is configured to be the maximum definition that can be achieved by the focusing area in the focusing process, in other words, by continuously updating the driving current provided to the open-loop motor, when the definition of the focusing area reaches the maximum definition, it is determined that the camera is focused, and at this time, the position where the lens in the camera is located is the focusing position.
Referring to fig. 2, the inventors of the present application have found that, during the focusing process, there is a graphical correlation between the driving current of the open-loop motor and the sharpness of the focusing area, and before the sharpness reaches the maximum value, the sharpness of the focusing area is positively correlated with the magnitude of the driving current, that is, when the driving current is increased, the sharpness of the focusing area is increased, and after the sharpness reaches the maximum value, the sharpness of the focusing area is negatively correlated with the magnitude of the driving current, that is, when the driving current is increased, the sharpness of the focusing area is decreased. Based on this, in the embodiment of the present application, a correlation function between the driving current of the open-loop motor and the definition of the image captured by the camera may be pre-established, as shown in fig. 2, a function curve of the correlation function is a parabola, which may be represented as:
y＝ax2；
where y denotes the sharpness of the focus region, a is a constant, x denotes a drive current supplied to the open-loop motor, and y is hereinafter referred to as ax2Is denoted by f (x).
Referring to fig. 3, the inventor of the present application further finds that, after deriving the correlation function f (x), the maximum value of the correlation function f (x) corresponds to a solution when the derivative function has a value of zero. Therefore, the maximum resolution that can be achieved by the focusing region can be quickly determined by solving the solution that makes the value of the derivative function zero.
Correspondingly, in the embodiment of the present application, a correlation function f (x) between the pre-established open-loop motor and the definition of the image acquired by the camera is further obtained, and a derivative function of the correlation function f (x) is obtained, and the derivative function of the correlation function f (x) is denoted as g (x) below.
In order to quickly obtain a solution that makes the value of the derivative function (i.e., the sharpness derivative value) zero, a newton iteration method is used in the embodiment of the present application to obtain a solution that makes the value of the derivative function zero.
Accordingly, when the driving current provided to the open-loop motor is updated according to the first definition, the next iteration solution that makes the value of the derivative function g (x) zero can be obtained through the newton iteration formula according to the derivative value corresponding to the first definition. It should be noted that the next iteration solution corresponding to the first definition cannot make the value of the derivative function g (x) zero, but can make the value of the derivative function g (x) go to zero, and accordingly the driving current supplied to the open-loop motor is updated according to the next iteration solution corresponding to the first definition, that is, the driving current supplied to the open-loop motor is updated to the magnitude of the next iteration solution corresponding to the first definition.
Similarly, when the lens moves to other positions except the initial position and the driving current provided for the open-loop motor needs to be updated, the next iteration solution for enabling the value of the derivative function to be zero can be obtained through the Newton iteration formula according to the derivative value corresponding to the definition of the focusing region at the other positions; and updates the drive current supplied to the open-loop motor according to the next iteration solution obtained at this time.
For example, when the driving current provided to the open-loop motor is updated according to the second definition, the next iteration solution that makes the value of the derivative function zero can be obtained through a newton iteration formula according to the derivative value corresponding to the second definition; and updating the drive current provided to the open-loop motor based on a next iterative solution corresponding to the second resolution.
Therefore, the driving current provided for the open-loop motor is iteratively updated by circularly utilizing the definition of the focusing area, and the camera can be rapidly focused.
Referring to fig. 4, a schematic diagram of a conventional focusing method in the related art and a focusing control method provided in the present application is shown. It can be seen that, in the conventional focusing process, the process of approaching the maximum definition, exceeding the maximum definition, and tracing back the maximum definition needs to move the lens back and forth, and in the one-time focusing process shown in fig. 4, the open-loop motor drives the lens to move 7 times to complete focusing. In the focusing process based on the newton iteration method provided by the application, the lens does not move back and forth but moves in a single direction, and each movement can ensure that the definition of a focusing area is higher, in the one-time focusing process shown in fig. 4, the open-loop motor only drives the lens to move for 3 times to complete focusing, so that the limitation of shortening the moving times of the lens driven by the open-loop motor is realized, and the focusing efficiency is improved.
Optionally, obtaining a next iterative solution that makes the value of the derivative function zero by a newton iterative formula includes:
in a newton iteration formula, a next iteration solution that makes the value of the derivative function zero is determined according to the initial drive current, the derivative value corresponding to the first definition, and the derivative value of the derivative function to the initial drive current.
In the embodiment of the present application, the newton iteration formula includes:
xk+1representing the next iterative solution, x, that makes the value of the derivative function zerokRepresents the drive current currently supplied to the open-loop motor, g (x)k) A derivative value g' (x) indicating the sharpness of the focused region at the current positionk) Representing the derivative value of the derivative function to the drive current currently supplied to the open-loop motor.
For example, when the next iteration solution corresponding to the first definition is obtained, the initial driving current, the derivative value corresponding to the first definition, and the derivative value of the derivative function to the initial driving current are substituted into the formula, where the initial driving current is the driving current currently provided to the open-loop motor, the derivative value corresponding to the first definition is the derivative value corresponding to the definition of the focus area at the current position, and the derivative value of the derivative function to the initial driving current is the derivative value of the derivative function to the driving current currently provided to the open-loop motor.
Optionally, in an embodiment, acquiring the first definition of the focus area of the camera according to the initial position image includes:
and obtaining a contrast value of the image content of the corresponding focusing area in the initial position image, and taking the contrast value as a first definition.
It should be noted that the sharper the image is, the higher its contrast value is. Therefore, the contrast value can be used to measure the sharpness of the image.
In the embodiment of the application, when the first definition of the focusing area of the camera is obtained according to the initial position image, the contrast value of the image content of the corresponding focusing area in the initial position image can be obtained, and the contrast value is used as the first definition of the focusing area.
Similarly, obtaining the second definition of the focusing area according to the updated position image includes:
and acquiring a contrast value of the image content of the corresponding focusing area in the updated position image, and taking the contrast value as a second definition.
Optionally, in an embodiment, acquiring the first definition of the focus area of the camera according to the initial position image includes:
and inputting the image content of the corresponding focusing area in the initial position image into a pre-trained definition evaluation model for definition evaluation, and evaluating the definition evaluation model to obtain the definition serving as the first definition.
It should be noted that, in the embodiment of the present application, a machine learning method is also adopted in advance to train a sharpness evaluation model, and the sharpness evaluation model is configured to quantitatively evaluate the sharpness of the input image. The model architecture and the training mode of the sharpness evaluation model are not specifically limited, and can be selected by a person of ordinary skill in the art according to actual needs, for example, a convolutional neural network model can be used as a basic model, and supervised model training is performed on the model by using an image sample with a marked sharpness, so that the sharpness evaluation model is obtained.
Correspondingly, in the embodiment of the application, when the definition of the image needs to be evaluated, the image needing to be evaluated can be input into the pre-trained definition evaluation model for definition evaluation, so that the definition of the image needing to be evaluated is obtained.
For example, referring to fig. 5, in the embodiment of the present application, when the first definition of the focusing area of the camera is obtained according to the initial position image, the image content of the corresponding focusing area may be captured from the initial position image, and the image content of the corresponding focusing area is input into a pre-trained definition evaluation model, and the definition obtained by the definition evaluation is used as the first definition of the focusing area.
Similarly, when the definition of the image acquired when the lens is moved to other positions except the initial position needs to be evaluated, the image content of the focusing area in the image acquired by the camera at other positions can be input into the pre-trained definition evaluation model for definition evaluation, so that the definition of the focusing area at other positions can be obtained.
Optionally, in an embodiment, the focus control method provided by the present application further includes:
acquiring a preview image of a camera, and inputting the preview image into a pre-trained visual attention model to identify a visual salient region;
and setting the recognized visual salient region as a focusing region of the camera.
It should be noted that, in the embodiment of the present application, a visual attention model configured to identify a visually significant region of an input image is also trained in advance. The model architecture and the training mode of the visual attention model are not particularly limited, and can be selected by a person of ordinary skill in the art according to actual needs, for example, a convolutional neural network model can be used as a basic model, and an image sample with a marked visual salient region is used for performing supervised model training on the model, so as to obtain the visual attention model. For example, the person is generally considered to be more conspicuous than the sky, the grass, and the buildings, and when the person, the sky, the grass, and the buildings are included in an image, the region corresponding to the person is identified as the visually conspicuous region by the visual attention model.
It is understood that people generally tend to take an object of their own interest as a subject of photographing, and tend to focus on the object of interest. Therefore, in the embodiment of the application, when focusing is triggered, a preview image of a camera closest to the moment of focusing triggering can be acquired, the preview image is input into a pre-trained visual attention model to identify a visually significant region, and the identified visually significant region is set as a focusing region of the camera.
In order to better execute the focusing control method provided by the application, the application provides a front-end image processor. Referring to fig. 6, fig. 6 is a schematic structural diagram of a front-end image processor 200 according to an embodiment of the present disclosure. The image processor 200 may include a central processing unit 210 and an image signal processing unit 220, wherein,
a central processing unit 210, configured to provide an initial driving current to the open-loop motor in the camera, so that the open-loop motor drives the lens in the camera to an initial position according to the initial driving current;
the image signal processing unit 220 is used for acquiring an image through the camera to obtain an initial position image and acquiring a first definition of a focusing area of the camera according to the initial position image;
the central processing unit 210 is further configured to update the driving current provided to the open-loop motor according to the first definition, so that the open-loop motor drives the lens to the updated position according to the updated driving current;
the image signal processing unit 220 is further configured to perform image acquisition through the camera to obtain an updated position image, and obtain a second definition of the focusing area according to the updated position image;
the central processing unit 210 is further configured to update the driving current provided to the open-loop motor according to the second resolution until the lens is driven to the in-focus position corresponding to the focus area.
The central processing unit 210 and the image signal processing unit 220 are realized by hardening during the hardware configuration process of circuit arrangement, programming and the like, so that the stability of the central processing unit 210 and the image signal processing unit 220 during the working process can be ensured, and the power consumption, the processing time and the like when the central processing unit 210 and the image signal processing unit 220 process data can be reduced.
The front-end image processor 200 provided by the present application can be configured in an electronic device having an image processor, an application processor and a camera for performing focus control on the camera. The electronic device may be a mobile electronic device such as a smart phone, a tablet computer, a palm computer, a notebook computer, or a fixed electronic device such as a desktop computer and a television. The type of the camera configured for the electronic device is not specifically limited, and may be configured by a person of ordinary skill in the art according to actual needs, including but not limited to a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, and the like.
For how the front-end image processor 200 performs focusing control on the camera, reference may be made to the relevant description in the focusing control method embodiment, which is not described herein again.
It should be noted that, in other embodiments, the front-end image processor 200 is configured to perform, in addition to performing focus control on the camera, first image processing on an image captured by the camera, and transmit the image after the first image processing to the image processor, and transmit the image after the second image processing to the application processor, and display the image by the application processor.
For example, the first image processing performed by the pre-image processor 200 includes at least one of a non-artificial intelligence-based optimization processing method such as a dead pixel correction processing, a time domain noise reduction processing, a 3D noise reduction processing, a linearization processing, and a black level correction processing, and an artificial intelligence-based optimization processing method such as a night scene algorithm, an HDR algorithm, a blurring algorithm, a noise reduction algorithm, and a super-resolution algorithm, and the second image processing performed by the image processor may be any image processing method different from the first image processing.
Optionally, in an embodiment, the central processing unit 210 is further configured to:
obtaining a correlation function between the driving current of the open-loop motor and the definition of an image acquired by the camera, and obtaining a derivative function of the correlation function;
according to the derivative value corresponding to the first definition, obtaining a next iteration solution which enables the value of the derivative function to be zero through a Newton iteration formula; and
the drive current provided to the open-loop motor is updated according to the next iteration solution.
Optionally, in an embodiment, the central processing unit 210 is configured to:
in the newton iteration formula, a next iteration solution that makes the value of the derivative function zero is determined according to the initial drive current, the derivative value corresponding to the first definition, and the derivative value of the derivative function to the initial drive current.
Optionally, in an embodiment, the newton's iterative formula includes:
wherein x isk+1Representing the next iterative solution, x, with the derivative function's value zerokRepresents the drive current currently supplied to the open-loop motor, g (x)k) A derivative value, g' (x) indicating the sharpness of the in-focus area at the current positionk) Representing the derivative value of the derivative function to the drive current currently supplied to the open-loop motor.
Optionally, in an embodiment, the image signal processing unit 220 is configured to:
and acquiring a contrast value of the image content of the corresponding focusing area in the initial position image, and taking the contrast value as the first definition.
Optionally, in an embodiment, the pre-image processor 200 further comprises a neural network processing unit for:
and inputting the image content of the corresponding focusing area in the initial position image into a pre-trained definition evaluation model for definition evaluation according to the instruction of the central processing unit, and evaluating the definition obtained by the definition evaluation model to be used as the first definition.
Optionally, in an embodiment, the neural network processing unit is further configured to:
acquiring a preview image of a camera, and inputting the preview image into a pre-trained visual attention model to identify a visual salient region; and
and setting the recognized visual salient region as a focusing region of the camera.
Referring to fig. 7, the present application further provides an electronic device 300, as shown in fig. 7, the electronic device 300 includes:
a camera 310 for acquiring an image;
the front-end image processor 200 as provided by the present application is used for performing a first image processing on an image acquired by a camera;
an image processor 320 for performing a second image processing on the image after the first image processing;
and an application processor 330, configured to display the image after the second image processing.
It should be noted that the camera 310 is composed of multiple parts, mainly including a lens, a motor, and an image sensor, etc. The lens is used for projecting an external optical signal to the image sensor; the image sensor is used for performing photoelectric conversion on the optical signal projected by the lens and converting the optical signal into an available electric signal to obtain original image data; the motor is used for driving the lens to move, so that the distance between the lens and the image sensor is adjusted to meet an imaging formula (or a lens imaging formula, a Gaussian imaging formula and the like), and imaging is clear. The lens is driven by a motor to move, so that the imaging process is clear, namely focusing. In the embodiment of the present application, the motor of the camera 310 is an open-loop motor.
In addition, the front-end image processing 200 performs a first image processing and a second image processing which are differentiated from the image processor 320, in addition to performing focus control on the camera (please refer to the related description in the above embodiment, which is not described herein again).
For example, the first image processing performed by the pre-image processor 200 includes at least one of a non-artificial intelligence-based optimization processing manner such as a dead pixel correction processing, a time domain noise reduction processing, a 3D noise reduction processing, a linearization processing, and a black level correction processing, and an artificial intelligence-based optimization processing manner such as a night scene algorithm, an HDR algorithm, a blurring algorithm, a noise reduction algorithm, and a super-resolution algorithm, and the second image processing performed by the image processor 320 may be any image processing manner different from the first image processing.
Please refer to fig. 8, and fig. 8 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present disclosure.
The electronic device 400 may include a camera 410, a memory 420, and an application processor 430. Those skilled in the art will appreciate that the configuration of electronic device 400 shown in fig. 8 is not intended to be limiting of electronic device 400 and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
The camera 410 is composed of multiple parts, and mainly includes a lens, a motor, an image sensor, and the like. The lens is used for projecting an external optical signal to the image sensor; the image sensor is used for performing photoelectric conversion on an optical signal projected by the lens and converting the optical signal into an available electric signal to obtain original image data; the motor is used for driving the lens to move, so that the distance between the lens and the image sensor is adjusted to meet an imaging formula (or a lens imaging formula, a Gaussian imaging formula and the like), and imaging is clear. The lens is driven by a motor to move, so that the imaging process is clear, namely focusing. In the embodiment of the present application, the motor of the camera head 410 is an open-loop motor.
The application processor 430 is a control center of the electronic device 400, connects various parts of the entire electronic device 400 using various interfaces and lines, and performs various functions of the electronic device 400 and processes data by running or executing computer programs stored in the memory 420 and calling data stored in the memory 420, thereby performing overall control of the electronic device 400.
In this embodiment, the application processor 430 in the electronic device 400 loads the executable codes corresponding to one or more computer programs into the memory 420, and the application processor 430 executes the following steps in the following order:
providing an initial driving current to an open loop motor in the camera 410, so that the open loop motor drives a lens in the camera 410 to an initial position according to the initial driving current;
acquiring an image through the camera 410 to obtain an initial position image, and acquiring a first definition of a focusing area of the camera 410 according to the initial position image;
updating the driving current provided for the open-loop motor according to the first definition, so that the open-loop motor drives the lens to an updated position according to the updated driving current;
acquiring an image through the camera 410 to obtain an updated position image, and acquiring a second definition of a focusing area according to the updated position image;
and updating the driving current provided for the open-loop motor according to the second definition until the lens is driven to the focusing position corresponding to the focusing area.
Optionally, in an embodiment, the application processor 430 is further configured to perform:
acquiring a correlation function between the driving current of the open-loop motor and the definition of an image acquired by the camera 410, and acquiring a derivative function of the correlation function;
while updating the drive current provided to the open loop motor according to the first definition, the application processor 430 is operable to perform:
according to the derivative value corresponding to the first definition, obtaining a next iteration solution which enables the value of the derivative function to be zero through a Newton iteration formula;
the drive current provided to the open-loop motor is updated according to the next iteration solution.
Optionally, in an embodiment, when obtaining a next iterative solution by newton's iterative formula such that the value of the derivative function is zero, the application processor 430 is configured to perform:
in the newton iteration formula, a next iteration solution that makes the value of the derivative function zero is determined according to the initial drive current, the derivative value corresponding to the first definition, and the derivative value of the derivative function to the initial drive current.
Optionally, in an embodiment, the newton's iterative formula includes:
wherein x isk+1Representing the next iterative solution, x, with the derivative function's value zerokRepresents the drive current currently supplied to the open-loop motor, g (x)k) A derivative value, g' (x) indicating the sharpness of the in-focus area at the current positionk) Representing the derivative value of the derivative function to the drive current currently supplied to the open-loop motor.
Optionally, in an embodiment, after acquiring the first definition of the focusing area of the camera 410 according to the initial position image, the application processor 430 is configured to execute:
and acquiring a contrast value of the image content of the corresponding focusing area in the initial position image, and taking the contrast value as the first definition.
Optionally, in an embodiment, when acquiring the first definition of the focus area of the camera 410 according to the initial position image, the application processor 430 is configured to perform:
and inputting the image content of the corresponding focusing area in the initial position image into a pre-trained definition evaluation model for definition evaluation, and evaluating the definition evaluation model to obtain the definition serving as the first definition.
Optionally, in an embodiment, the application processor 430 is further configured to perform:
acquiring a preview image of the camera 410, and inputting the preview image into a pre-trained visual attention model to identify a visual salient region;
the recognized visually significant region is set as a focus region of the camera 410.
It should be noted that the electronic device 400 provided in the embodiment of the present application and the focusing control method in the foregoing embodiment belong to the same concept, and specific implementation processes thereof are described in the foregoing related embodiments, and are not described herein again.
The focusing control method, the front-end image processor and the electronic device provided by the embodiment of the application are described in detail above. The principles and embodiments of the present application are illustrated herein using specific examples, which are presented only to aid in the understanding of the present application. Meanwhile, for a person skilled in the art, according to the idea of the present application, the specific implementation and the application range may be changed. In view of the above, the description should not be taken as limiting the application.
Claims (11)
1. A focus control method, comprising:
providing an initial driving current for an open-loop motor in a camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current;
acquiring an image through the camera to obtain an initial position image, and acquiring a first definition of a focusing area of the camera according to the initial position image;
acquiring a correlation function between the driving current of the open-loop motor and the definition of the focusing area, acquiring a derivative function of the correlation function, and acquiring a next iteration solution for enabling the value of the derivative function to be zero through a Newton iteration formula according to a derivative value corresponding to the first definition;
updating the driving current provided for the open-loop motor according to the next iteration solution, so that the open-loop motor drives the lens to an updated position according to the updated driving current;
acquiring an image through the camera to obtain an updated position image, and acquiring a second definition of the focusing area according to the updated position image;
and updating the driving current provided for the open-loop motor according to the second definition until the lens is driven to a focusing position corresponding to the focusing area.
2. The focus control method of claim 1, wherein obtaining the next iterative solution that makes the value of the derivative function zero by a newton's iterative formula comprises:
and in the Newton iteration formula, determining a next iteration solution which enables the value of the derivative function to be zero according to the initial driving current, the derivative value corresponding to the first definition and the derivative value of the derivative function to the initial driving current.
3. The focus control method of claim 2, wherein the newton's iterative formula comprises:
wherein x isk+1Representing the next iterative solution, x, with the derivative function value zerokRepresents a driving current currently supplied to the open loop motor, g (x)k) A derivative value g' (x) representing the sharpness of the focusing area at the current positionk) A derivative value representing the derivative function to the drive current currently provided to the open-loop motor.
4. The focus control method of claim 1, wherein said obtaining a first resolution of a focus area of the camera from the initial position image comprises:
and obtaining a contrast value of the image content corresponding to the focusing area in the initial position image, and taking the contrast value as the first definition.
5. The focus control method of claim 1, wherein the obtaining a first definition of a focus area of the camera from the initial position image comprises:
and inputting the image content of the initial position image corresponding to the focusing area into a pre-trained definition evaluation model for definition evaluation, and evaluating the definition evaluation model to obtain definition serving as the first definition.
6. The focus control method of any of claims 1-5, further comprising:
acquiring a preview image of the camera, and inputting the preview image into a pre-trained visual attention model to identify a visual salient region;
and setting the identified visual salient region as a focusing region of the camera.
7. A pre-image processor, comprising:
the central processing unit is used for providing an initial driving current for an open-loop motor in the camera, so that the open-loop motor drives a lens in the camera to an initial position according to the initial driving current;
the image signal processing unit acquires an image through the camera to obtain an initial position image and acquires a first definition of a focusing area of the camera according to the initial position image;
the central processing unit is further configured to obtain a correlation function between the driving current of the open-loop motor and the sharpness of the focusing region, obtain a derivative function of the correlation function, and obtain a next iteration solution that makes a value of the derivative function zero through a newton iteration formula according to a derivative value corresponding to the first sharpness; updating the driving current provided for the open-loop motor according to the next iteration solution, so that the open-loop motor drives the lens to an updated position according to the updated driving current;
the image signal processing unit is also used for acquiring images through the camera to obtain an updated position image and acquiring a second definition of the focusing area according to the updated position image;
the central processing unit is further configured to update the driving current provided to the open-loop motor according to the second definition until the lens is driven to a focusing position corresponding to the focusing area.
8. The front-end image processor of claim 7, further comprising a neural network processing unit for inputting image contents of the initial position image corresponding to the focusing area into a pre-trained sharpness evaluation model for sharpness evaluation according to an instruction of the central processing unit, and evaluating the sharpness evaluation model to obtain sharpness as the first sharpness.
9. The front-end image processor of claim 8, wherein the neural network processing unit is further configured to obtain a preview image of the camera, and input the preview image into a pre-trained visual attention model for identification of visually significant regions; and setting the identified visual salient region as a focusing region of the camera.
10. An electronic device, comprising:
the camera is used for collecting images;
the front-facing image processor of any one of claims 7 to 9, configured to perform a first image processing on an image captured by the camera;
the image processor is used for carrying out second image processing on the image after the first image processing;
and the application processor is used for displaying the image after the second image processing.
11. An electronic device, comprising:
a memory for storing a computer program;
an application processor for executing the computer program to perform the focus control method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632623.2A CN112911135B (en) | 2020-12-31 | 2020-12-31 | Focusing control method, front-end image processor and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632623.2A CN112911135B (en) | 2020-12-31 | 2020-12-31 | Focusing control method, front-end image processor and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112911135A CN112911135A (en) | 2021-06-04 |
| CN112911135B true CN112911135B (en) | 2022-07-15 |
Family
ID=76112010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011632623.2A Active CN112911135B (en) | 2020-12-31 | 2020-12-31 | Focusing control method, front-end image processor and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112911135B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103472658A (en) * | 2012-06-06 | 2013-12-25 | 鸿富锦精密工业（深圳）有限公司 | Auto-focusing method |
| TW201400912A (en) * | 2012-06-19 | 2014-01-01 | Hon Hai Prec Ind Co Ltd | Method for searching position in auto-focus |
| CN106488122A (en) * | 2016-10-14 | 2017-03-08 | 北京信息科技大学 | A kind of dynamic auto focusing algorithm based on improved sobel method |
| CN107437050A (en) * | 2016-05-28 | 2017-12-05 | 上海大学 | Miniature self-service finds safe landing locations method from gyroplane with monocular adjustable focus liquid lens |
| CN108429884A (en) * | 2018-06-05 | 2018-08-21 | 昆山丘钛微电子科技有限公司 | A kind of focusing method, device, equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5371417B2 (en) * | 2008-12-26 | 2013-12-18 | キヤノン株式会社 | LENS DRIVE DEVICE, OPTICAL DEVICE, AND LENS DRIVE DEVICE CONTROL METHOD |
-
2020
- 2020-12-31 CN CN202011632623.2A patent/CN112911135B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103472658A (en) * | 2012-06-06 | 2013-12-25 | 鸿富锦精密工业（深圳）有限公司 | Auto-focusing method |
| TW201400912A (en) * | 2012-06-19 | 2014-01-01 | Hon Hai Prec Ind Co Ltd | Method for searching position in auto-focus |
| CN107437050A (en) * | 2016-05-28 | 2017-12-05 | 上海大学 | Miniature self-service finds safe landing locations method from gyroplane with monocular adjustable focus liquid lens |
| CN106488122A (en) * | 2016-10-14 | 2017-03-08 | 北京信息科技大学 | A kind of dynamic auto focusing algorithm based on improved sobel method |
| CN108429884A (en) * | 2018-06-05 | 2018-08-21 | 昆山丘钛微电子科技有限公司 | A kind of focusing method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112911135A (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10917571B2 (en) | Image capture device control based on determination of blur value of objects in images | |
| CN108322646B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN109889724B (en) | Image blurring method and device, electronic equipment and readable storage medium | |
| CN103167233B (en) | Show device and method | |
| CN103297696A (en) | Photographing method, photographing device and photographing terminal | |
| CN110971841B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| JPWO2012132129A1 (en) | Imaging apparatus, imaging method, and program | |
| CN113099122A (en) | Shooting method, shooting device, shooting equipment and storage medium | |
| CN111405185B (en) | Zoom control method and device for camera, electronic equipment and storage medium | |
| CN110868547A (en) | Photographing control method, photographing control device, electronic equipment and storage medium | |
| CN111968052B (en) | Image processing method, image processing apparatus, and storage medium | |
| CN114390201A (en) | Focusing method and device thereof | |
| CN113747067A (en) | Photographing method and device, electronic equipment and storage medium | |
| US9686470B2 (en) | Scene stability detection | |
| CN112911135B (en) | Focusing control method, front-end image processor and electronic equipment | |
| JP2021128537A (en) | Image processing device, image processing method, program and storage medium | |
| WO2023001110A1 (en) | Neural network training method and apparatus, and electronic device | |
| CN110602397A (en) | Image processing method, device, terminal and storage medium | |
| JP2023078061A (en) | Imaging exposure control method and apparatus, device and storage medium | |
| WO2022226963A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| CN115496664A (en) | Model training method and device, electronic equipment and readable storage medium | |
| CN111724300B (en) | Single picture background blurring method, device and equipment | |
| CN114500837A (en) | Shooting method and device and electronic equipment | |
| CN114723603A (en) | Image processing method, image processing apparatus, and storage medium | |
| CN112399092A (en) | Shooting method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |