CN112135062A - Camera focusing control method and device based on FPGA - Google Patents
Camera focusing control method and device based on FPGA Download PDFInfo
- Publication number
- CN112135062A CN112135062A CN202011324968.1A CN202011324968A CN112135062A CN 112135062 A CN112135062 A CN 112135062A CN 202011324968 A CN202011324968 A CN 202011324968A CN 112135062 A CN112135062 A CN 112135062A
- Authority
- CN
- China
- Prior art keywords
- frequency domain
- step length
- moving step
- current position
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Automatic Focus Adjustment (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a method and a device for controlling camera focusing based on FPGA, which comprises a parameter configuration module, a focusing control module, an image acquisition module, an image processing module and a comparison and judgment module, wherein the image processing module sequentially outputs a plurality of sampling areas of each line of a cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for carrying out frequency domain transformation on a plurality of sampling areas sequentially input into the same line in a pipeline mode and then carrying out accumulation, and the final accumulation results of the plurality of parallel pipelines are summed to obtain a definition function value of the current position image. The invention optimizes the automatic focusing of the microscope by combining the parallel pipeline hardware acceleration of the FPGA with the control of the image acquisition device, and has the advantages of high focusing speed and high processing efficiency.
Description
Technical Field
The invention relates to an automatic focusing technology of a microscope, in particular to a method and a device for controlling camera focusing based on an FPGA (field programmable gate array), which are used for controlling a camera to move through the FPGA so as to realize automatic focusing of a slide by the camera.
Background
For the automatic focusing of a slide of a microscope, the current research mainly researches the implementation of an algorithm of the automatic focusing from the aspect of software, and the main focus of the research is the implementation of a definition function. However, as the resolution of images is higher, the requirements of image processing on the performance of the processor are higher, and in the special application fields such as medical treatment, the image processing has higher requirements on real-time performance, and even though various special processors are started to be applied to hardware acceleration for realizing algorithm tasks at present, the efficiency of image processing can be effectively improved, but the system has defects in the aspects of system maintenance and upgrading. At present, how to realize the combination of software and hardware acceleration to optimize the automatic focusing technology of the microscope is still a key technical problem to be solved urgently. At present, a DSP, a GPU or a CPU is mainly used for processing images by taking a frame as a unit, and an FPGA can be directly connected with an image sensor chip to obtain an image data stream, so that real-time pipeline operation is carried out on the images by taking a line as a unit; the FPGA can also realize the parallel processing of images, the processing delay is fixed, and the highest real-time performance can be achieved; moreover, the FPGA has the programmable characteristic, can be reconfigured according to the requirement, and has stronger universality.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: aiming at the problems in the prior art, the invention provides a camera focusing control method and device based on FPGA, optimizes the automatic focusing of a microscope by combining the parallel pipeline hardware acceleration of FPGA and the control of an image acquisition device, and has the advantages of high focusing speed and high processing efficiency.
In order to solve the technical problems, the invention adopts the technical scheme that:
a method of camera focus control based on an FPGA, comprising:
1) initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
2) controlling a motor of the image acquisition device to move the camera to the current position according to the moving step length;
3) controlling and caching a current position image shot by a camera of the image acquisition device;
4) sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, wherein the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input into the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
5) judging whether the current position exceeds the moving range, and if not, skipping to execute the step 2); otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and skipping to execute the step 2); and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
An apparatus for FPGA-based camera focus control, comprising:
the parameter configuration module is used for initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
the focusing control module comprises a motor control interface which is used for being connected with a motor of the image acquisition device and is used for controlling the motor of the image acquisition device to move the camera to the current position according to the moving step length through the motor control interface;
the image acquisition module comprises an image acquisition interface which is connected with a camera of the image acquisition device and is used for acquiring and caching the current position image shot by the camera through the image acquisition interface;
the image processing module is used for sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input to the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
the comparison and judgment module is used for judging whether the current position exceeds the moving range or not, and if the current position does not exceed the moving range, the focusing control module is called to move the camera to a new current position according to a new current moving step length; otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and calling a focusing control module to move the camera to a new current position according to the new current moving step length; and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
Optionally, the parameter configuration module further includes a light source control interface connected to the light source control end of the image acquisition device, and the parameter configuration module is further configured to configure the control parameters of the light source of the image acquisition device through the light source control interface.
Optionally, the image processing module includes an image collecting unit, a plurality of parallel pipelines and an accumulating unit, the image collecting unit is configured to output a plurality of sampling regions in each line of the cached current position image to the parallel pipelines in sequence, the number of the parallel pipelines corresponds to the number of lines of the sampling regions in the current position image, each parallel pipeline is configured to accumulate frequency domain transform coefficients after performing frequency domain transform on a plurality of sampling regions in the same line in sequence in a pipeline manner, and the accumulating unit is configured to sum up final accumulation results of the frequency domain transform coefficients of the plurality of parallel pipelines to obtain a sharpness function value of the current position image.
Optionally, the parallel pipeline comprises: the line-frequency domain converter is used for performing frequency domain conversion aiming at the input sampling region; the buffer is used for buffering the frequency domain transformation result of each sampling region; the column frequency domain converter is used for performing column frequency domain conversion on the cached results of the line frequency domain conversion of each sampling region to obtain frequency domain conversion coefficients; the accumulation unit comprises a preceding-stage accumulator and a subsequent-stage accumulator, the preceding-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficients obtained by the various rows of frequency domain converters to obtain the frequency domain transformation coefficient accumulation result of each row of sampling areas, and the subsequent-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficient accumulation result of each row of sampling areas to obtain the accumulation results of all the sampling areas.
Optionally, the image processing module includes three parallel pipelines L1, L2, and L3, the image capturing unit uniformly extracts 9 sampling regions of a specified pixel size from the cached current position image, where the first row of sampling regions is P1, P4, and P7, the second row of sampling regions is P2, P5, and P8, and the third row of sampling regions is P3, P6, and P9, sequentially outputs the first row of sampling regions P1, P4, and P7 of the cached current position image to the parallel pipeline L1, sequentially outputs the second row of sampling regions P2, P5, and P8 of the cached current position image to the parallel pipeline L2, and sequentially outputs the third row of sampling regions P3, P6, and P9 of the cached current position image to the parallel pipeline L3; a line frequency domain converter of the parallel pipeline L1 sequentially performs line frequency domain conversion on first line sampling regions P1, P4 and P7 and buffers the line frequency domain conversion into corresponding buffers, a column frequency domain converter of the parallel pipeline L1 sequentially performs column frequency domain conversion on the buffered first line sampling regions P1, P4 and P7 and sends the results of the column frequency domain conversion into a preceding-stage accumulator, a line frequency domain converter of the parallel pipeline L2 sequentially performs line frequency domain conversion on second line sampling regions P2, P5 and P8 and buffers the results of the column frequency domain conversion into corresponding buffers, and a column frequency domain converter of the parallel pipeline L2 sequentially performs column frequency domain conversion on the buffered second line sampling regions P2, P5 and P8 and sends the results of the column frequency domain conversion into the preceding-stage accumulator; the row frequency domain converter of the parallel pipeline L3 sequentially performs row frequency domain conversion on the third row sampling regions P3, P6 and P9 and caches the third row sampling regions in corresponding caches, and the column frequency domain converter of the parallel pipeline L3 sequentially performs column frequency domain conversion on the cached third row sampling regions P3, P6 and P9 and sends the results of the column frequency domain conversion to the former-stage accumulator; the first-stage accumulator sums up the first columns of sampling areas P1, P2 and P3 to obtain a frequency domain conversion coefficient accumulation result Sum123 and outputs the frequency domain conversion coefficient accumulation result Sum123 to a later-stage accumulator, the second-stage accumulator sums up the second columns of sampling areas P4, P5 and P6 to obtain a frequency domain conversion coefficient accumulation result Sum456 and outputs the frequency domain conversion coefficient accumulation result Sum456 to a later-stage accumulator, the third-stage accumulator sums up the third columns of sampling areas P7, P8 and P9 to obtain a frequency domain conversion coefficient accumulation result Sum789 and outputs the frequency domain conversion coefficient accumulation result Sum789 to the later-stage accumulator, and the later-stage accumulator sums up the frequency domain conversion coefficient accumulation result Sum123, the frequency domain conversion coefficient accumulation result Sum456 and the frequency domain conversion coefficient accumulation result Sum789 to obtain frequency domain conversion coefficients Sum.
Optionally, the specified pixel-sized sampling region refers to a 128x128 pixel-sized sampling region.
Optionally, the parameter configuration module initializes the moving step lengths of the wheels to be arranged from large to small, the initial current moving step length is the largest moving step length, and the moving range is the size of the image along the moving direction.
Optionally, the moving direction is a Z-axis direction.
Optionally, the comparing and determining module determines that the moving range of the new current moving step is [ zi-2，zi+2]Wherein z isiAnd finding out the position with the best definition for the current moving step length of the current round, and obtaining the new current moving step length.
Optionally, when the motor of the image acquisition device is controlled by the focus control module through the motor control interface to move the camera to the current position according to the movement step length, the movement directions of the movement step lengths of two adjacent wheels are opposite.
Compared with the prior art, the invention has the following advantages: the image processing module sequentially outputs a plurality of sampling areas of each line of a cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on a plurality of sampling areas sequentially input into the same line in a pipeline mode, then performing accumulation, and summing the final accumulation results of the plurality of parallel pipelines to obtain a definition function value of the current position image. The invention optimizes the automatic focusing of the microscope by combining the parallel pipeline hardware acceleration of the FPGA with the control of the image acquisition device, and has the advantages of high focusing speed and high processing efficiency.
Drawings
FIG. 1 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a plurality of parallel pipelines and an accumulation unit according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of sampling region division according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating an example of a focusing process with multiple movement steps according to an embodiment of the present invention.
FIG. 5 is a schematic diagram of a basic control flow of an apparatus according to an embodiment of the present invention.
Fig. 6 is a detailed control flow diagram of an apparatus according to an embodiment of the present invention.
Detailed Description
The embodiment provides a camera focus control method based on an FPGA, which comprises the following steps:
1) initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
2) controlling a motor of the image acquisition device to move the camera to the current position according to the moving step length;
3) controlling and caching a current position image shot by a camera of the image acquisition device;
4) sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, wherein the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input into the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
5) judging whether the current position exceeds the moving range, and if not, skipping to execute the step 2); otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and skipping to execute the step 2); and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
As shown in fig. 1, the apparatus for controlling focusing of a camera based on FPGA of the present embodiment includes:
the parameter configuration module is used for initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
the focusing control module comprises a motor control interface which is used for being connected with a motor of the image acquisition device and is used for controlling the motor of the image acquisition device to move the camera to the current position according to the moving step length through the motor control interface;
the image acquisition module comprises an image acquisition interface which is connected with a camera of the image acquisition device and is used for acquiring and caching the current position image shot by the camera through the image acquisition interface;
the image processing module is used for sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input to the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
the comparison and judgment module is used for judging whether the current position exceeds the moving range or not, and if the current position does not exceed the moving range, the focusing control module is called to move the camera to a new current position according to a new current moving step length; otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and calling a focusing control module to move the camera to a new current position according to the new current moving step length; and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
As can be seen from fig. 1, the image acquisition device comprises a motor, a camera and a light source, wherein the motor can move under the control of the FPGA, so as to move the area-array camera and realize the focal length adjustment of the camera and the slide; a camera (area-array camera) for taking an image; the light source is used for polishing the object. In order to realize automatic light supplement by controlling the light source, the parameter configuration module in this embodiment further includes a light source control interface for connecting with the light source control end of the image acquisition device, and the parameter configuration module is further configured to configure the control parameters of the light source of the image acquisition device through the light source control interface.
The image processing module adopts a mode of replacing an original image with a sampling region for calculating the definition function value of the current position image, so that the calculated amount can be effectively reduced, and the calculation efficiency is improved.
In this embodiment, the image processing module includes an image collecting unit (not shown in the figure), a plurality of parallel pipelines, and an accumulating unit, where the image collecting unit is configured to sequentially output a plurality of sampling regions in each line of the cached current position image to the parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling regions in the current position image, each parallel pipeline is configured to sequentially input a plurality of sampling regions in the same line, perform frequency domain transformation in a pipeline manner, and then accumulate frequency domain transformation coefficients, and the accumulating unit is configured to sum final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a sharpness function value of the current position image.
As shown in fig. 2, the parallel pipeline includes: the line-frequency domain converter is used for performing frequency domain conversion aiming at the input sampling region; the buffer is used for buffering the frequency domain transformation result of each sampling region; the column frequency domain converter is used for performing column frequency domain conversion on the cached results of the line frequency domain conversion of each sampling region to obtain frequency domain conversion coefficients; the accumulation unit comprises a preceding-stage accumulator and a subsequent-stage accumulator, the preceding-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficients obtained by the various rows of frequency domain converters to obtain the frequency domain transformation coefficient accumulation result of each row of sampling areas, and the subsequent-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficient accumulation result of each row of sampling areas to obtain the accumulation results of all the sampling areas.
As shown in fig. 2, the image processing module in this embodiment includes three parallel pipelines L1, L2, and L3, the image capturing unit uniformly extracts 9 sampling regions of specified pixel sizes from the cached current position image, where the first row of sampling regions is P1, P4, and P7, the second row of sampling regions is P2, P5, and P8, and the third row of sampling regions is P3, P6, and P9, sequentially outputs the first row of sampling regions P1, P4, and P7 of the cached current position image to the parallel pipeline L1, sequentially outputs the second row of sampling regions P2, P5, and P8 of the cached current position image to the parallel pipeline L2, and sequentially outputs the third row of sampling regions P3, P6, and P9 of the cached current position image to the parallel pipeline L3; a line frequency domain converter of the parallel pipeline L1 sequentially performs line frequency domain conversion on first line sampling regions P1, P4 and P7 and buffers the line frequency domain conversion into corresponding buffers, a column frequency domain converter of the parallel pipeline L1 sequentially performs column frequency domain conversion on the buffered first line sampling regions P1, P4 and P7 and sends the results of the column frequency domain conversion into a preceding-stage accumulator, a line frequency domain converter of the parallel pipeline L2 sequentially performs line frequency domain conversion on second line sampling regions P2, P5 and P8 and buffers the results of the column frequency domain conversion into corresponding buffers, and a column frequency domain converter of the parallel pipeline L2 sequentially performs column frequency domain conversion on the buffered second line sampling regions P2, P5 and P8 and sends the results of the column frequency domain conversion into the preceding-stage accumulator; the row frequency domain converter of the parallel pipeline L3 sequentially performs row frequency domain conversion on the third row sampling regions P3, P6 and P9 and caches the third row sampling regions in corresponding caches, and the column frequency domain converter of the parallel pipeline L3 sequentially performs column frequency domain conversion on the cached third row sampling regions P3, P6 and P9 and sends the results of the column frequency domain conversion to the former-stage accumulator; the first-stage accumulator sums up the first columns of sampling areas P1, P2 and P3 to obtain a frequency domain conversion coefficient accumulation result Sum123 and outputs the frequency domain conversion coefficient accumulation result Sum123 to a later-stage accumulator, the second-stage accumulator sums up the second columns of sampling areas P4, P5 and P6 to obtain a frequency domain conversion coefficient accumulation result Sum456 and outputs the frequency domain conversion coefficient accumulation result Sum456 to a later-stage accumulator, the third-stage accumulator sums up the third columns of sampling areas P7, P8 and P9 to obtain a frequency domain conversion coefficient accumulation result Sum789 and outputs the frequency domain conversion coefficient accumulation result Sum789 to the later-stage accumulator, and the later-stage accumulator sums up the frequency domain conversion coefficient accumulation result Sum123, the frequency domain conversion coefficient accumulation result Sum456 and the frequency domain conversion coefficient accumulation result Sum789 to obtain frequency domain conversion coefficients Sum.
In this embodiment, the sampling region of the specified pixel size refers to a sampling region of 128 × 128 pixels.
In this embodiment, the parameter configuration module initializes and sets the moving step length of each wheel to be arranged from large to small, the initial current moving step length is the largest moving step length, and the moving range is the size of the image along the moving direction.
In this embodiment, the moving direction is the Z-axis direction.
In order to further narrow the moving range of each round, the comparing and determining module in this embodiment determines the moving range of the new current moving step as [ zi-2，zi+2]Wherein z isiAnd finding out the position with the best definition for the current moving step length of the current round, and obtaining the new current moving step length. For example, as shown in fig. 4, if the first round finds the position with the best definition to be 80 and the second round moves by 20 steps, the moving range of the second round move step is determined to be 60,100](ii) a The second round finds out the position with the best definition as 70, the third round moves the step size as 2, then determine the third round moves the step size and moves the range as 66,74]. In FIG. 4, each line frame is a roundAnd moving the moving position of the step length, acquiring an image at the position, calculating a definition function value, and obtaining a gray picture frame as the moving position with the best definition under the step length of the movement.
In order to further improve the moving efficiency of each round, in this embodiment, when the focus control module controls the motor of the image acquisition device to move the camera to the current position according to the moving step length through the motor control interface, the moving directions of the moving step lengths of two adjacent rounds are opposite. For example, as shown in fig. 4, the moving direction is a positive z-axis direction when the moving step of the first round is 40, the moving direction is a negative z-axis direction when the moving step of the second round is 10, and the moving direction is a positive z-axis direction when the moving step of the third round is 2, so that the homing operation of each round of the motor can be avoided, and the next round of the moving initial position can be directly entered. In this embodiment, a total of three wheels move by step size, so that the position of the final found 72 is the final focusing result.
As shown in fig. 5, the device for controlling the focusing of the camera based on the FPGA of the present embodiment is implemented by an FPGA image processing board based on an FPGA chip, and the main working process thereof is as follows: the slide position is kept fixed, and the camera is arranged on a motor platform; the FPGA controls the motor to move according to the step length, so that the focal length of the camera and the slide is changed; after the motor moves every time, the FPGA controls the camera to shoot, image data is collected, image processing is carried out, the definition function value of each image is calculated, and the clearest position is obtained after the current step length movement is finished; then adjusting the step length, and repeating the above processes again; and after the clearest position of the image under the minimum step length is obtained, the focusing of the camera is realized. The main working contents of the FPGA are as follows: the control motor moves, control light source switch, control camera shooting, image data acquisition buffering, image processing, definition comparison etc.. The acquisition module comprises a motor platform, an area-array camera and a light source; the area-array camera is installed on the motor platform and can move along with the movement of the motor, as shown in fig. 6, the specific working steps of the FPGA image processing board include:
1) the parameter configuration module is used for initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel; the parameter configuration module 1 is used for presetting various working parameters of the FPGA, and in the embodiment, the parameter configuration module also comprises light source intensity, motor movement times, image length and width, image processing coordinates and the like;
2) the focusing control module controls a motor of the image acquisition device to move the camera to the current position according to the moving step length through the motor control interface;
3) the image acquisition module acquires and caches a current position image shot by the camera through the image acquisition interface;
4) the image processing module sequentially outputs a plurality of sampling areas of each line of the cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input into the same line in a pipeline mode, then accumulating the sampling areas, and summing the final accumulation results of the plurality of parallel pipelines to obtain a definition function value of the current position image; in order to give consideration to both definition resolution and FPGA parallel pipeline implementation and represent each position of an image, 9 regions are uniformly selected from a 1920x1200 whole image, the size of each region is 128x128, row-column frequency domain processing calculation is carried out on each region, calculation results of the 9 regions are accumulated finally, the accumulated result of frequency domain transformation coefficients is used as a definition function value, theoretically, the clearer the image is, the richer specific details of the image are, the larger the transformation coefficient value after frequency domain transformation is, the larger the accumulated result of the transformation coefficients is, and conversely, the clearer the value is, the clearer the image is represented.
5) The comparison and judgment module judges whether the current position exceeds the moving range, and if the current position does not exceed the moving range, the comparison and judgment module jumps to execute the focusing control module; otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if so, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and jumping to the execution focusing control module; and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is directed to methods, apparatus (systems), and computer program products according to embodiments of the application wherein instructions, which execute via a flowchart and/or a processor of the computer program product, create means for implementing functions specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.
Claims (10)
1. A method for controlling camera focus based on FPGA is characterized by comprising the following steps:
1) initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
2) controlling a motor of the image acquisition device to move the camera to the current position according to the moving step length;
3) controlling and caching a current position image shot by a camera of the image acquisition device;
4) sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, wherein the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input into the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
5) judging whether the current position exceeds the moving range, and if not, skipping to execute the step 2); otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and skipping to execute the step 2); and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
2. An apparatus for controlling camera focus based on an FPGA, comprising:
the parameter configuration module is used for initializing and setting the moving step length, the initial current moving step length and the moving range of each wheel;
the focusing control module comprises a motor control interface which is used for being connected with a motor of the image acquisition device and is used for controlling the motor of the image acquisition device to move the camera to the current position according to the moving step length through the motor control interface;
the image acquisition module comprises an image acquisition interface which is connected with a camera of the image acquisition device and is used for acquiring and caching the current position image shot by the camera through the image acquisition interface;
the image processing module is used for sequentially outputting a plurality of sampling areas of each line of the cached current position image to parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling areas in the current position image, each parallel pipeline is used for performing frequency domain transformation on the plurality of sampling areas sequentially input to the same line in a pipeline mode, accumulating the frequency domain transformation coefficients, and summing the final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain a definition function value of the current position image;
the comparison and judgment module is used for judging whether the current position exceeds the moving range or not, and if the current position does not exceed the moving range, the focusing control module is called to move the camera to a new current position according to a new current moving step length; otherwise, comparing the definition function values of the images at the positions obtained under the moving step length of the round, and finding out the position with the best definition; judging whether a smaller moving step length still exists, if the smaller moving step length still exists, selecting the smaller moving step length as a new current moving step length, determining the moving range of the new current moving step length, and calling a focusing control module to move the camera to a new current position according to the new current moving step length; and judging that no smaller moving step length exists, and outputting the position with the best definition found out under the moving step length of the current round as a focusing result.
3. The device for controlling focusing of a camera based on FPGA of claim 2, wherein the parameter configuration module further comprises a light source control interface for connecting with a light source control end of the image capturing device, and the parameter configuration module is further configured to configure control parameters of a light source of the image capturing device through the light source control interface.
4. The FPGA-based camera focus control apparatus of claim 2, wherein the image processing module comprises an image acquisition unit, a plurality of parallel pipelines, and an accumulation unit, wherein the image acquisition unit is configured to sequentially output a plurality of sampling regions of each line of the cached current position image to the parallel pipelines, the number of the parallel pipelines corresponds to the number of lines of the sampling regions in the current position image, each parallel pipeline is configured to perform frequency domain transformation on a plurality of sampling regions sequentially input to the same line in a pipeline manner and then accumulate the frequency domain transformation coefficients, and the accumulation unit is configured to sum final accumulation results of the frequency domain transformation coefficients of the plurality of parallel pipelines to obtain the sharpness function value of the current position image.
5. The apparatus of FPGA-based camera focus control of claim 4, wherein the parallel pipeline comprises: the line-frequency domain converter is used for performing frequency domain conversion aiming at the input sampling region; the buffer is used for buffering the frequency domain transformation result of each sampling region; the column frequency domain converter is used for performing column frequency domain conversion on the cached results of the line frequency domain conversion of each sampling region to obtain frequency domain conversion coefficients; the accumulation unit comprises a preceding-stage accumulator and a subsequent-stage accumulator, the preceding-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficients obtained by the various rows of frequency domain converters to obtain the frequency domain transformation coefficient accumulation result of each row of sampling areas, and the subsequent-stage accumulator is used for accumulating and calculating the frequency domain transformation coefficient accumulation result of each row of sampling areas to obtain the accumulation results of all the sampling areas.
6. The apparatus of FPGA-based camera focus control of claim 5, wherein said image processing module comprises three parallel pipelines L1, L2, L3, said image capturing unit uniformly extracts 9 sampling regions of a specified pixel size from the buffered current position image, wherein a first line of sampling regions is P1, P4, P7, a second line of sampling regions is P2, P5, P8, a third line of sampling regions is P3, P6, P9, outputs the first line of sampling regions P1, P4, P7 of the buffered current position image to the parallel pipeline L1, outputs the second line of sampling regions P2, P5, P8 of the buffered current position image to the parallel pipeline L2, outputs the third line of sampling regions P3, P6, P9 of the buffered current position image to the parallel pipeline L3; a line frequency domain converter of the parallel pipeline L1 sequentially performs line frequency domain conversion on first line sampling regions P1, P4 and P7 and buffers the line frequency domain conversion into corresponding buffers, a column frequency domain converter of the parallel pipeline L1 sequentially performs column frequency domain conversion on the buffered first line sampling regions P1, P4 and P7 and sends the results of the column frequency domain conversion into a preceding-stage accumulator, a line frequency domain converter of the parallel pipeline L2 sequentially performs line frequency domain conversion on second line sampling regions P2, P5 and P8 and buffers the results of the column frequency domain conversion into corresponding buffers, and a column frequency domain converter of the parallel pipeline L2 sequentially performs column frequency domain conversion on the buffered second line sampling regions P2, P5 and P8 and sends the results of the column frequency domain conversion into the preceding-stage accumulator; the row frequency domain converter of the parallel pipeline L3 sequentially performs row frequency domain conversion on the third row sampling regions P3, P6 and P9 and caches the third row sampling regions in corresponding caches, and the column frequency domain converter of the parallel pipeline L3 sequentially performs column frequency domain conversion on the cached third row sampling regions P3, P6 and P9 and sends the results of the column frequency domain conversion to the former-stage accumulator; the first-stage accumulator sums up the first columns of sampling areas P1, P2 and P3 to obtain a frequency domain conversion coefficient accumulation result Sum123 and outputs the frequency domain conversion coefficient accumulation result Sum123 to a later-stage accumulator, the second-stage accumulator sums up the second columns of sampling areas P4, P5 and P6 to obtain a frequency domain conversion coefficient accumulation result Sum456 and outputs the frequency domain conversion coefficient accumulation result Sum456 to a later-stage accumulator, the third-stage accumulator sums up the third columns of sampling areas P7, P8 and P9 to obtain a frequency domain conversion coefficient accumulation result Sum789 and outputs the frequency domain conversion coefficient accumulation result Sum789 to the later-stage accumulator, and the later-stage accumulator sums up the frequency domain conversion coefficient accumulation result Sum123, the frequency domain conversion coefficient accumulation result Sum456 and the frequency domain conversion coefficient accumulation result Sum789 to obtain frequency domain conversion coefficients Sum.
7. The apparatus of claim 6, wherein the specified pixel size sampling area is a 128x128 pixel size sampling area.
8. The apparatus of claim 2, wherein the parameter configuration module initially sets the moving steps of each wheel to be arranged from large to small, the initial current moving step is the largest moving step, and the moving range is the size of the image along the moving direction.
9. The apparatus of claim 8, wherein the comparison module determines the new current moving step as a moving range [ z [ z ] ]i-2，zi+2]Wherein z isiAnd finding out the position with the best definition for the current moving step length of the current round, and obtaining the new current moving step length.
10. The FPGA-based camera focus control device of claim 9, wherein the focus control module controls the motor of the image capture device via the motor control interface to move the camera to the current position according to the moving step length, and the moving directions of the moving step lengths of two adjacent wheels are opposite.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011324968.1A CN112135062B (en) | 2020-11-24 | 2020-11-24 | Camera focusing control method and device based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011324968.1A CN112135062B (en) | 2020-11-24 | 2020-11-24 | Camera focusing control method and device based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112135062A true CN112135062A (en) | 2020-12-25 |
| CN112135062B CN112135062B (en) | 2021-02-05 |
Family
ID=73852255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011324968.1A Active CN112135062B (en) | 2020-11-24 | 2020-11-24 | Camera focusing control method and device based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112135062B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112988395A (en) * | 2021-04-20 | 2021-06-18 | 宁波兰茜生物科技有限公司 | Pathological analysis method and device of extensible heterogeneous edge computing framework |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102053339A (en) * | 2010-11-23 | 2011-05-11 | 天津市亚安科技电子有限公司 | Automatic focusing method |
| CN102457666A (en) * | 2010-10-19 | 2012-05-16 | 赵菲 | Automatic focusing device based on FPGA |
| CN103945133A (en) * | 2014-05-08 | 2014-07-23 | 山东神戎电子股份有限公司 | Auto-focus device and method for visible light lens |
| CN104284095A (en) * | 2014-10-28 | 2015-01-14 | 福建福光数码科技有限公司 | Quick and automatic focusing method and system for long-focus visible-light industrial lens |
| CN107102328A (en) * | 2017-04-17 | 2017-08-29 | 王辉 | Real time imagery signal processing method and FPGA based on FPGA |
| US20180359410A1 (en) * | 2017-06-08 | 2018-12-13 | Intel Corporation | Method and system of camera control and image processing with a multi-frame-based window for image data statistics |
| CN111866365A (en) * | 2019-04-29 | 2020-10-30 | 武汉精立电子技术有限公司 | Industrial camera capable of automatically focusing and control method |
-
2020
- 2020-11-24 CN CN202011324968.1A patent/CN112135062B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457666A (en) * | 2010-10-19 | 2012-05-16 | 赵菲 | Automatic focusing device based on FPGA |
| CN102053339A (en) * | 2010-11-23 | 2011-05-11 | 天津市亚安科技电子有限公司 | Automatic focusing method |
| CN103945133A (en) * | 2014-05-08 | 2014-07-23 | 山东神戎电子股份有限公司 | Auto-focus device and method for visible light lens |
| CN104284095A (en) * | 2014-10-28 | 2015-01-14 | 福建福光数码科技有限公司 | Quick and automatic focusing method and system for long-focus visible-light industrial lens |
| CN107102328A (en) * | 2017-04-17 | 2017-08-29 | 王辉 | Real time imagery signal processing method and FPGA based on FPGA |
| US20180359410A1 (en) * | 2017-06-08 | 2018-12-13 | Intel Corporation | Method and system of camera control and image processing with a multi-frame-based window for image data statistics |
| CN111866365A (en) * | 2019-04-29 | 2020-10-30 | 武汉精立电子技术有限公司 | Industrial camera capable of automatically focusing and control method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112988395A (en) * | 2021-04-20 | 2021-06-18 | 宁波兰茜生物科技有限公司 | Pathological analysis method and device of extensible heterogeneous edge computing framework |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112135062B (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8169486B2 (en) | Image acquisition method and apparatus | |
| JP5222472B2 (en) | Image processing apparatus, image restoration method, and program | |
| EP2560375B1 (en) | Image processing device, image capture device, program, and image processing method | |
| US20150103190A1 (en) | Image acquisition method and apparatus with mems optical image stabilization (ois) | |
| CN112135062B (en) | Camera focusing control method and device based on FPGA | |
| EP3062502A1 (en) | Blurless image capturing system | |
| JP3251127B2 (en) | Video data processing method | |
| KR20130073986A (en) | Fast repeated integral images | |
| JP4241814B2 (en) | Image correction apparatus and method, and electronic apparatus | |
| JP6094100B2 (en) | Moving object detection method | |
| US8121429B2 (en) | Image processing apparatus, image-capturing apparatus, image processing method, and program | |
| CN110738625A (en) | Image resampling method, device, terminal and computer readable storage medium | |
| JP3920659B2 (en) | AF evaluation value calculation device | |
| JP5616989B2 (en) | Image processing apparatus and threshold setting processing program | |
| WO2020237644A1 (en) | Fpga architecture-based realtime depth of field synthesis algorithm and system | |
| JP7121678B2 (en) | Image processing device, program and learning method | |
| CN116862953B (en) | Motion background-oriented real-time detection tracking device and method for small moving target | |
| JP4760484B2 (en) | Camera shake correction apparatus, camera shake correction method, and program | |
| JP3209420B2 (en) | Convolution processing method and apparatus | |
| CN102739924B (en) | Image processing method and system | |
| López-Martínez et al. | Fast image restoration algorithm based on camera microscanning | |
| CN114972298B (en) | Urban drainage pipeline video detection method and system | |
| CN117726921A (en) | FPGA-implemented ORB feature extraction accelerator based on stream processing and non-blocking | |
| JP4013722B2 (en) | Scanning imaging device | |
| CN117611461A (en) | Multi-frame moving image fusion method and system for robot vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 6-1-604, No.55 Fuye street, Luotuo street, Zhenhai District, Ningbo City, Zhejiang Province Patentee after: Lansi (Ningbo) Intelligent Technology Co.,Ltd. Address before: Room 6-1-604, No.55 Fuye street, Luotuo street, Zhenhai District, Ningbo City, Zhejiang Province Patentee before: Ningbo Lanxi Biotechnology Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20201225 Assignee: Changsha Lansi Intelligent Technology Co.,Ltd. Assignor: Lansi (Ningbo) Intelligent Technology Co.,Ltd. Contract record no.: X2023980033656 Denomination of invention: A method and device for camera focus control based on FPGA Granted publication date: 20210205 License type: Common License Record date: 20230321 |
|
| EE01 | Entry into force of recordation of patent licensing contract |