CN114845043A - Automatic focusing method, system, electronic device and computer readable storage medium - Google Patents
Automatic focusing method, system, electronic device and computer readable storage medium Download PDFInfo
- Publication number
- CN114845043A CN114845043A CN202210272066.0A CN202210272066A CN114845043A CN 114845043 A CN114845043 A CN 114845043A CN 202210272066 A CN202210272066 A CN 202210272066A CN 114845043 A CN114845043 A CN 114845043A
- Authority
- CN
- China
- Prior art keywords
- color
- face
- roi
- lens
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 210000001747 pupil Anatomy 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 210000000887 face Anatomy 0.000 description 3
- 230000009194 climbing Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 238000001559 infrared map Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/30—Transforming light or analogous information into electric information
- H04N5/33—Transforming infrared radiation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application relates to the technical field of automatic focusing, and discloses an automatic focusing method, an automatic focusing system, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: carrying out face detection on a color picture shot by a color lens of a camera, and determining a first face area and a pupil distance in the color picture; determining the distance between the face and the camera according to the pupil distance, the size of the first face area and the size of the color map; determining a second face area in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the position of the first face area and the size of the first face area; according to the distance between the face and the camera and the second face area, the infrared lens is automatically focused, an additional distance measuring device is not needed, the cost of the depth camera is reduced, the calculated amount is greatly reduced, quick and high-precision automatic focusing can be realized, and the precision of depth recovery is further improved.
Description
Technical Field
The present disclosure relates to the field of auto-focusing technologies, and in particular, to an auto-focusing method, an auto-focusing system, an electronic device, and a computer-readable storage medium.
Background
The depth camera can acquire the depth information of a target object in real time, and is technically supported for complex application scenes such as motion capture identification, face identification, three-dimensional modeling in the automatic driving field, cruising and obstacle avoidance, part scanning detection sorting in the industrial field, monitoring in the security field, people counting and the like, and has wide consumption level and industrial level application requirements.
However, the auto-focusing technology of the camera includes two implementations of active auto-focusing and passive auto-focusing: the active automatic focusing is realized according to the depth value of a focused object from an infrared lens, and a laser range finder or an ultrasonic range finder and other devices are required for ranging, so that the cost of a depth camera is increased by the additional devices; the passive automatic focusing is to focus by using the definition of an image formed by a focused object on an image plane, and an algorithm such as a hill climbing method or a gradient descent method is needed to iterate, that is, the focal length is adjusted one time until the image is the clearest, so that the calculation amount is large, the focusing speed is slow, the requirement of a depth camera on a high frame rate cannot be met, and the image shot by the depth camera has low definition.
Disclosure of Invention
An object of the embodiments of the present application is to provide an auto-focusing method, system, electronic device, and computer-readable storage medium, which do not require an additional distance measuring device, reduce the cost of a depth camera, greatly reduce the amount of computation, and enable fast and high-precision auto-focusing, thereby improving the precision of depth recovery.
In order to solve the above technical problem, an embodiment of the present application provides an auto-focusing method, including the following steps: carrying out face detection on a color image shot by a color lens of a camera, and determining a first face area and a pupil distance in the color image; determining the distance between the face and the camera according to the pupil distance, the size of the first face area and the size of the color image; determining a second face area in an infrared image shot by the infrared lens of the camera according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the position of the first face area and the size of the first face area; and automatically focusing the infrared lens according to the distance between the face and the camera and the second face area.
An embodiment of the present application further provides an auto-focusing system, including: the system comprises a shooting module, a face detection module, an estimation module and an execution module; the shooting module is used for shooting through a color lens of the camera to obtain a color image and shooting through an infrared lens of the camera to obtain an infrared image; the face detection module is used for carrying out face detection on the color image and determining a first face area and a pupil distance in the color image; the estimation module is used for determining the distance between a human face and a camera according to the interpupillary distance, the size of the first human face area and the size of the color image, and determining a second human face area in the infrared image according to the focal length of the color lens, the initial focal length of the infrared lens, the position of the first human face area and the size of the first human face area; the execution module is used for automatically focusing the infrared lens according to the distance between the face and the camera and the second face area.
An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described auto-focusing method.
Embodiments of the present application also provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the auto-focusing method described above.
The automatic focusing method, the system, the electronic device and the computer readable storage medium provided by the embodiment of the application perform face detection on a color image shot by a color lens of a camera, determine a first face area and a pupil distance in the color image, determine a distance between a face and the camera according to the pupil distance, the size of the first face area in the color image and the size of the color image, determine a second face area in an infrared image shot by the infrared lens of the same camera according to the focal length of the color lens, an initial focal length of the infrared lens of the same camera, the position of the first face area in the color image and the size of the first face area in the color image, and finally perform automatic focusing on the infrared lens according to the distance between the face and the camera and the second face area in the infrared image, and compared with the traditional active automatic focusing technology, the application does not need an additional distance measuring device, the cost of the depth camera is reduced, compared with an active automatic focusing technology for carrying out full-image depth information calculation at first, the pupil distance is used, the distance between a human face and the camera is estimated through a fitting formula by information such as the human face area size of a color image, the calculated amount is greatly reduced, calculation resources are saved, compared with a passive automatic focusing technology, iterative focusing is not needed, quick and high-precision automatic focusing can be realized, the high frame rate requirement of the depth camera can be met, the quality of an infrared image and a speckle pattern shot by an infrared lens of the camera is improved, and the precision of depth recovery is further improved.
In addition, the determining a second face region in the infrared image captured by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the position of the first face region, and the size of the first face region includes: determining coordinates of a first reference point in the first face area, and taking the coordinates of the first reference point as the position of the first face area; the first face area is a rectangle, and the first reference point is any one of vertexes of the rectangle; determining the coordinate of a second reference point in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera and the coordinate of the first reference point; determining a target width and a target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face area and the height of the first face area; determining a second face area in the infrared image according to the coordinates of the second datum point, the target width and the target height; according to the embodiment of the application, when the server estimates the face area in the infrared image, the server can determine the second reference point in the infrared image according to the position of the first reference point in the first face area, in combination with the parameters of the color lens and the infrared lens, in combination with the width and the height of the first face area and in combination with the parameters of the color lens and the infrared lens, and finally extend in the corresponding direction by taking the coordinate of the second reference point as a reference, so that the second face area in the infrared image is obtained, and the second face area estimated in this way is more reasonable, accurate and reliable, so that the accuracy and the effect of automatic focusing of the subsequent infrared lens are improved.
In addition, the determining coordinates of a second reference point in the infrared map according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, and the coordinates of the first reference point includes: determining the coordinate of a second reference point in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the coordinate of the first reference point and a preset offset; determining a target width and a target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face region and the height of the first face region, including: according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face region, the height of the first face region and the preset offset, the target width and the target height are determined, and because the initial focal length of the infrared lens is used when the second face region is estimated, some deviation may exist with a real face region in an infrared image, therefore, when the coordinate, the target width and the target height of the second reference point are determined, the preset offset is taken into consideration, the preset offset is reliable empirical data, and the deviation caused by the reasons can be eliminated.
In addition, the distance between the face and the camera is determined according to the interpupillary distance, the size of the first face area and the size of the color image through a preset fitting model, and the fitting model is obtained through the following steps: obtaining a plurality of training samples; each training sample comprises a group of corresponding sample color images and sample depth images; performing face detection on the sample color image, and determining a face area and a pupil distance of a face in the sample color image; the method comprises the steps that a sample depth map is fitted according to the interpupillary distance of a face in a sample color map, the size of a face area in the sample color map and the size of the sample color map to obtain a fitting model, and a server performs fitting in advance based on massive training samples to obtain the fitting model for determining the distance between the face and a camera.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
FIG. 1 is a flow diagram of an auto-focus method according to one embodiment of the present application;
FIG. 2 is a first flowchart for determining a second face region in an infrared image captured by an infrared lens according to a focal length of a color lens, an initial focal length of an infrared lens of a camera, a position of a first face region, and a size of the first face region, according to an embodiment of the present application;
FIG. 3 is a second flowchart for determining a second face region in an infrared image captured by an infrared lens according to a focal length of a color lens, an initial focal length of an infrared lens of a camera, a position of a first face region, and a size of the first face region, according to another embodiment of the present application;
FIG. 4 is a flow chart for obtaining a fitting model according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an autofocus system according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in the examples of the present application, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present application, and the embodiments may be mutually incorporated and referred to without contradiction.
The automatic focusing technology of the camera includes two implementation modes of active automatic focusing and passive automatic focusing, the active automatic focusing is realized by the camera according to the depth value of a focused object from an infrared lens, the camera needs to measure the depth value of the object according to devices such as a laser range finder or an ultrasonic range finder, and although the focusing speed is higher according to the active automatic focusing technology of the laser range finder or the ultrasonic range finder, the additional range finder is expensive, and the cost of the depth camera is increased.
For a depth camera, depth information required by active auto-focusing can be acquired by the depth camera itself, but the acquisition process generally finishes acquiring all depth information of a depth map before a target area is determined, that is, depth recovery and depth information calculation are performed on the whole image in advance, and all depth information is stored, which causes that the calculation amount of the focusing process is very large, the focusing speed is very slow, and the requirement of the depth camera on high frame rate cannot be met, thereby directly causing that the image shot by the camera has low definition.
The passive automatic focusing is to focus by using the definition of an image formed by a focused object on an image plane, and generally requires to use algorithms such as a hill climbing method or a gradient descent method to iterate, that is, to adjust the focal length once until the image is the clearest, although an additional distance measuring device is not needed, the calculation amount of the focusing process is large, the focusing speed is slow, and the high frame rate requirement of a depth camera cannot be met, so that the definition of the image shot by the camera is low directly.
In order to solve the above problems of slow focusing speed, high camera cost, and incapability of meeting the high frame rate requirement of a depth camera, an embodiment of the present application provides an auto-focusing method, which is applied to an electronic device, where the electronic device may be a terminal or a server, and the electronic device in this embodiment and in the following embodiments is described by taking the server as an example.
The specific flow of the auto-focusing method of the present embodiment can be shown in fig. 1, and includes:
Specifically, when the server performs auto-focusing on the infrared lens of the camera, the color image captured by the color lens of the camera and the infrared image captured by the infrared lens may be obtained first, and the color image and the infrared image obtained by the server are corresponding to each other.
In one example, the depth camera is provided with a color lens and an infrared lens, the color lens of the depth camera is a fixed focus lens, the focal length is unchangeable and unadjustable, the infrared lens of the depth camera is a zoom lens, the focal length can be adjusted, and when the server needs to automatically focus the infrared lens of the depth camera, the server firstly shoots a target object through the color lens of the camera to obtain a color image, and then shoots the same target object through the infrared lens to obtain an infrared image corresponding to the color image.
In specific implementation, the server may perform face detection on the color image shot by the color lens of the camera according to a preset face detection algorithm to determine a first face area in the color image and a pupil distance of a face in the color image, where the preset face detection algorithm may be set by a person skilled in the art according to actual needs.
In the specific implementation, a server detects a Region Of Interest (ROI) Of a face in a color image shot by a color lens Of a camera according to a preset face detection algorithm, that is, detects a first face Region in the color image, the first face Region detected by the server may be a minimum rectangle containing the face in the color image, the server may acquire position coordinates Of the first face Region, determine a width Of the first face Region and a height Of the first face Region, after the first face Region in the color image is detected, the server may determine positions Of two eyes Of the face in the first face Region, and calculate a pupil distance Of the face according to a distance between the two eyes.
And step 102, determining the distance between the face and the camera according to the pupil distance, the size of the first face area and the size of the color map.
Specifically, after determining a first face area and a pupil distance in the color map, the server may determine a distance between the face and the camera according to the pupil distance, the size of the first face area, and the size of the color map.
In one example, the server determines the distance between the face and the camera according to the pupil distance, the size of the first face area and the size of the color map, which can be implemented by the following formula:
wherein, a is a preset constant coefficient, the preset constant coefficient can be set by a person skilled in the art according to experience, d _ eye is the interpupillary distance of the human face, Color _ W is the width of the Color image, Color _ H is the width of the Color image, Color _ ROI _ W is the width of the first face region, Color _ ROI _ H is the height of the first face region, Z is the height of the first face region camera And determining the distance between the face and the camera for the server.
And 103, determining a second face area in the infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the position of the first face area and the size of the first face area.
In the specific implementation, considering that the traditional three-image alignment of a color image, an infrared image and a depth image needs to be solved by pixel-by-pixel alignment depending on internal and external parameters of an infrared lens, internal and external parameters of a color lens and depth values of corresponding pixel points, the method only needs to estimate a rough face region ROI in the infrared image, and a server only needs to estimate a second face region in the infrared image according to the focal length of the color lens, the initial focal length of the infrared lens, the position of a first face region, the size of the first face region and a preset estimation formula.
And 104, automatically focusing the infrared lens according to the distance between the face and the camera and the second face area.
Specifically, after the server determines the distance between the face and the camera and the second face area in the infrared image, the server may perform automatic focusing on the infrared lens and adjust the focal length of the infrared lens according to the distance between the face and the camera and the second face area based on a preset focusing formula, where the preset focusing formula may be set and selected by a person skilled in the art according to actual needs, and the embodiment of the present application is not specifically limited thereto.
In this embodiment, a color image shot by a color lens of a camera is subjected to face detection, a first face area and a pupil distance in the color image are determined, a distance between a face and the camera is determined according to the pupil distance, the size of the first face area in the color image and the size of the color image, then a second face area in an infrared image shot by the infrared lens of the same camera is determined according to a focal length of the color lens, an initial focal length of the infrared lens of the same camera, a position of the first face area in the color image and the size of the first face area in the color image, and finally an infrared lens is subjected to auto-focusing according to the distance between the face and the camera and the second face area in the infrared image, compared with the conventional active auto-focusing technology, the method does not need an additional distance measuring device, reduces the cost of a depth camera, and compared with the active auto-focusing technology of advanced full-image depth information calculation, this application uses information such as interpupillary distance, the face area size of colored drawing to estimate the distance between people's face and the camera through the fitting formula, the amount of calculation that has significantly reduced has practiced thrift computing resource, compare in passive form auto focus technique, this application need not the iteration and focuses, can realize quick and the auto focus of high accuracy, can satisfy the high frame rate demand of degree of depth camera, promote the quality of infrared picture and the speckle pattern that the infrared camera lens of camera shot, and then promote the precision that the degree of depth resumes.
In an embodiment, the size of the first face area includes a width of the first face area and a height of the first face area, and the server determines the second face area in the infrared image captured by the infrared lens according to a focal length of the color lens, an initial focal length of an infrared lens of the camera, a position of the first face area, and a size of the first face area, which may be implemented by the steps shown in fig. 2, and specifically includes:
in step 201, coordinates of a first reference point are determined in the first face area, and the coordinates of the first reference point are used as the position of the first face area.
In a specific implementation, the first face area detected by the server is a minimum rectangle containing a face in the color map, the server may determine a first reference point in the first face area, acquire coordinates of the first reference point, and use the coordinates of the first reference point as a position of the first face area, where the first reference point may be any one of vertices of the rectangle.
And step 202, determining the coordinate of a second reference point in the infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera and the coordinate of the first reference point.
Specifically, after the server determines the coordinates of the first reference point, the server may determine the position coordinates of a second reference point in an infrared image captured by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, and the coordinates of the first reference point.
In one example, the server may determine the coordinates of the second reference point in the infrared image captured by the infrared lens by the following formula:
IR_ROI(i)＝[F 1 *Color_ROI(i)]/F 2 ，IR_ROI(j)＝[F 1 *Color_ROI(j)]/F 2
in the formula, F 1 Is the focal length of the color lens, F 2 For the initial focal length of the IR lens, Color _ ROI (i) is the abscissa of the first fiducial, Color _ ROI (j) is the ordinate of the first fiducial, IR _ ROI (i) is the abscissa of the second fiducial, and IR _ ROI (j) is the ordinate of the second fiducial.
And step 203, determining the target width and the target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face area and the height of the first face area.
Specifically, after the server determines the position coordinates of the second reference point in the infrared image, the server may continue to determine the target width and the target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face region, and the height of the first face region.
In one example, the server may determine the target width and the target height by the following formulas:
IR_ROI_W＝[F 1 *Color_ROI_W]/F 2 ，IR_ROI_H＝[F 1 *Color_ROI_H]/F 2
in the formula, F 1 Is the focal length of the color lens, F 2 For the initial focal length of the infrared lens, Color _ ROI _ W is the width of the first face region, Color _ ROI _ H is the height of the first face region, IR _ ROI _ W is the target width, and IR _ ROI _ H is the target height.
And step 204, determining a second face area in the infrared image according to the coordinates, the target width and the target height of the second reference point.
Specifically, after the server determines the target width and the target height, the server may determine the second face area in the infrared image according to the coordinates of the second reference point, the target width, and the target height.
In one example, the first reference point is a top left corner vertex of the first face region, so that a second reference point determined by the server based on the first reference point should also be a top left corner vertex of the second face region, and the server extends rightward for a target width number of pixel points and extends downward for a target height number of pixel points from the second reference point to obtain a rectangle, that is, the second face region.
This embodiment, the server is when estimating the face area in the infrared picture, can be earlier according to the position of the first benchmark in the first face area, the parameter of colored camera lens and infrared camera lens is combined again, confirm the second benchmark in the infrared picture, again according to the width and the height in first face area, and combine the parameter of colored camera lens and infrared camera lens, confirm the due width and the height in second face area, use the coordinate of second benchmark again at last, it should have the direction to extend to, thereby obtain the second face area in the infrared picture, the second face area who calculates like this is more reasonable, it is accurate, reliable, thereby promote the precision and the effect of the auto focus of follow-up infrared camera lens.
In another embodiment, the size of the first face area includes a width of the first face area and a height of the first face area, and the server determines the second face area in the infrared image captured by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the position of the first face area, and the size of the first face area, which may be implemented by the steps shown in fig. 3, and specifically includes:
Step 301 is substantially the same as step 201, and is not described herein again.
Specifically, after the server determines the coordinates of the first reference point, the server may determine the coordinates of the second reference point in the infrared image according to the focal length of the color lens, the initial focal length of the infrared lens, and the coordinates of the first reference point, and by combining a preset offset.
In one example, the server may determine the coordinates of the second reference point in the infrared map captured by the infrared lens by the following formula:
IR_ROI(i)＝{[F 1 *Color_ROI(i)]/F 2 }-S，IR_ROI(j)＝{[F 1 *Color_ROI(j)]/F 2 }-S
in the formula, F 1 Is the focal length of the color lens, F 2 The initial focal length of the infrared lens, S is a preset offset, Color _ roi (i) is the abscissa of the first reference point, Color _ roi (j) is the ordinate of the first reference point, IR _ roi (i) is the abscissa of the second reference point, and IR _ roi (j) is the ordinate of the second reference point.
Specifically, after the server determines the coordinates of the second reference point, the server may determine the target width and the target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face area, the height of the first face area, and in combination with a preset offset.
In one example, the server may determine the target width and the target height by the following formulas:
IR_ROI_W＝{[F 1 *Color_ROI_W]/F 2 }+2S，IR_ROI_H＝{[F 1 *Color_ROI_H]/F 2 }+2S
in the formula, F 1 Is the focal length of the color lens, F 2 The initial focal length of the infrared lens is set, S is a preset offset, Color _ ROI _ W is the width of the first face region, Color _ ROI _ H is the height of the first face region, IR _ ROI _ W is a target width, and IR _ ROI _ H is a target height.
And step 304, determining a second face area in the infrared image according to the coordinates, the target width and the target height of the second reference point.
Step 304 is substantially the same as step 204, and is not described herein again.
In this embodiment, since the initial focal length of the infrared lens is used when estimating the second face region, there may be some deviation from the real face region in the infrared image, and therefore, when determining the coordinates, the target width, and the target height of the second reference point, the embodiment of the present application takes the preset offset into consideration, and the preset offset is reliable empirical data, and the deviation caused by the above reasons can be eliminated.
In an embodiment, the server may determine the distance between the face and the camera according to the interpupillary distance, the size of the first face area, and the size of the color map through a preset fitting model, and the fitting model used by the server may be obtained through the steps shown in fig. 4, which specifically includes:
and step 403, fitting the sample depth map according to the interpupillary distance of the face in the sample color map, the size of the face area in the sample color map and the size of the sample color map to obtain a fitting model.
In the specific implementation, the server can acquire a plurality of training samples in advance, each training sample comprises a group of corresponding sample color images and sample depth images, the server performs face detection on the sample color images to determine face areas and pupil distances of faces in the sample color images, the sample depth images corresponding to the sample color images are fitted according to the pupil distances of the faces, the sizes of the face areas in the sample color images and the sizes of the sample color images to obtain fitting models, and the server performs fitting in advance based on massive training samples to obtain the fitting models for determining the distances between the faces and the cameras.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
Another embodiment of the present application relates to an autofocus system, and the implementation details of the autofocus system of the present embodiment are specifically described below, and the following are provided only for the convenience of understanding, and are not necessary for implementing the present embodiment, and a schematic diagram of the autofocus system of the present embodiment may be as shown in fig. 5, and includes: a photographing module 501, a face detection module 502, an estimation module 503 and an execution module 504.
The shooting module 501 is configured to shoot through a color lens of a camera to obtain a color image, and shoot an infrared image through an infrared lens of the camera.
The face detection module 502 is configured to perform face detection on the color map, and determine a first face area and a pupil distance in the color map.
The estimation module 503 is configured to determine a distance between the face and the camera according to the interpupillary distance, the size of the first face area, and the size of the color map, and determine a second face area in the infrared image according to the focal length of the color lens, the initial focal length of the infrared lens, the position of the first face area, and the size of the first face area.
The execution module 504 is configured to perform auto-focusing on the infrared lens according to a distance between the face and the camera and the second face area.
It should be noted that, all the modules involved in this embodiment are logic modules, and in practical application, one logic unit may be one physical unit, may also be a part of one physical unit, and may also be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, a unit that is not so closely related to solving the technical problem proposed by the present application is not introduced in the present embodiment, but this does not indicate that there is no other unit in the present embodiment.
Another embodiment of the present application relates to an electronic device, as shown in fig. 6, including: at least one processor 601; and a memory 602 communicatively coupled to the at least one processor 601; the memory 602 stores instructions executable by the at least one processor 601, and the instructions are executed by the at least one processor 601 to enable the at least one processor 601 to execute the auto-focusing method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
Another embodiment of the present application relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the present application, and that various changes in form and details may be made therein without departing from the spirit and scope of the present application in practice.
Claims (10)
1. An auto-focusing method, comprising:
carrying out face detection on a color image shot by a color lens of a camera, and determining a first face area and a pupil distance in the color image;
determining the distance between the face and a camera according to the interpupillary distance, the size of the first face area and the size of the color map;
determining a second face area in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of an infrared lens of the camera, the position of the first face area and the size of the first face area;
and automatically focusing the infrared lens according to the distance between the face and the camera and the second face area.
2. The auto-focusing method according to claim 1, wherein the size of the first face region includes a width of the first face region and a height of the first face region, and the determining a second face region in an infrared image captured by the infrared lens according to the focal length of the color lens, an initial focal length of an infrared lens of the camera, a position of the first face region, and the size of the first face region comprises:
determining the coordinates of a first reference point in the first face area, and taking the coordinates of the first reference point as the position of the first face area; the first face area is a rectangle, and the first reference point is any one of vertexes of the rectangle;
determining the coordinate of a second reference point in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera and the coordinate of the first reference point;
determining a target width and a target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face area and the height of the first face area;
determining a second face area in the infrared image according to the coordinates of the second datum point, the target width and the target height; wherein the target width is a width of the second face region, and the target height is a height of the second face region.
3. The auto-focusing method according to claim 2, wherein the coordinates of the second reference point in the infrared image photographed by the infrared lens are determined by the following formula:
IR_ROI(i)＝[F 1 *Color_ROI(i)]/F 2 ，IR_ROI(j)＝[F 1 *Color_ROI(j)]/F 2
wherein, F 1 Is the focal length of the color lens, F 2 (ii) is the initial focal length of the infrared lens of the camera, Color _ roi (i) is the abscissa of the first reference point, Color _ roi (j) is the ordinate of the first reference point, IR _ roi (i) is the abscissa of the second reference point, IR _ roi (j) is the ordinate of the second reference point;
determining the target width and the target height by the following formula:
IR_ROI_W＝[F 1 *Color_ROI_W]/F 2 ，IR_ROI_H＝[F 1 *Color_ROI_H]/F 2
wherein, Color _ ROI _ W is a width of the first face region, Color _ ROI _ H is a height of the first face region, IR _ ROI _ W is the target width, and IR _ ROI _ H is the target height.
4. The auto-focusing method of claim 2, wherein determining coordinates of a second reference point in an infrared image captured by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, and the coordinates of the first reference point comprises:
determining the coordinate of a second reference point in an infrared image shot by the infrared lens according to the focal length of the color lens, the initial focal length of the infrared lens of the camera, the coordinate of the first reference point and a preset offset;
determining a target width and a target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face region and the height of the first face region, including:
and determining the target width and the target height according to the focal length of the color lens, the initial focal length of the infrared lens, the width of the first face area, the height of the first face area and the preset offset.
5. The auto-focusing method according to claim 4, wherein the coordinates of the second reference point in the infrared image taken by the infrared lens are determined by the following formula:
IR_ROI(i)＝{[F 1 *Color_ROI(i)]/F 2 }-S，IR_ROI(j)＝{[F 1 *Color_ROI(j)]/F 2 }-S
wherein, F 1 Is the focal length of the color lens, F 2 An initial focal length of an infrared lens of the camera, S the preset offset, Color _ roi (i) the abscissa of the first reference point, Color _ roi (j) the ordinate of the first reference point, IR _ roi (i) the abscissa of the second reference point, IR _ roi (j) the ordinate of the second reference point;
determining the target width and the target height by the following formula:
IR_ROI_W＝{[F 1 *Color_ROI_W]/F 2 }+2S，IR_ROI_H＝{[F 1 *Color_ROI_H]/F 2 }+2S
wherein, Color _ ROI _ W is a width of the first face region, Color _ ROI _ H is a height of the first face region, IR _ ROI _ W is the target width, and IR _ ROI _ H is the target height.
6. The auto-focusing method according to any one of claims 1 to 5, wherein the distance between the face and the camera is determined from the interpupillary distance, the size of the first face region and the size of the color map by the following formula:
wherein, A is a preset constant coefficient, d _ eye is the interpupillary distance, Color _ W is the width of the Color image, Color _ H is the width of the Color image, and Color _ ROI _ W is the secondA width of a face region, Color _ ROI _ H being a height of the first face region, Z camera Is the distance between the face and the camera.
7. The auto-focusing method of any one of claims 1 to 5, wherein the distance between the face and the camera is determined according to the interpupillary distance, the size of the first face region and the size of the color map by a preset fitting model, and the fitting model is obtained by the following steps:
obtaining a plurality of training samples; each training sample comprises a group of corresponding sample color images and sample depth images;
performing face detection on the sample color image, and determining a face area and a pupil distance of a face in the sample color image;
and fitting the sample depth map according to the interpupillary distance of the face in the sample color map, the size of the face area in the sample color map and the size of the sample color map to obtain a fitting model.
8. An auto-focus system, comprising: the system comprises a shooting module, a face detection module, an estimation module and an execution module;
the shooting module is used for shooting through a color lens of the camera to obtain a color image and shooting through an infrared lens of the camera to obtain an infrared image;
the face detection module is used for carrying out face detection on the color image and determining a first face area and a pupil distance in the color image;
the estimation module is used for determining the distance between a human face and a camera according to the interpupillary distance, the size of the first human face area and the size of the color image, and determining a second human face area in the infrared image according to the focal length of the color lens, the initial focal length of the infrared lens, the position of the first human face area and the size of the first human face area;
the execution module is used for automatically focusing the infrared lens according to the distance between the face and the camera and the second face area.
9. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the auto-focus method of any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the auto-focusing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210272066.0A CN114845043B (en) | 2022-03-18 | 2022-03-18 | Automatic focusing method, system, electronic device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210272066.0A CN114845043B (en) | 2022-03-18 | 2022-03-18 | Automatic focusing method, system, electronic device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114845043A true CN114845043A (en) | 2022-08-02 |
| CN114845043B CN114845043B (en) | 2024-03-15 |
Family
ID=82561596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210272066.0A Active CN114845043B (en) | 2022-03-18 | 2022-03-18 | Automatic focusing method, system, electronic device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114845043B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070206938A1 (en) * | 2006-03-02 | 2007-09-06 | Fujifilm Corporation | Distance measuring apparatus and method |
| CN101487703A (en) * | 2009-02-13 | 2009-07-22 | 浙江工业大学 | Fast full-view stereo photography measuring apparatus |
| US20150003819A1 (en) * | 2013-06-28 | 2015-01-01 | Nathan Ackerman | Camera auto-focus based on eye gaze |
| CN106412433A (en) * | 2016-10-09 | 2017-02-15 | 深圳奥比中光科技有限公司 | Automatic focusing method and system based on RGB-IR depth camera |
| CN108401114A (en) * | 2012-05-07 | 2018-08-14 | 株式会社尼康 | Focus detection device |
| CN111084603A (en) * | 2019-11-20 | 2020-05-01 | 北京健康有益科技有限公司 | Pupil distance measuring method and system based on depth camera |
| WO2021052487A1 (en) * | 2019-09-18 | 2021-03-25 | 华为技术有限公司 | Method and apparatus for obtaining extended depth of field image, and electronic device |
| CN113780045A (en) * | 2020-12-21 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Method and apparatus for training distance prediction model |
| CN113822920A (en) * | 2021-09-29 | 2021-12-21 | 北京的卢深视科技有限公司 | Method for acquiring depth information by structured light camera, electronic equipment and storage medium |
-
2022
- 2022-03-18 CN CN202210272066.0A patent/CN114845043B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070206938A1 (en) * | 2006-03-02 | 2007-09-06 | Fujifilm Corporation | Distance measuring apparatus and method |
| CN101487703A (en) * | 2009-02-13 | 2009-07-22 | 浙江工业大学 | Fast full-view stereo photography measuring apparatus |
| CN108401114A (en) * | 2012-05-07 | 2018-08-14 | 株式会社尼康 | Focus detection device |
| US20150003819A1 (en) * | 2013-06-28 | 2015-01-01 | Nathan Ackerman | Camera auto-focus based on eye gaze |
| CN106412433A (en) * | 2016-10-09 | 2017-02-15 | 深圳奥比中光科技有限公司 | Automatic focusing method and system based on RGB-IR depth camera |
| WO2021052487A1 (en) * | 2019-09-18 | 2021-03-25 | 华为技术有限公司 | Method and apparatus for obtaining extended depth of field image, and electronic device |
| CN111084603A (en) * | 2019-11-20 | 2020-05-01 | 北京健康有益科技有限公司 | Pupil distance measuring method and system based on depth camera |
| CN113780045A (en) * | 2020-12-21 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Method and apparatus for training distance prediction model |
| CN113822920A (en) * | 2021-09-29 | 2021-12-21 | 北京的卢深视科技有限公司 | Method for acquiring depth information by structured light camera, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 方淑仙等: "基于生物形状知识的人眼定位方法", 计算机系统应用, no. 10 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114845043B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105627932B (en) | A kind of distance measuring method and device based on binocular vision | |
| US9998650B2 (en) | Image processing apparatus and image pickup apparatus for adding blur in an image according to depth map | |
| CN107087107B (en) | Image processing apparatus and method based on dual camera | |
| CN109151439B (en) | Automatic tracking shooting system and method based on vision | |
| US8111910B2 (en) | Stereoscopic image processing device, method, recording medium and stereoscopic imaging apparatus | |
| JP6663040B2 (en) | Depth information acquisition method and apparatus, and image acquisition device | |
| CN106960454B (en) | Depth of field obstacle avoidance method and equipment and unmanned aerial vehicle | |
| CN110799921A (en) | Shooting method and device and unmanned aerial vehicle | |
| JP2017112602A (en) | Image calibrating, stitching and depth rebuilding method of panoramic fish-eye camera and system thereof | |
| CN106529495A (en) | Obstacle detection method of aircraft and device | |
| KR20170056698A (en) | Autofocus method, device and electronic apparatus | |
| WO2019105261A1 (en) | Background blurring method and apparatus, and device | |
| US20150178595A1 (en) | Image processing apparatus, imaging apparatus, image processing method and program | |
| WO2023072030A1 (en) | Automatic focusing method and apparatus for lens, and electronic device and computer-readable storage medium | |
| CN113052907B (en) | Positioning method of mobile robot in dynamic environment | |
| JP7378219B2 (en) | Imaging device, image processing device, control method, and program | |
| CN103824303A (en) | Image perspective distortion adjusting method and device based on position and direction of photographed object | |
| CN109902675B (en) | Object pose acquisition method and scene reconstruction method and device | |
| CN116051736A (en) | Three-dimensional reconstruction method, device, edge equipment and storage medium | |
| CN113538538B (en) | Binocular image alignment method, electronic device, and computer-readable storage medium | |
| US9635245B2 (en) | Focus position detection device and focus position detection method | |
| CN117058183A (en) | Image processing method and device based on double cameras, electronic equipment and storage medium | |
| CN114845043B (en) | Automatic focusing method, system, electronic device and computer readable storage medium | |
| CN114677264B (en) | Image alignment method, system, electronic device and storage medium | |
| CN113959398B (en) | Distance measurement method and device based on vision, drivable equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |