CN107707807B - Image acquisition control method and device - Google Patents

Image acquisition control method and device Download PDF

Info

Publication number
CN107707807B
CN107707807B CN201510546276.4A CN201510546276A CN107707807B CN 107707807 B CN107707807 B CN 107707807B CN 201510546276 A CN201510546276 A CN 201510546276A CN 107707807 B CN107707807 B CN 107707807B
Authority
CN
China
Prior art keywords
imaging unit
sub
lens
imaging
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510546276.4A
Other languages
Chinese (zh)
Other versions
CN107707807A (en
Inventor
周梁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zhigu Ruituo Technology Services Co Ltd
Original Assignee
Beijing Zhigu Ruituo Technology Services Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zhigu Ruituo Technology Services Co Ltd filed Critical Beijing Zhigu Ruituo Technology Services Co Ltd
Priority to CN201510546276.4A priority Critical patent/CN107707807B/en
Publication of CN107707807A publication Critical patent/CN107707807A/en
Application granted granted Critical
Publication of CN107707807B publication Critical patent/CN107707807B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The application discloses an image acquisition control method and device, wherein the method comprises the following steps: determining a desired sharp imaging spatial region of a scene; tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region; the adjusted light field camera performs image acquisition on the scene. The method and the device are beneficial to realizing customized image acquisition of a desired clear imaging space region in a scene, and can better meet diversified practical application requirements.

Description

Image acquisition control method and device
Technical Field
The present application relates to the field of terminal technologies, and in particular, to a method and an apparatus for controlling image acquisition.
Background
With the continuous development of image acquisition technology, equipment supporting image acquisition is continuously developed, and people have more and more personalized requirements on acquired images.
Different from the traditional camera, the light field camera usually arranges a sub-lens array between a main lens and an imaging unit array such as a CCD (charge coupled device), the sub-lens array records the light field information of a scene to be shot in different directions on a focal plane of the sub-lens array, the four-dimensional light field information of a three-dimensional scene such as space, visual angle and the like can be recorded through single exposure, the 'shooting first and focusing later' is supported, the shot image is processed to generate rich image effect, and various imaging applications such as digital refocusing, visual angle change, depth image, three-dimensional reconstruction and the like can be met. Currently, research on image acquisition technologies based on light field cameras is drawing more and more attention from industry.
Disclosure of Invention
The following presents a simplified summary of the application in order to provide a basic understanding of some aspects of the application. It should be understood that this summary is not an exhaustive overview of the present application. It is not intended to identify key or critical elements of the application or to delineate the scope of the application. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.
The embodiment of the application provides an image acquisition control method and device.
In a first aspect, an embodiment of the present application provides an image acquisition control method, including:
determining a desired sharp imaging spatial region of a scene;
tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
the adjusted light field camera performs image acquisition on the scene.
In a second aspect, an embodiment of the present application further provides an image capture control apparatus, including:
a desired sharp imaging spatial region determining module for determining a desired sharp imaging spatial region of the scene;
a tilt control module for tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a mean circle of confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
and the image acquisition control module is used for acquiring the images of the scene by the adjusted light field camera.
In a third aspect, an embodiment of the present application further provides another image capture control apparatus, including:
the system comprises a processor, a communication interface, a memory and a communication bus; the processor, the communication interface and the memory complete mutual communication through the communication bus;
the memory is used for storing at least one instruction; the instructions cause the processor to:
determining a desired sharp imaging spatial region of a scene;
tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
the adjusted light field camera performs image acquisition on the scene.
According to the technical scheme provided by the embodiment of the application, in the process of image acquisition application of a scene based on a light field camera, a desired clear imaging space region of the scene can be determined, and the size of an average circle of confusion of at least one first part of the scene, which is located in the desired clear imaging space region, imaged on the corresponding at least one first imaging unit is reduced by at least inclining the at least one first imaging unit used for image acquisition of the desired clear imaging space region, so that the imaging quality of the at least one first part is improved. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by at least inclining the at least one first imaging unit corresponding to the light field camera, the customized image acquisition of the expected clear imaging space region with higher imaging quality in the scene can be favorably realized, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
These and other advantages of the present application will become more apparent from the following detailed description of alternative embodiments thereof, which is to be read in connection with the accompanying drawings.
Drawings
The present application may be better understood by reference to the following description taken in conjunction with the accompanying drawings, in which like or similar reference numerals are used throughout the figures to designate like or similar components. The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present application and, together with the detailed description, serve to further illustrate the principles and advantages of the application. In the drawings:
fig. 1 is a flowchart of an image acquisition control method according to an embodiment of the present application;
fig. 2 is an example of a preview image of a scene acquired by a light field camera provided in an embodiment of the present application;
fig. 3 is a schematic view of a scene depth direction according to an embodiment of the present disclosure;
fig. 4a is a first example of fitting surface determination and first imaging unit inclination provided by the embodiment of the present application;
fig. 4b is a second example of the fitting surface determination and the first imaging unit inclination provided in the embodiment of the present application;
FIG. 5 is a first example of intersection of extension planes of three planes related to Sahm's law provided by an embodiment of the present application;
fig. 6 is a second example of intersection of extension planes of three planes related to the schemer's law provided by the embodiment of the present application;
fig. 7 is an example of a tilt direction of a first imaging unit according to an embodiment of the present disclosure;
fig. 8a is a fitting surface determination and a first imaging unit and first sub-lens tilt example provided by an embodiment of the present application;
fig. 8b is a third example of intersection of extension planes of three planes related to the schemer's law provided by the embodiment of the present application;
fig. 9 is a structural example of a light field camera provided in an embodiment of the present application;
fig. 10 is a logic block diagram of a first image acquisition control device according to an embodiment of the present application;
fig. 11 is a logic block diagram of a second image acquisition control device according to an embodiment of the present application;
fig. 12 is a logic block diagram of a third image acquisition control device according to an embodiment of the present application;
fig. 13 is a logic block diagram of a fourth image acquisition control device according to an embodiment of the present application.
Skilled artisans appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of embodiments of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.
It is also noted herein that, in order to avoid obscuring the present application with unnecessary detail, only the device structures and/or process steps that are germane to the solution according to the present application are depicted in the drawings and description, and the representation and description of components and processes that are not germane to the present application and known to those of ordinary skill in the art are omitted.
The following detailed description of the present application will be made in conjunction with the accompanying drawings (like numerals represent like elements throughout the several figures) and examples. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.
It will be understood by those within the art that the terms "first", "second", etc. in this application are used only to distinguish one step, device or module from another, and do not denote any particular technical meaning or necessarily logical order therebetween.
Fig. 1 is a flowchart of an image acquisition control method according to an embodiment of the present application. The execution main body of the image acquisition control method provided by the embodiment of the application can be a certain image acquisition control device. The device representation form of the image acquisition control device is not limited, for example, the image acquisition control device may be a certain independent component; alternatively, the image acquisition control device may be integrated as a certain functional module in an imaging device, where the imaging device may include, but is not limited to, a light field camera or a mobile phone including the light field camera, a tablet computer, and the like, and the embodiment of the present application is not limited thereto. Specifically, as shown in fig. 1, an image acquisition control method provided in an embodiment of the present application includes:
s101: a desired clearly imaged spatial region of the scene is determined.
S102: tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first portion is at least one portion of the scene located in the desired sharp imaging spatial region.
S103: the adjusted light field camera performs image acquisition on the scene.
When an object is imaged by an imaging device such as a light field camera, ideally, an object plane and an image plane are in point-to-point correspondence, but in an actual application process, due to the influence of factors such as light wave properties and aberration, an imaging light beam of a point on the object cannot be converged at one point, but forms a diffused perfect circle, ellipse or other circular projection on an image plane, which is called a circle of confusion (circle of confusion) and can also be called a circle of confusion, a scattering disc and the like. If the size of the circle of confusion is small, the imaging of the object corresponding to the circle of confusion is relatively clear, and the naked eye can be regarded as the focused imaging of the point; accordingly, if the size of a circle of confusion exceeds a certain tolerance, the imaging of the object to which the circle of confusion corresponds is relatively blurred.
Generally, a light field camera includes a main lens, a sub-lens array and an imaging unit array, which are sequentially arranged along a depth direction, when the light field camera is in a state where tilting operations of an imaging unit and/or a sub-lens and the like mentioned in the embodiments of the present application are not performed, a plane where the main lens is located, a plane where the sub-lens array is located, and a plane where the imaging unit array is located are parallel to each other and are respectively perpendicular to the depth direction, each sub-lens in the sub-lens array is located on a same plane perpendicular to the depth direction, an optical axis of each sub-lens is parallel to the depth direction, each imaging unit (which may be regarded as an imaging unit array or an image sensing unit) in the imaging unit array includes a plurality of pixels distributed in an array, each imaging unit is also located on a same plane perpendicular to the depth direction, and a normal of each imaging unit is. The plane parallel to the main lens in the embodiments of the present application includes but is not limited to: the plane of the main lens, the plane of the sub-lens in the initial state of non-tilt, the plane of the imaging unit in the initial state of non-tilt, or other planes parallel to the plane of the main lens.
According to the classical geometric optics theory, the light path diagram of the light field camera can be equivalent to that each sub-lens re-images the image (such as real image or virtual image) formed by the main lens on the imaging unit corresponding to the sub-lens, the pixel points at different positions on the imaging unit can store the light field information of objects with different depths in the scene, and the circle of confusion corresponding to at least part of a certain object in the scene and imaged at a certain pixel point of the imaging unit array through a certain sub-lens is:
Figure GDA0002214699520000071
in the above formula: c represents the equivalent diameter of the circle of confusion; f denotes the focal length of the sub-lens; v represents the distance between the image formed by at least part of the object passing through the main lens and the sub lens in sequence and the center of the sub lens, and the distance is equivalent to the image distance of the sub lens; v. ofnRepresenting a distance between a sub-lens array plane and an imaging unit array plane; n represents an aperture value (f-number).
In an application scene of image acquisition based on a light field camera, each imaging unit in an imaging unit array comprises a plurality of pixel points distributed in the array, and usually, not all the pixel points of the imaging unit participate in actual image acquisition, but part of the pixel points participate in actual image acquisition. The average value of the diffusion circles of the pixel points of the image information of the object to be shot actually recorded in the imaging unit can be regarded as the size of the average diffusion circle of the object to be shot in the imaging unit.
In the image acquisition application process of a scene based on a light field camera, a desired clear imaging space region of the scene can be determined, and the size of an average diffusion circle of at least one first part of the scene, which is located in the desired clear imaging space region, imaged on the corresponding at least one first imaging unit is reduced by at least inclining at least one first imaging unit used for image acquisition of the desired clear imaging space region, so that the imaging quality of the at least one first part is improved. In addition, the light field camera can bring more flexible post-interaction capability to a user through high-dimensional information acquisition, however, imaging effect can be related to information directly obtained through optical acquisition through post-interaction calculation, and during the period of carrying out optical acquisition on a scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of carrying out optical acquisition on at least one first part can be improved by at least inclining at least one first imaging unit corresponding to the light field camera, the customized image acquisition of the expected clear imaging space region needing higher imaging quality in the scene is facilitated, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
According to the technical scheme, the determination mode of the expected clear imaging space region is very flexible, and diversified practical application requirements can be met. For example, if a local spatial region in the scene is of greater interest, the corresponding local spatial region in the scene may be determined to be the desired clearly imaged spatial region, so as to improve the imaging quality of at least a portion of the scene located within the desired clearly imaged spatial region by tilting at least the corresponding first imaging unit; or, if a certain object in the scene is concerned, the spatial distribution area of the object in the scene can be determined as the expected clear imaging spatial area, so as to improve the imaging quality of the object by at least inclining the corresponding first imaging unit; and so on. It is understood that the mentioned part in the embodiments of the present application may include a whole object in the scene, or a part of an object in the scene, etc.
Optionally, the determining a desired sharp imaging spatial region of the scene includes: determining that a portion of a preview image of the scene acquired by the light field camera is a desired sharp region; determining first spatial distribution information of at least one first part corresponding to the expected clear area in the scene; determining the desired sharp imaging spatial region from the first spatial distribution information. For example, the preview image may be identified, and a local area of interest to a user or a device may be determined as the desired clear area according to the identification result, or the local area of the preview image may be determined as the desired clear area through operation information such as touch control of the light field camera outside the preview image, and the desired clear area may be determined by combining the desired clear area with spatial distribution of each part in a scene. The scheme selects the expected clear imaging space region of the scene based on the preview image of the scene, and is simple and easy to implement.
Further optionally, determining that a portion of a preview image of the scene acquired by the light field camera is a desired sharp region comprises: acquiring selection information of the preview image; determining, based at least on the selection information, that a corresponding portion of the preview image is the desired legibility region. Selecting on the two-dimensional picture of the preview image by means of finger touch, stylus touch, mouse selection and the like, and determining that the corresponding local part of the preview image is the expected clear area according to the selection information; after determining the desired sharp region, a desired sharp imaging spatial region of the scene may be determined in conjunction with spatial distribution information of portions of the scene. The scheme can improve the interaction convenience and flexibility of the determination of the expected clear area or the expected clear imaging space area of the scene, and improve the user experience. The preview image of the light field camera is a two-dimensional image of a certain view angle of the scene; the selection information of the preview image may include, but is not limited to: the direction information is selected, or the area information is selected, or the object information is selected, so that rich interaction modes are provided, and the use convenience of a user is improved. Optional interaction mode: for example, the selection information may include selection direction information, the user may arbitrarily scratch a direction of his desired sharp image by a finger on the preview image display screen, the image acquisition control device determines a spatial region of a predetermined size as the desired sharp region in the direction, and the desired sharp image spatial region of the scene may be determined by combining the desired sharp region and spatial distribution information of each part in the scene; for another example, the selection information may include selection area information, as shown in fig. 2, the user may draw a two-dimensional area with a certain size on the display screen of the preview image by using a finger, and the drawn two-dimensional area may be directly used as the desired clear area, or, the two-dimensional area may be adjusted by scaling, rotating, correcting, etc. and the adjusted two-dimensional area may be used as the desired clear area, and two-dimensional coordinates { x _ i, y _ i } of a set of image points corresponding to a portion of an image of each object in the scene in the preview image in the desired clear area may be obtained, and then, in combination with spatial distribution information of the desired clear area and the portions in the scene (as shown in fig. 3), a set of depth coordinates { z _ i } corresponding to the set of image points may be determined according to the depth distribution information of the portions in the scene, and according to the first spatial distribution information (the first spatial distribution information includes the set of two-dimensional coordinates { x _ i, y _ i } and a set of depth coordinates { z _ i }, which may be represented as a set { x _ i, y _ i, z _ i }), a desired clear imaging spatial region of the scene may be determined; for another example, the selection information may include selection object information, and the user may touch an object on the preview image display screen with a finger to determine a desired clear imaging spatial region of the scene according to spatial distribution information of the object in the scene.
In the technical solution provided in the embodiment of the present application, optionally, before the adjusted light field camera performs image acquisition on the scene, the method further includes: determining a maximum depth sub-region covered by the desired sharp imaging spatial region; and carrying out focusing adjustment on the light field camera so that the focusing surface of the light field camera after adjustment is positioned in the maximum depth subarea.
Depth of Field (DoF) generally refers to an object distance range in which a camera can clearly image a scene shot by the camera, that is, a certain Depth range in front of and behind a focusing object plane of the camera in a Depth direction can clearly image the scene. After the operations of aperture adjustment, focusing and the like of the light field camera are completed, the depth of field range of the light field camera is usually determined accordingly, and the depth of field range is a certain object distance range relative to the main lens of the light field camera in the depth direction. Through one-time optical acquisition of the light field camera, the circle of confusion of partial imaging in a scene within the range of depth of field is small in size, so that the imaging can be generally clear, and the circle of confusion of partial imaging outside the range of depth of field in the scene is large in size, so that the imaging is generally fuzzy. The method comprises the steps of determining a maximum depth sub-area covered by the expected clear imaging space area, namely acquiring the maximum depth and the minimum depth of each part in the scene, wherein the part is located in the expected clear imaging space area, and the depth range between the maximum depth and the minimum depth is the maximum depth sub-area, adjusting the focusing position of the light field camera to a position in the maximum depth sub-area, such as a center position in the maximum depth sub-area or a certain depth position before the center position, and the like, so that the depth range of the light field camera is distributed in the maximum depth sub-area, and improving the imaging quality of the part in the expected clear imaging space area, which is located in the depth range.
Optionally, after any first imaging unit of the at least one first imaging unit is tilted with respect to the plane parallel to the main lens, an average circle of confusion of a first portion corresponding to the first imaging unit imaged on the first imaging unit is smaller than or equal to an allowable circle of confusion. If the circle of confusion is less than or equal to a certain allowable circle of confusion, the object can be considered to be in focus imaged; accordingly, if the circle of confusion is greater than the permissible circle of confusion, the object may be considered out-of-focus imaging. The size of the circle of confusion is related to factors such as object distance, magnification, etc., and the size of the circle of confusion can be predetermined, for example, the size of the circle of confusion is 1/30mm, etc. This solution corresponds to that, during the tilt control process of the respective elements (including the first imaging unit, or the first imaging unit and the first sub-lens) affecting the acquisition of at least a first partial image in the desired sharp imaging spatial region, the average circle of confusion after the tilting of the respective elements affecting the acquisition of the respective first partial image is smaller than or equal to an allowable circle of confusion as the convergence condition for the tilt control of the respective elements, that is, during the tilt control process of the respective elements, if the respective elements affecting the acquisition of the respective first partial image are in a certain tilt state and the average circle of confusion corresponding to the tilt state is smaller than or equal to the allowable circle of confusion, the tilt control affecting the respective elements can be completed, so that the adjusted light field camera acquiring the image acquisition of the scene to be photographed can obtain a higher imaging quality of the respective first portion, and then the overall higher imaging quality of the expected clear imaging space region is obtained, and the customized image acquisition of the expected clear imaging space region in the scene to be shot is realized.
In the technical solution provided by the embodiment of the present application, the average circle of confusion of the at least one first portion of the scene located in the desired clear imaging spatial region imaged on the corresponding imaging unit can be changed by tilting the at least one first imaging unit, thereby changing the quality of the at least one first portion imaged.
(one) alternative implementation, in which at least a portion of the scene that is located in the desired sharp imaging spatial region can be changed by tilting the at least one first imaging unit, that is, tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce an average circle of confusion for imaging the at least one first portion on the at least one first imaging unit, includes: tilting the at least one first imaging unit relative to a plane parallel to the main lens to reduce a circle of mean confusion of the at least one first portion imaged on the at least one first imaging unit. The same or similar tilt control may be applied to each of the at least one first imaging unit to improve the overall imaging quality of at least a portion of the desired sharp imaging spatial region.
The description is not given by taking any one of the at least one first imaging unit as an example. The average value of the diffusion circles of the pixel points actually recorded with the first partial image information in the first imaging unit can be regarded as the size of the average diffusion circle of the corresponding first part in the first imaging unit. Before and after the first imaging unit tilts, the number of the pixel points and/or the number of the pixel points of the first imaging unit participating in the corresponding first part of actual image acquisition may be the same or different. Before the first imaging unit is tilted, the average value of the diffusion circles of the pixel points actually recorded with the first partial image information in the first imaging unit is used as the size of the average diffusion circle imaged by the same part before the first imaging unit is tilted; and taking the average value of the diffusion circles of the pixel points actually recorded with the first partial image information in the first unit after the first imaging unit is inclined as the size of the average diffusion circle imaged by the same part after the first imaging unit is inclined.
According to the imaging formula of the circle of confusion, the embodiment of the present application can change the distance v between the first imaging unit and the sub-lens (i.e., the first sub-lens) corresponding to the first imaging unit by tilting the first imaging unit, so as to cause the change of the corresponding circle of confusion, and meanwhile, the circle of confusion after the first imaging unit is tilted is smaller than the circle of confusion before the first imaging unit is tilted as the convergence condition for the tilt control of the first imaging unit, so that after the first imaging unit is tilted in a certain manner relative to the plane parallel to the main lens, the circle of confusion which the corresponding first part can reach imaging is reduced relative to the circle of confusion of the corresponding first part before the first imaging unit is tilted. It can be understood that a plane on which the first imaging unit is tilted before, that is, a plane on which the imaging unit array is tilted before, is parallel to a plane on which the main lens is located, and that tilting the first imaging unit with respect to the plane on which the first imaging unit is tilted before or the plane on which the first imaging unit is tilted before with respect to the imaging unit array is equivalent to tilting the first imaging unit with respect to the plane parallel to the main lens.
In this way, the size of the circle of confusion of at least one first portion of the scene located in the desired sharp imaging spatial region on the corresponding at least one first imaging unit is reduced, thereby improving the imaging quality of the at least one first portion. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by tilting the at least one first imaging unit corresponding to the light field camera, the customized image acquisition of the expected clear imaging space region which needs higher imaging quality in the scene is facilitated, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
In the technical scheme provided by the embodiment of the application, the first imaging unit can be controlled to incline relative to the plane parallel to the main lens according to the convergence condition of reducing the corresponding average diffusion circle (even less than or equal to the allowable diffusion circle), and the implementation mode is very flexible.
Optionally, tilting any one of the at least one first imaging unit with respect to a plane parallel to the main lens to reduce a circle of mean confusion of a first portion corresponding to the first imaging unit imaged on the first imaging unit, includes: determining a desired inclination angle of the first imaging unit according to a focal length of a first sub-lens, an included angle between a fitting surface and a plane parallel to a main lens, and a distance from an optical center of the first sub-lens to the fitting surface, wherein the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera; tilting the first imaging unit with respect to the plane parallel to the main lens at least according to the desired tilt angle to reduce a circle of mean confusion of the respective first portion imaged on the first imaging unit.
The fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera, and the main or important distribution trend of the spatial distribution of each image point of each image formed by the at least one first part through the main lens is reflected through the plane. The fitting surface can be determined before the at least one first sub-lens is tilted relative to the imaging surface, and the determining manner of the fitting surface is very flexible, for example, second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera can be determined according to first spatial distribution information of the at least one first part in the scene; and determining the fitting surface according to the second spatial distribution information. Optionally, according to the coordinates { x _ i, y _ i, z _ i } of each object point in at least a part of the scene located in the desired clear imaging space region, coordinates { x _ i ', y _ i ', z _ i ' } (i.e. second spatial distribution information) of each image point of the at least a part of the image formed by the main lens may be obtained, a minimum bounding box (minimum bounding box) surrounding the image points may be determined according to a Principal Component Analysis (PCA) algorithm model, as shown in fig. 4a and 8a, and then a plane within the minimum bounding box may be determined according to a Random Sample Consensus (sac) algorithm model, so that the sum of distances from each image point within the minimum bounding box to the plane is minimum, and the plane thus obtained is the fitting plane. The scheme is simple and easy to implement.
In the light field camera, after any one of the at least one first imaging unit is tilted, the plane where the first imaging unit is located is not parallel to the plane where the first sub-lens is located. Referring to scheimpflug's law, when the extension plane of the fitting surface, the extension plane of the first sub-lens corresponding to the first imaging unit, and the extension plane of the tilted first imaging unit intersect with a straight line, as shown in fig. 5, the region in which the first portion corresponding to the first imaging unit can obtain clear images is the largest. The inclination angle of the first imaging unit satisfying the above-described law may be a desired inclination angle of the first imaging unit as described in the embodiment of the present application.
Alternatively, as shown in fig. 6, the desired tilt angle of the first imaging unit may be calculated using the following equation:
Figure GDA0002214699520000141
in the above formula: θ represents a desired tilt angle of the first imaging unit; f denotes a focal length of the first sub-lens,
Figure GDA0002214699520000142
Represents the included angle between the fitting surface and the plane parallel to the main lens (such as the plane where the sub-lens array is located), and u represents the distance from the optical center of the first sub-lens to the fitting surface along the direction perpendicular to the plane parallel to the main lens, namely the object distance from the optical center of the first sub-lens to the fitting surface, wherein the included angle represents the included angle between the fitting surface and the plane parallel to the main lens (such as the plane where the sub-lens array is located), and the included angle represents the distance from the optical center of the firstThe first sub-lens is a sub-lens of the sub-lens array of the light field camera corresponding to the first imaging unit. The first imaging unit is inclined relative to the plane parallel to the main lens at a desired inclination angle determined at least according to the scheme, so that the inclination angle of the first imaging unit relative to the plane parallel to the main lens is as close as possible to or even equal to the desired inclination angle, and when the first imaging unit is in an inclined state for image acquisition, the average circle of confusion of the corresponding first part imaged by the first imaging unit is as small as possible. This arrangement improves the efficiency of the tilt control of the first imaging unit.
According to the Schlemm's law, inclining the first imaging unit along the center of the first imaging unit relative to a plane parallel to the main lens (such as a plane where the sub-lens array is located or a plane where the first imaging unit is located before the first imaging unit inclines), and enabling an extension plane of the first imaging unit to intersect with extension planes of the corresponding first sub-lens and the fitting plane to form a straight line, wherein the fitting plane can reach the maximum definition; in a scene corresponding to the scheme, although included angles between the plurality of first sub-lenses distributed in the vertical direction and the fitting surface are the same, the vertical coordinate positions with different optical centers cause that object distances from the optical centers of the plurality of first sub-lenses to the fitting surface are different, so that different first imaging units corresponding to different first sub-lenses need to be inclined by different angles to make the fitting surface clear, and the relationship between the size of the angle required to be inclined by each first imaging unit and the relative position between the center of each first imaging unit and the Sammy cross line is caused.
As a further alternative, as shown in fig. 4a and 4b, in practical application, the first imaging units with different distances from the intersection line of the virtual surface and the extension surface of the plane (the plane parallel to the main lens) where the sub-lens array is located need to be inclined at different angles to satisfy the above-mentioned schemer's law, for example, the first imaging unit a ' (the object distance u from the optical center of the first sub-lens a corresponding to the first imaging unit a ') to the virtual surface, which is closer to the intersection line of the virtual surface and the extension surface of the plane where the sub-lens array is locatedaSmaller) angle of inclination θ1A first imaging unit B '(corresponding to the first imaging unit B') which is farther from the intersection line of the extension plane of the plane where the fitting plane and the sub-lens array are located and is larger than the object distance u from the optical center of the first sub-lens B to the fitting planebGreater) angle of inclination θ2Therefore, it is further optional that before tilting the first imaging unit with respect to a plane parallel to the main lens, the method further includes: comparing the desired tilt angle to an imaging unit tilt adjustment angle threshold; in response to the desired tilt angle exceeding the imaging unit tilt adjustment angle threshold, adjusting away a focus position of the light field camera. In the optical camera, the inclination adjustment of the imaging unit is limited by machinery, a pixel structure and the like, so that the inclination of the imaging unit has certain capacity limitation, an imaging unit inclination adjustment angle threshold value can be determined according to the capacity of the imaging unit for allowing the inclination, the expected inclination angle is compared with the imaging unit inclination adjustment angle threshold value, if the expected inclination angle is larger than the imaging unit inclination adjustment angle threshold value, the focusing position of the optical camera can be adjusted far, namely the object distance of the main lens is increased, according to the imaging formula of the thin lens, the object distance of the main lens is increased, the image distance of the main lens is reduced, as the distance between the main lens and the sub lens is fixed, the object distance of the sub lens is increased, namely the distance between the fitting surface and the first sub lens is increased to a certain extent, so that the distance between the fitting surface and the intersection line of the first imaging unit corresponding to the first sub lens is increased from the first imaging unit, therefore, the plane where the first imaging unit is inclined by a small angle is intersected with the intersection line possibly, and the image of the scene acquired by the light field camera based on the adjusted light field is beneficial to improving the imaging quality of the corresponding first part.
Optionally, tilting any one of the at least one first imaging unit with respect to a plane parallel to the main lens to reduce a circle of mean confusion of a first portion corresponding to the first imaging unit imaged on the first imaging unit, includes: determining an allowable included angle range of the first imaging unit relative to a plane parallel to the main lens; and inclining the first imaging unit relative to the plane parallel to the main lens at least according to the allowable included angle range so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit. By adopting the scheme, the first imaging unit is inclined relative to the plane parallel to the main lens, and the included angle between the plane of the inclined first imaging unit and the plane parallel to the main lens falls into the allowable included angle range. The allowable included angle range can be flexibly determined according to the requirement that the imaging unit can bring corresponding average circle of confusion to reduce after being inclined, and the embodiment of the application does not limit the allowable included angle range. This arrangement improves the efficiency of the tilt control of the first imaging unit.
The allowable included angle range can be predetermined according to the requirement that the first imaging unit can bring corresponding average circle of confusion to be reduced after being inclined, and the determination method can include but is not limited to determination by adopting an experimental means, simulation, formula derivation and the like. Optionally, determining an allowable included angle range of the first imaging unit with respect to a plane parallel to the main lens includes: determining a desired inclination angle of the first imaging unit according to a focal length of a first sub-lens, an included angle between a fitting surface and a plane parallel to a main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, wherein the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera; determining the allowable included angle range according to at least the expected inclination angle. The method for determining the desired tilt angle may refer to the above description, and after determining the desired tilt angle, an angle range in which the desired tilt angle is within a certain allowable error range may be taken as the allowable included angle range in consideration of the first imaging unit tilt adjustment accuracy limit, the imaging quality requirement for the first portion, and the like. The range of the allowable included angle determined by the scheme is reasonable, and the inclination control of the first imaging unit relative to the plane parallel to the main lens is carried out based on the range of the allowable included angle, so that the efficiency is high and the realization is easy.
Optionally, tilting any one of the at least one first imaging unit with respect to a plane parallel to the main lens to reduce a circle of mean confusion of a first portion corresponding to the first imaging unit imaged on the first imaging unit, includes: and inclining the first imaging unit along a direction of increasing an included angle between a fitting surface and a plane where the first imaging unit is located relative to a plane parallel to the main lens so as to reduce an average circle of confusion of imaging of the corresponding first part on the first imaging unit, wherein the fitting surface is a plane fitted according to second spatial distribution information of images formed by the at least one first part through the main lens of the light field camera. As shown in fig. 7, an included angle between a plane on which the first imaging unit is tilted and a plane on which the first portion corresponding to the first imaging unit is imaged by the main lens of the light field camera is a1The first imaging unit can be tilted relative to a plane parallel to the main lens (e.g. a plane where the first imaging unit is tilted) in a direction that increases the included angle, and the included angle between the plane where the first imaging unit is tilted and a corresponding plane where the first portion is imaged by the main lens of the light field camera is a2，A2Greater than A1It can be seen that the probability that the extension of the plane where the first imaging unit is located, the fitting plane, and the plane where the first sub-lens corresponding to the first imaging unit is located, which extend to avoid intersecting the same straight line, is significantly increased, so that the average circle of confusion of the image formed by the corresponding first portion in the first imaging unit can be reduced, and the imaging quality of the corresponding first portion in the acquired image is improved. Further, if the inclination direction and the desired inclination angle are collectively taken as the basis for the inclination control of the first imaging unit, the efficiency of the inclination control of the first imaging unit can be further improved; alternatively, if the tilt direction and the allowable angle range are used together as the basis for the tilt control of the first imaging unit, it can be further mentioned thatThe efficiency of the tilt control of the first imaging unit is high.
(two) another alternative implementation, an alternative implementation, in which at least a portion of the scene that is located in the desired sharp imaging spatial region may be changed by tilting the at least one first imaging unit and the at least one first sub-lens corresponding to the at least one first imaging unit, that is, at least tilting the at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce an average circle of confusion for the at least one first portion to image on the at least one first imaging unit, includes: and tilting the at least one first imaging unit and the at least one first sub-lens relative to a plane parallel to the main lens to reduce the average circle of confusion of the at least one first portion imaged on the at least one first imaging unit, wherein the at least one first sub-lens is a sub-lens of the sub-lens array of the light field camera corresponding to the at least one first imaging unit.
The description will be given by taking an example of tilting any one of the at least one first imaging unit and the first sub-lens corresponding to the first imaging unit. The average value of the diffusion circles of the pixel points of the first partial image information corresponding to the first imaging unit is actually recorded in the first imaging unit, and the average value can be regarded as the size of the average diffusion circle of the corresponding first part in the first imaging unit. Before and after the first imaging unit and the first sub-lens corresponding to the first imaging unit are inclined, the number of pixel points and/or the number of pixel points of the first imaging unit participating in the corresponding first part of actual image acquisition may be the same or different. Before the first imaging unit and the corresponding first sub-lens are tilted, the mean value of the diffusion circles of the pixel points actually recorded with the corresponding first partial image information in the first imaging unit can be used as the size of the average diffusion circle of the same part of the image formed before the first imaging unit and the corresponding first sub-lens are tilted; after the first imaging unit and the corresponding first sub-lens are tilted, the average value of the diffusion circles of the pixel points actually recorded with the corresponding first partial image information in the first imaging unit is used as the size of the average diffusion circle of the same part of the image formed after the first imaging unit and the corresponding first sub-lens are tilted.
Each sub-lens in the sub-lens array of the light field camera can be regarded as a thin lens, and the optical imaging formula of the thin lens is as follows:
Figure GDA0002214699520000191
in the above formula: u represents the object distance of the sub-lens, namely the distance between the image of the part to be shot formed by the main lens of the light field camera and the sub-lens; v represents the image distance of the sub-lens; f denotes the focal length of the sub-lens.
Combining equations (1) and (3), the embodiment of the present application can change the object distance u of the first sub-lens by tilting the first sub-lens to cause the image distance v of the first sub-lensnAnd/or changing the distance v between the first sub-lens and the second sub-lens in a manner of tilting the first sub-lens and/or the first imaging unit to further cause the change of the corresponding circle of confusion, and meanwhile, taking the circle of confusion after the first imaging unit and the first sub-lens are tilted smaller than the circle of confusion before the first imaging unit and the first sub-lens are tilted as a convergence condition for tilt control of the first imaging unit and the first sub-lens, so that after the first imaging unit and the first sub-lens are tilted in a manner relative to the plane parallel to the main lens, the corresponding circle of confusion is reduced relative to the corresponding circle of confusion before the first imaging unit and the first sub-lens are tilted.
In this way, the size of the average diffusion circle imaged on the corresponding at least one first imaging unit of at least one first portion of the scene located in the desired sharp imaging spatial region is reduced by tilting the at least one first imaging unit for image acquisition of the desired sharp imaging spatial region and the at least one first sub-lens corresponding to the at least one imaging unit, thereby improving the imaging quality of the at least one first portion. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by means of the at least one first imaging unit and the at least one first sub-lens corresponding to the inclined light field camera, the customized image acquisition of the expected clear imaging space region which needs higher imaging quality in the scene is facilitated, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
In the technical scheme provided by the embodiment of the application, any one of the first imaging unit and the first sub-lens corresponding to the first imaging unit can be subjected to tilt control according to the convergence condition for reducing the corresponding average diffusion circle (even smaller than or equal to the allowable diffusion circle), and the implementation mode is very flexible.
Optionally, tilting any one of the at least one first imaging unit and a first sub-lens of the at least one first sub-lens corresponding to the first imaging unit with respect to a plane parallel to the main lens to reduce an average circle of confusion of a first portion of the at least one first portion corresponding to the first imaging unit that is imaged on the first imaging unit, includes: determining a first inclination angle of the first imaging unit and a second inclination angle of the corresponding first sub-lens according to the focal length of the corresponding first sub-lens, an included angle between a fitting surface and the plane parallel to the main lens, a distance from the optical center of the corresponding first sub-lens to the fitting surface along a direction vertical to the plane parallel to the main lens, and a distance from the optical center of the corresponding first sub-lens to the first imaging unit along a direction vertical to the plane parallel to the main lens; and inclining the first imaging unit relative to the plane parallel to the main lens according to the first inclination angle and inclining the first sub-lens relative to the plane parallel to the main lens according to the second inclination angle so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit.
In the light field camera, after the first imaging unit and the first sub-lens are respectively tilted, a plane where the first imaging unit is located and a plane where the first sub-lens is located may not be parallel any more. Referring to the law of Schlemm, when the extension of the fitting surface, the extension of the plane where the first sub-lens is inclined, and the extension of the plane where the first imaging unit is inclined intersect with a straight line, the area where the corresponding first portion can obtain clear imaging is the largest. The inclination angle of the first imaging unit satisfying the above-mentioned law is the first inclination angle in the embodiment of the present application, and the inclination angle of the first sub-lens satisfying the above-mentioned law is the second inclination angle in the embodiment of the present application.
As shown in fig. 8a and 8b, the following relations can be obtained according to fig. 8b, for example, after the first imaging unit and the first sub-lens are tilted, and the image formed by the first portion through the main lens is re-imaged through the sub-lens:
Figure GDA0002214699520000211
Figure GDA0002214699520000212
v＝v1+v2......................................................(6)
Figure GDA0002214699520000213
Figure GDA0002214699520000214
from equations (4) to (9), and the angular relationship in the optical path shown in fig. 8b, it can be found that:
Figure GDA0002214699520000216
wherein: α represents a first tilt angle of the first imaging unit; β represents a second tilt angle of the first sub-lens; f denotes a focal length of the first sub-lens;
Figure GDA0002214699520000217
representing the angle between the fitting surface and a plane parallel to the main lens (e.g. the plane in which the first imaging unit is tilted); u represents the distance from the optical center of the corresponding first sub-lens to the fitting surface through the direction perpendicular to the plane parallel to the main lens (such as the plane where the first sub-lens is inclined, or the plane where the first imaging unit is inclined); v denotes a distance from an optical center of the corresponding first sub-lens to the first imaging unit in a direction perpendicular to the plane parallel to the main lens (e.g., a plane on which the first sub-lens is tilted before, or a plane on which the first imaging unit is tilted before), for example, if a center of the first sub-lens before and after tilting remains unchanged and a center of the first imaging unit before and after tilting remains unchanged, a distance between the center of the first sub-lens and the center of the first imaging unit is a distance from the optical center of the corresponding first sub-lens to the first imaging unit in a direction perpendicular to the plane parallel to the main lens. The scheme improves the efficiency of the common inclination control of the first imaging unit and the first sub-lens, and the adjusted light field camera can acquire the corresponding imaging with higher quality of the first part by image acquisition.
In combination with any one of the technical solutions provided in the embodiments of the present application, optionally, before at least tilting the at least one first imaging unit, the method further includes: at least the at least one first imaging unit to be tilted in the imaging unit array is determined. By adopting the scheme, at least one imaging unit used for image acquisition of the expected clear imaging space region can be determined in the imaging unit array of the light field camera to serve as the at least one first imaging unit to be inclined, and then at least the first imaging unit is inclined relative to the plane parallel to the main lens, so that the average circle of confusion of at least one first part of the expected clear imaging space region imaged on the corresponding first imaging unit is reduced, and the imaging quality of the corresponding first part is improved.
Further optionally, the determining at least the at least one first imaging unit to be tilted in the imaging unit array comprises: determining at least one sub-lens in the sub-lens array corresponding to a desired sharp region of a preview image of the scene as the at least one first sub-lens; and determining the at least one first imaging unit corresponding to the at least one first sub lens according to the corresponding relation between the sub lens in the sub lens array and the imaging unit of the imaging unit array. In an alternative implementation manner, in the light field camera, as shown in fig. 9, the sub-lens array includes sub-lenses having the same focal length, and the distance between the imaging unit array and the sub-lens array is equal to the focal length of the sub-lenses; under the state that the imaging unit and the sub-lens of the light field camera are not inclined, the spatial resolution of an image (such as the preview image) of a certain view angle of a scene to be shot based on the light field camera corresponds to the distribution of the sub-lenses of the sub-lens array, each sub-lens corresponds to a certain imaging unit in the imaging unit array, each imaging unit comprises a plurality of pixels distributed in an array, and the information of light rays from the sub-lens array in different directions of a certain part of the scene to be shot is recorded by the imaging unit, namely, in the light field camera, the corresponding relation exists between the pixel distribution of the preview image and the sub-lens distribution of the sub-lens array, and the corresponding relation exists between the sub-lenses of the sub-lens array and the imaging units in the imaging unit array, therefore, according to the part corresponding to the depth distribution subarea in the preview image of the scene to be shot acquired by the light field camera, and determining a first sub-lens influencing the image acquisition of the first depth distribution sub-area, and determining the first imaging unit corresponding to the first sub-lens according to the corresponding relation between the sub-lens in the sub-lens array and the imaging unit of the imaging unit array. By adopting the scheme, the at least one first imaging unit and the at least one first sub-lens to be tilted can be determined, and the method is simple and easy to implement.
It is understood by those skilled in the art that, in any method described above in the embodiments of the present application, the sequence number of each step does not mean the execution sequence, and the execution sequence of each step should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 10 is a logic block diagram of a first image acquisition control apparatus according to an embodiment of the present application. As shown in fig. 10, a first image acquisition control apparatus provided in an embodiment of the present application includes: a desired clear imaging spatial region determining module 101, a tilt control module 102, and an image acquisition control module 103. A desired sharp imaging spatial region determining module 101 is used to determine a desired sharp imaging spatial region of the scene; the tilt control module 102 is configured to tilt at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region; the image acquisition control module 103 is configured to perform image acquisition on the scene by the adjusted light field camera.
The image acquisition control device provided by the embodiment of the application can determine the expected clear imaging space region of a scene in the process of image acquisition application of the scene based on a light field camera, and the size of an average diffusion circle of at least one first part of the scene, which is positioned in the expected clear imaging space region, imaged on the corresponding at least one first imaging unit is reduced by at least inclining the at least one first imaging unit used for image acquisition of the expected clear imaging space region, so that the imaging quality of the at least one first part is improved. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by at least inclining the at least one first imaging unit corresponding to the light field camera, the customized image acquisition of the expected clear imaging space region with higher imaging quality in the scene can be favorably realized, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
The device representation form of the image acquisition control device is not limited, for example, the image acquisition control device may be a certain independent component; alternatively, the image acquisition control device may be integrated as a certain functional module in an imaging device, where the imaging device may include, but is not limited to, a light field camera or a mobile phone including the light field camera, a tablet computer, and the like, and the embodiment of the present application is not limited thereto.
Optionally, as shown in fig. 11, the desired clear imaging spatial region determining module 101 includes: a desired sharp region determining sub-module 1011, a first spatial distribution information determining sub-module 1012, and a desired sharp imaging spatial region determining sub-module 1013. A desired sharp region determination sub-module 1011 is configured to determine that a part of a preview image of the scene acquired by the light field camera is a desired sharp region; the first spatial distribution information determining sub-module 1012 is configured to determine first spatial distribution information of a first portion in the scene corresponding to the desired clear region; the desired sharp imaging spatial region determining sub-module 1013 is configured to determine the desired sharp imaging spatial region from the first spatial distribution information. The scheme selects the expected clear imaging space region of the scene based on the preview image of the scene, and is simple and easy to implement.
Further optionally, the desired clear area determination sub-module 1011 includes: a selection information acquiring unit 10111 and a desired clear region determining unit 10112. A selection information acquisition unit 10111 for acquiring selection information for the preview image; the desired clear region determining unit 10112 is configured to determine a corresponding part of the preview image as the desired clear region at least according to the selection information. The scheme can improve the interaction convenience and flexibility of the determination of the expected clear area or the expected clear imaging space area of the scene, and improve the user experience. The preview image of the light field camera is a two-dimensional image of a certain view angle of the scene; the selection information of the preview image may include, but is not limited to: the direction information is selected, or the area information is selected, or the object information is selected, so that rich interaction modes are provided, and the use convenience of a user is improved.
Optionally, after any first imaging unit of the at least one first imaging unit is tilted with respect to the plane parallel to the main lens, an average circle of confusion of a first portion corresponding to the first imaging unit imaged on the first imaging unit is smaller than or equal to an allowable circle of confusion.
Optionally, the image acquisition control device further includes: a maximum depth sub-area determination module 104 and a first focus adjustment module 105. A maximum depth sub-region determination module 104 for determining a maximum depth sub-region of the desired sharp imaging volume area coverage; the first focus adjustment module 105 is configured to perform focus adjustment on the light field camera, so that the focus plane adjusted by the light field camera is located in the maximum depth sub-region. According to the scheme, the focusing position of the light field camera is adjusted to a certain position in the maximum depth sub-area, so that the depth of field range of the light field camera is distributed in the maximum depth sub-area, and the imaging quality of the part, located in the depth of field range, in the expected clear imaging area is improved.
Optionally, as shown in fig. 12, the tilt control module 102 includes: a first tilt control submodule 1021. The first tilt control submodule 1021 is configured to tilt the at least one first imaging unit with respect to a plane parallel to the main lens to reduce a mean circle of confusion for imaging the at least one first portion on the at least one first imaging unit. The method can determine a desired sharp imaging spatial region of a scene, and the size of the mean circle of confusion of at least one first part of the scene located in the desired sharp imaging spatial region on the corresponding at least one first imaging unit is reduced by tilting the at least one first imaging unit for image acquisition of the desired sharp imaging spatial region, thereby improving the imaging quality of the at least one first part. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by tilting the at least one first imaging unit corresponding to the light field camera, the customized image acquisition of the expected clear imaging space region which needs higher imaging quality in the scene is facilitated, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
Optionally, the first tilt control submodule 1021 includes: a desired tilt angle determining unit 10211 and a first tilt control unit 10212. The desired tilt angle determining unit 10211 is configured to determine a desired tilt angle of any first imaging unit according to a focal length of any first sub-lens, an included angle between a fitting surface and a plane parallel to the main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, where the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first portion passing through the main lens of the light field camera; the first tilt control unit 10212 is configured to tilt the first imaging unit with respect to the plane parallel to the main lens according to at least the desired tilt angle to reduce a circle of mean confusion of the respective first portion imaged on the first imaging unit. The first imaging unit is inclined relative to the plane parallel to the main lens at a desired inclination angle determined at least according to the scheme, so that the inclination angle of the first imaging unit relative to the plane parallel to the main lens is as close as possible to or even equal to the desired inclination angle, and when the first imaging unit is in an inclined state for image acquisition, the average circle of confusion of the corresponding first part imaged by the first imaging unit is as small as possible. This arrangement improves the efficiency of the tilt control of the first imaging unit.
Optionally, the image acquisition control device further includes: an angle comparing module 106 and a second focus adjusting module 107. The angle comparison module 106 is used for comparing the expected inclination angle with an imaging unit inclination adjustment angle threshold; the second focus adjustment module 107 is configured to adjust a focus position of the light field camera away in response to the desired tilt angle exceeding the imaging unit tilt adjustment angle threshold. The scheme is equivalent to that the distance between the fitting surface and the first sub-lens is increased through focusing adjustment to a certain extent, so that the distance between the fitting surface and the intersection line of the first imaging unit corresponding to the first sub-lens and the first imaging unit is increased, the intersection line and the plane where the first imaging unit is inclined by a small angle can be intersected, and the adjusted light field camera is favorable for improving the imaging quality of the corresponding first part.
Optionally, the first tilt control submodule 1021 includes: an allowable angle range determining unit 10213 and a range tilt control unit 10214. The allowable included angle range determining unit 10213 is used for determining an allowable included angle range of any one of the first imaging units relative to a plane parallel to the main lens; the range tilt control unit 10214 is configured to tilt the first imaging unit with respect to the plane parallel to the main lens at least according to the allowable included angle range to reduce the average circle of confusion of the corresponding first portion imaged on the first imaging unit. The allowable included angle range can be flexibly determined according to the requirement that the imaging unit can bring corresponding average circle of confusion to reduce after being inclined, and the embodiment of the application does not limit the allowable included angle range. This arrangement improves the efficiency of the tilt control of the first imaging unit.
Further optionally, the allowable included angle range determining unit 10213 includes: a desired tilt angle determining subunit 102131 and an allowable angle range determining unit 102132. The expected tilt angle determining subunit 102131 is configured to determine an expected tilt angle of any first imaging unit according to a focal length of any first sub-lens, an included angle between a fitting surface and a plane parallel to the main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, where the first sub-lens is a sub-lens corresponding to the first imaging unit in the sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first portion passing through the main lens of the light field camera; the allowable included angle range determining unit 102312 is used for determining the allowable included angle range according to at least the expected inclination angle. The range of the allowable included angle determined by the scheme is reasonable, and the inclination control of the first imaging unit relative to the plane parallel to the main lens is carried out based on the range of the allowable included angle, so that the efficiency is high and the realization is easy.
Optionally, the first tilt control submodule 1021 includes: a tilt direction control unit 1025. The direction inclination control unit 1025 is configured to incline the first imaging unit along a direction that increases an included angle between a fitting surface and a plane where the first imaging unit is located with respect to a plane parallel to the main lens, so as to reduce an average circle of confusion of the corresponding first portion imaged on the first imaging unit, wherein the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first portion passing through the main lens of the light field camera. The probability that the extension of the plane where the first imaging unit is located, the extension of the fitting plane where the first imaging unit is located and the extension of the plane where the first sub-lens corresponding to the first imaging unit is located do not intersect with each other in the same straight line is obviously increased, so that the average circle of confusion of the image formed by the corresponding first part in the first imaging unit can be reduced, and the imaging quality of the corresponding first part in the acquired image is improved.
Further, if the inclination direction and the desired inclination angle are collectively taken as the basis for the inclination control of the first imaging unit, the efficiency of the inclination control of the first imaging unit can be further improved; alternatively, if the tilt direction and the allowable angle range are collectively used as the basis for the tilt control of the first imaging unit, the efficiency of the tilt control of the first imaging unit can be further improved.
Optionally, the tilt control module 102 includes: a second tilt control sub-module 1022. The second tilt control sub-module 1022 is configured to tilt the at least one first imaging unit and the at least one first sub-lens with respect to a plane parallel to the main lens to reduce a circle of mean confusion of the at least one first portion imaged on the at least one first imaging unit, wherein the at least one first sub-lens is a sub-lens of the sub-lens array of the light field camera corresponding to the at least one first imaging unit. The method can determine a desired sharp imaging spatial region of a scene, and reduce the size of an average diffusion circle imaged on at least one corresponding first imaging unit by at least one first imaging unit used for image acquisition of the desired sharp imaging spatial region and at least one first sub-lens corresponding to the at least one imaging unit in the scene, thereby improving the imaging quality of the at least one first portion. In addition, during the optical acquisition of the scene image by adopting the technical scheme provided by the embodiment of the application, because the expected clear imaging space region of the scene can be determined according to actual needs, and the imaging quality of the optical acquisition of the at least one first part can be improved by means of the at least one first imaging unit and the at least one first sub-lens corresponding to the inclined light field camera, the customized image acquisition of the expected clear imaging space region which needs higher imaging quality in the scene is facilitated, and the requirements of diversified image acquisition application, post-interaction application based on optical acquisition information and the like can be better met.
Further optionally, the second tilt control sub-module 1022 includes: a tilt angle determining unit 10221 and a second tilt control unit 10222. A tilt angle determining unit 10221 for determining a first tilt angle of the first imaging unit and a second tilt angle of the first sub-lens according to a focal length of the first sub-lens, an included angle between a fitting surface and the plane parallel to the main lens, a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, and a distance from the optical center of the first sub-lens to the first imaging unit along a direction perpendicular to the plane parallel to the main lens; a second tilt control unit 10222 is used for tilting the first imaging unit with respect to the plane parallel to the main lens according to the first tilt angle and tilting the first sub-lens with respect to the plane parallel to the main lens according to the second tilt angle to reduce the average circle of confusion of the corresponding first portion imaged on the first imaging unit. The scheme improves the efficiency of the common inclination control of the first imaging unit and the first sub-lens, and the adjusted light field camera can acquire the corresponding imaging with higher quality of the first part by image acquisition.
Optionally, the image acquisition control device further includes: a second spatial distribution information determination module 108 and a fitting surface determination module 109. The second spatial distribution information determining module 108 is configured to determine second spatial distribution information of images of the at least one first portion formed by the main lens of the light field camera according to first spatial distribution information of the at least one first portion in the scene; the fitting surface determining module 109 is configured to determine the fitting surface according to the second spatial distribution information. Optionally, the sum of the distances from each image to the fitting surface is the smallest. The scheme is simple and easy to implement.
Optionally, as shown in fig. 11, the image acquisition control device further includes: a to-be-tilted element determining module 1010. The to-be-tilted element determining module 1010 is configured to determine at least the at least one first imaging unit to be tilted in the imaging unit array. As the determination manner of the at least one first imaging unit waiting for the tilting element is very flexible, the embodiment of the present application is not limited thereto.
Further optionally, the to-be-tilted element determining module 1010 includes: a first sub-lens determining sub-module 10101 and a first imaging unit determining sub-module 10102. The first sub-lens determining sub-module 10101 is configured to determine at least one sub-lens in the sub-lens array corresponding to a desired sharp region of a preview image of the scene, as the at least one first sub-lens; the first imaging unit determining sub-module 10102 is configured to determine the at least one first imaging unit corresponding to the at least one first sub-lens according to a correspondence relationship between sub-lenses in the sub-lens array and imaging units of the imaging unit array. By adopting the scheme to determine the at least one first sub-lens and the at least one first imaging unit, the method is simple and easy to implement.
Fig. 13 is a schematic structural diagram of a fourth image acquisition control device provided in the embodiment of the present application, and the specific embodiment of the present application does not limit the specific implementation manner of the image acquisition control device 1300. As shown in fig. 13, the image acquisition control apparatus 1300 may include:
a Processor (Processor)1310, a Communications Interface 1320, a Memory 1330, and a communication bus 1340. Wherein:
the processor 1310, communication interface 1320, and memory 1330 communicate with each other via a communication bus 1340.
A communication interface 1320 for communicating with, for example, an array of deformable imaging elements.
The processor 1310 is configured to execute the program 1332, and may specifically execute relevant steps in any of the above method embodiments.
For example, the program 1332 may include program code that includes computer operating instructions.
Processor 1310 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present Application.
The memory 1330 stores the program 1332. The Memory 1330 may include a Random Access Memory (RAM) and may further include a Non-volatile Memory (Non-volatile Memory), such as at least one disk Memory.
For example, in an alternative implementation, processor 1310, by executing program 1332, may perform the following steps: determining a desired sharp imaging spatial region of a scene; tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region; the adjusted light field camera performs image acquisition on the scene.
In other alternative implementations, the processor 1310 may also execute the steps mentioned in any of the other embodiments through executing the program 1332, which is not described herein again.
For specific implementation of each step in the program 1332, reference may be made to corresponding descriptions in corresponding steps, modules, sub-modules, and units in the foregoing embodiments, and details are not described here again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
In the foregoing embodiments of the present application, the sequence numbers and/or the sequence orders of the embodiments are only for convenience of description, and do not represent the advantages or the disadvantages of the embodiments. The description of each embodiment has different emphasis, and for parts which are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments. For the description of the implementation principle or process of the embodiments of the apparatus, device or system, reference may be made to the description of the corresponding method embodiments, which are not repeated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In the embodiments of the apparatus, method, system, etc. of the present application, it is apparent that each component (system, subsystem, module, sub-module, unit, sub-unit, etc.) or each step may be decomposed, combined, and/or recombined after being decomposed. These decompositions and/or recombinations are to be considered as equivalents of the present application. Also, in the above description of specific embodiments of the application, features described and/or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, in combination with or instead of the features in the other embodiments.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
Finally, it should be noted that: the above embodiments are merely illustrative, and not restrictive, and those skilled in the relevant art can make various changes and modifications without departing from the spirit and scope of the present application, and therefore all equivalent technical solutions also fall within the scope of the present application, and the scope of the present application is defined by the appended claims.

Claims (37)

1. An image acquisition control method, comprising:
determining a desired sharp imaging spatial region of a scene;
tilting at least one first imaging unit relative to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
the adjusted light field camera performs image acquisition on the scene.
2. The method of claim 1, wherein determining the desired sharp imaging spatial region of the scene comprises:
determining that a portion of a preview image of the scene acquired by the light field camera is a desired sharp region;
determining first spatial distribution information of a first part corresponding to the expected clear area in the scene;
determining the desired sharp imaging spatial region from the first spatial distribution information.
3. The method of claim 2, wherein determining that the portion of the preview image of the scene acquired by the light field camera is a desired sharp region comprises:
acquiring selection information of the preview image;
determining, based at least on the selection information, that a corresponding portion of the preview image is the desired legibility region.
4. The method of claim 3, wherein the selection information comprises: the direction information is selected, or the area information is selected, or the object information is selected.
5. The method according to any of claims 1-4, wherein after any of the at least one first imaging unit is tilted with respect to the plane parallel to the main lens of the light field camera, a mean circle of confusion for a first portion corresponding to the first imaged unit to image on the first imaging unit is less than or equal to an allowable circle of confusion.
6. The method according to any of claims 1-4, wherein before the adjusted light field camera performs image acquisition on the scene, further comprising:
determining a maximum depth sub-region covered by the desired sharp imaging spatial region;
and carrying out focusing adjustment on the light field camera so that the focusing surface of the light field camera after adjustment is positioned in the maximum depth subarea.
7. The method of any of claims 1-4, wherein tilting the at least one first imaging unit relative to a plane parallel to a main lens of the light field camera to reduce a mean circle of confusion for the at least one first portion to image on the at least one first imaging unit comprises:
tilting the at least one first imaging unit relative to a plane parallel to the main lens to reduce a circle of mean confusion of the at least one first portion imaged on the at least one first imaging unit.
8. The method of claim 7, wherein tilting any one of the at least one first imaging unit relative to a plane parallel to the main lens to reduce a circle of mean confusion for a first portion corresponding to the first imaging unit to image on the first imaging unit comprises:
determining a desired inclination angle of the first imaging unit according to a focal length of a first sub-lens, an included angle between a fitting surface and a plane parallel to a main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, wherein the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera;
tilting the first imaging unit with respect to the plane parallel to the main lens at least according to the desired tilt angle to reduce a circle of mean confusion of the respective first portion imaged on the first imaging unit.
9. The method of claim 8, wherein prior to tilting the first imaging unit relative to a plane parallel to the main lens, further comprising:
comparing the desired tilt angle to an imaging unit tilt adjustment angle threshold;
in response to the desired tilt angle exceeding the imaging unit tilt adjustment angle threshold, adjusting away a focus position of the light field camera.
10. The method of claim 7, wherein tilting any one of the at least one first imaging unit relative to a plane parallel to the main lens to reduce a circle of mean confusion for a first portion corresponding to the first imaging unit to image on the first imaging unit comprises:
determining an allowable included angle range of the first imaging unit relative to a plane parallel to the main lens;
and inclining the first imaging unit relative to the plane parallel to the main lens at least according to the allowable included angle range so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit.
11. The method of claim 10, wherein determining the range of allowable angles of the first imaging unit relative to a plane parallel to the main lens comprises:
determining a desired inclination angle of the first imaging unit according to a focal length of a first sub-lens, an included angle between a fitting surface and a plane parallel to a main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, wherein the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera;
determining the allowable included angle range according to at least the expected inclination angle.
12. The method of claim 7, wherein tilting any one of the at least one first imaging unit relative to a plane parallel to the main lens to reduce a circle of mean confusion for a first portion corresponding to the first imaging unit to image on the first imaging unit comprises:
and inclining the first imaging unit along a direction of increasing an included angle between a fitting surface and a plane where the first imaging unit is located relative to a plane parallel to the main lens so as to reduce an average circle of confusion of imaging of the corresponding first part on the first imaging unit, wherein the fitting surface is a plane fitted according to second spatial distribution information of images formed by the at least one first part through the main lens of the light field camera.
13. The method of any of claims 1-4, wherein tilting at least the at least one first imaging unit relative to a plane parallel to a main lens of the light field camera to reduce a mean circle of confusion for the at least one first portion to image on the at least one first imaging unit comprises:
and inclining the at least one first imaging unit and the at least one first sub-lens relative to a plane parallel to the main lens to reduce the average circle of confusion of the at least one first part imaged on the at least one first imaging unit, wherein the at least one first sub-lens is a sub-lens corresponding to the at least one first imaging unit in the sub-lens array of the light field camera.
14. The method of claim 13, wherein tilting any one of the at least one first imaging unit and a corresponding one of the at least one first sub-lens with respect to a plane parallel to the main lens to reduce an average circle of confusion for a corresponding one of the at least one first portion with respect to the first imaging unit to image on the first imaging unit comprises:
determining a first inclination angle of the first imaging unit and a second inclination angle of the corresponding first sub-lens according to the focal length of the corresponding first sub-lens, an included angle between a fitting surface and the plane parallel to the main lens, a distance from the optical center of the corresponding first sub-lens to the fitting surface along a direction vertical to the plane parallel to the main lens, and a distance from the optical center of the corresponding first sub-lens to the first imaging unit along a direction vertical to the plane parallel to the main lens;
and inclining the first imaging unit relative to the plane parallel to the main lens according to the first inclination angle and inclining the first sub-lens relative to the plane parallel to the main lens according to the second inclination angle so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit.
15. The method of any of claims 8-9, 11-12, and 14, wherein prior to tilting at least a first imaging unit relative to a plane parallel to the main lens, further comprising:
determining second spatial distribution information of images of the at least one first portion formed by a main lens of the light field camera according to first spatial distribution information of the at least one first portion in the scene;
and determining the fitting surface according to the second spatial distribution information.
16. The method of claim 15, wherein the sum of the distances from each of the images to the fitting surface is minimized.
17. The method of any of claims 1-4, 8-12, 14, 16, wherein prior to tilting the at least one first imaging unit, further comprising:
at least the at least one first imaging unit to be tilted in the imaging unit array is determined.
18. The method of claim 17, wherein said determining at least said at least a first imaging unit of said array of imaging units to be tilted comprises:
determining at least one sub-lens in a sub-lens array corresponding to a desired sharp region of a preview image of the scene as at least a first sub-lens;
and determining the at least one first imaging unit corresponding to the at least one first sub lens according to the corresponding relation between the sub lens in the sub lens array and the imaging unit of the imaging unit array.
19. An image acquisition control apparatus, comprising:
a desired sharp imaging spatial region determining module for determining a desired sharp imaging spatial region of the scene;
a tilt control module for tilting at least one first imaging unit with respect to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
and the image acquisition control module is used for acquiring the images of the scene by the adjusted light field camera.
20. The image acquisition control device of claim 19 wherein the desired sharp imaging spatial region determination module comprises:
a desired sharpness zone determination sub-module for determining that a part of a preview image of the scene acquired by the light field camera is a desired sharpness zone;
a first spatial distribution information determining sub-module, configured to determine first spatial distribution information of a first portion corresponding to the desired clear area in the scene;
a desired sharp imaging spatial region determining sub-module for determining the desired sharp imaging spatial region based on the first spatial distribution information.
21. The image acquisition control device of claim 20 wherein the desired clear region determination sub-module comprises:
a selection information acquisition unit for acquiring selection information of the preview image;
and the expected clear area determining unit is used for determining that the corresponding part of the preview image is the expected clear area at least according to the selection information.
22. The image acquisition control device according to claim 21, wherein the selection information includes: the direction information is selected, or the area information is selected, or the object information is selected.
23. The image acquisition control device according to any one of claims 19 to 22, wherein after any one of the at least one first imaging unit is tilted with respect to the plane parallel to the main lens of the light field camera, a first portion corresponding to the first imaged unit is imaged on the first imaging unit with an average circle of confusion that is less than or equal to an allowable circle of confusion.
24. The image acquisition control device according to any one of claims 19 to 22, further comprising:
a maximum depth sub-region determining module for determining a maximum depth sub-region covered by the desired sharp imaging space region;
and the first focusing adjustment module is used for carrying out focusing adjustment on the light field camera so as to enable the focusing surface of the light field camera after adjustment to be positioned in the maximum depth sub-area.
25. The image acquisition control device according to any one of claims 19 to 22, wherein the tilt control module comprises:
a first tilt control submodule for tilting the at least one first imaging element relative to a plane parallel to the main lens to reduce a mean circle of confusion of the at least one first portion imaged on the at least one first imaging element.
26. The image acquisition control device of claim 25, wherein the first tilt control submodule comprises:
a desired tilt angle determining unit, configured to determine a desired tilt angle of any one of the first imaging units according to a focal length of any one of the first sub-lenses, an included angle between a fitting surface and a plane parallel to the main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, where the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first portion passing through the main lens of the light field camera;
a first tilt control unit for tilting the first imaging unit with respect to the plane parallel to the main lens at least according to the desired tilt angle to reduce a circle of mean confusion of the respective first portion imaged on the first imaging unit.
27. The image acquisition control device according to claim 26, further comprising:
an angle comparison module for comparing the desired tilt angle with an imaging unit tilt adjustment angle threshold;
a second focus adjustment module for adjusting a focus position of the light field camera away in response to the desired tilt angle exceeding the imaging unit tilt adjustment angle threshold.
28. The image acquisition control device of claim 25, wherein the first tilt control submodule comprises:
an allowable included angle range determining unit, configured to determine an allowable included angle range of any one of the first imaging units with respect to a plane parallel to the main lens;
and the range inclination control unit is used for inclining the first imaging unit relative to the plane parallel to the main lens at least according to the allowable included angle range so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit.
29. The image acquisition control device according to claim 28, wherein the allowable included angle range determination unit includes:
a desired tilt angle determining subunit, configured to determine a desired tilt angle of any one of the first imaging units according to a focal length of any one of the first sub-lenses, an included angle between a fitting surface and a plane parallel to the main lens, and a distance from an optical center of the first sub-lens to the fitting surface along a direction perpendicular to the plane parallel to the main lens, where the first sub-lens is a sub-lens corresponding to the first imaging unit in a sub-lens array of the light field camera, and the fitting surface is a plane fitted according to second spatial distribution information of each image formed by the at least one first portion passing through the main lens of the light field camera;
and the allowable included angle range determining unit is used for determining the allowable included angle range at least according to the expected inclination angle.
30. The image acquisition control device of claim 25, wherein the first tilt control submodule comprises:
and the direction inclination control unit is used for inclining the first imaging unit along a direction of increasing an included angle between a fitting surface and a plane where the first imaging unit is located relative to a plane parallel to the main lens so as to reduce an average circle of confusion of imaging of the corresponding first part on the first imaging unit, wherein the fitting surface is a plane obtained by fitting according to second spatial distribution information of each image formed by the at least one first part through the main lens of the light field camera.
31. The image acquisition control device according to any one of claims 19 to 22, wherein the tilt control module comprises:
and the second inclination control sub-module is used for inclining the at least one first imaging unit and the at least one first sub-lens relative to a plane parallel to the main lens so as to reduce the average circle of confusion of the at least one first part imaged on the at least one first imaging unit, wherein the at least one first sub-lens is a sub-lens corresponding to the at least one first imaging unit in the sub-lens array of the light field camera.
32. The image acquisition control device of claim 31, wherein the second tilt control sub-module comprises:
the inclination angle determining unit is used for determining a first inclination angle of the first imaging unit and a second inclination angle of the first sub-lens according to the focal length of the first sub-lens, an included angle between a fitting surface and a plane parallel to the main lens, the distance from the optical center of the first sub-lens to the fitting surface along the direction vertical to the plane parallel to the main lens, and the distance from the optical center of the first sub-lens to the first imaging unit along the direction vertical to the plane parallel to the main lens;
and the second inclination control unit is used for inclining the first imaging unit relative to the plane parallel to the main lens according to the first inclination angle and inclining the first sub-lens relative to the plane parallel to the main lens according to the second inclination angle so as to reduce the average circle of confusion of the corresponding first part imaged on the first imaging unit.
33. The image acquisition control device according to any one of claims 26 to 27, 29 to 30, and 32, further comprising:
a second spatial distribution information determining module, configured to determine second spatial distribution information of each image of the at least one first portion formed by passing through a main lens of the light field camera according to first spatial distribution information of the at least one first portion in the scene;
and the fitting surface determining module is used for determining the fitting surface according to the second spatial distribution information.
34. The image acquisition control device according to claim 33, wherein a sum of distances from the respective images to the fitting surface is smallest.
35. The image acquisition control device according to any one of claims 19 to 22, 26 to 30, 32, 34, further comprising:
a to-be-tilted element determining module for determining at least the at least one first imaging unit to be tilted in the imaging unit array.
36. The image acquisition control device according to claim 35, wherein the to-be-tilted element determination module includes:
a first sub-lens determining sub-module, configured to determine at least one sub-lens in the sub-lens array corresponding to a desired sharpness zone of a preview image of the scene as at least one first sub-lens;
and the first imaging unit determining sub-module is used for determining the at least one first imaging unit corresponding to the at least one first sub-lens according to the corresponding relation between the sub-lens in the sub-lens array and the imaging unit of the imaging unit array.
37. An image acquisition control apparatus, comprising:
the system comprises a processor, a communication interface, a memory and a communication bus; the processor, the communication interface and the memory complete mutual communication through the communication bus;
the memory is used for storing at least one instruction; the instructions cause the processor to:
determining a desired sharp imaging spatial region of a scene;
tilting at least one first imaging unit relative to a plane parallel to a main lens of the light field camera to reduce a circle of mean confusion of at least one first portion imaged on the at least one first imaging unit, wherein: the at least one first imaging unit is at least one imaging unit in the imaging unit array of the light field camera for image acquisition of the desired sharp imaging spatial region, and the at least one first part is at least one part of the scene located in the desired sharp imaging spatial region;
the adjusted light field camera performs image acquisition on the scene.
CN201510546276.4A 2015-08-31 2015-08-31 Image acquisition control method and device Active CN107707807B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510546276.4A CN107707807B (en) 2015-08-31 2015-08-31 Image acquisition control method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510546276.4A CN107707807B (en) 2015-08-31 2015-08-31 Image acquisition control method and device

Publications (2)

Publication Number Publication Date
CN107707807A CN107707807A (en) 2018-02-16
CN107707807B true CN107707807B (en) 2020-01-03

Family

ID=61168205

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510546276.4A Active CN107707807B (en) 2015-08-31 2015-08-31 Image acquisition control method and device

Country Status (1)

Country Link
CN (1) CN107707807B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003037767A (en) * 2001-07-24 2003-02-07 Minolta Co Ltd Digital camera
CN101971610A (en) * 2008-03-11 2011-02-09 佳能株式会社 Image capturing apparatus and image processing method
WO2014110484A2 (en) * 2013-01-11 2014-07-17 Digimarc Corporation Next generation imaging methods and systems
CN104486537A (en) * 2014-10-27 2015-04-01 北京智谷技术服务有限公司 Light field acquisition control method and device
CN104506762A (en) * 2014-12-25 2015-04-08 北京智谷睿拓技术服务有限公司 Optical field collecting control method and device and optical field collecting equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003037767A (en) * 2001-07-24 2003-02-07 Minolta Co Ltd Digital camera
CN101971610A (en) * 2008-03-11 2011-02-09 佳能株式会社 Image capturing apparatus and image processing method
WO2014110484A2 (en) * 2013-01-11 2014-07-17 Digimarc Corporation Next generation imaging methods and systems
CN104486537A (en) * 2014-10-27 2015-04-01 北京智谷技术服务有限公司 Light field acquisition control method and device
CN104506762A (en) * 2014-12-25 2015-04-08 北京智谷睿拓技术服务有限公司 Optical field collecting control method and device and optical field collecting equipment

Also Published As

Publication number Publication date
CN107707807A (en) 2018-02-16

Similar Documents

Publication Publication Date Title
JP6047807B2 (en) Method and electronic device for realizing refocusing
CN104410784B (en) Optical field acquisition control method and device
CN108076278B (en) Automatic focusing method and device and electronic equipment
KR102028297B1 (en) Device and method for generating panoramic image
CN104506762B (en) Optical field acquisition control method and device, optical field acquisition equipment
JP6736471B2 (en) Image processing method and image processing apparatus
US8400555B1 (en) Focused plenoptic camera employing microlenses with different focal lengths
WO2018153149A1 (en) Automatic focusing method and apparatus based on region of interest
CN108432230B (en) Imaging device and method for displaying an image of a scene
CN104486537B (en) Optical field acquisition control method and device
CN108335323B (en) Blurring method of image background and mobile terminal
JP6862569B2 (en) Virtual ray tracing method and dynamic refocus display system for light field
US20150177062A1 (en) Information processing apparatus, information processing method, and storage medium
WO2014008320A1 (en) Systems and methods for capture and display of flex-focus panoramas
EP3549334A1 (en) Method for rendering a final image from initial images acquired by a camera array, corresponding device, computer program product and computer-readable carrier medium
CN107534731B (en) Image processing apparatus, image processing method, and program
CN106375650B (en) Focusing method and device, image capture device
CN106375651B (en) Focusing method and device, image capture device
CN114363522A (en) Photographing method and related device
CN106254753B (en) Light field display control method and device, light field show equipment
CN106254804B (en) Light field display control method and device, light field show equipment
CN106303208B (en) Image Acquisition control method and device
CN106375648B (en) Image Acquisition control method and device
CN107707807B (en) Image acquisition control method and device
US20150271470A1 (en) Method of using a light-field camera to generate a full depth-of-field image, and light field camera implementing the method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant