WO2023272485A1

WO2023272485A1 - Control method, image capture apparatus, and image capture system

Info

Publication number: WO2023272485A1
Application number: PCT/CN2021/103091
Authority: WO
Inventors: 吴军
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2023-01-05
Also published as: CN117015975A

Abstract

A control method, an image capture apparatus, and an image capture system. The control method comprises: acquiring a focus information set for at least one object in a focus frame of a current image; obtaining a first operation, the first operation comprising a zoom operation or the first operation being for focus information in the focus information set; and in response to the first operation, controlling a lens of an image capture apparatus to automatically focus on the basis of the focus information set.

Description

Control method, shooting device and shooting system

technical field

The present application relates to the technical field of control, and in particular to a control method, a photographing device and a photographing system.

Background technique

Manual focus is to focus in steps by turning the zoom ring of the lens or pressing the direction keys on the body, so as to obtain a clear image of the target object. In the age of digital cameras, manual focus can be a powerful complement to autofocus. However, the operation convenience of manual focusing is relatively low, and the focusing accuracy is highly dependent on the user's focusing operation level and eyesight level.

Contents of the invention

In view of this, embodiments of the present application provide a control method, a photographing device, and a photographing system to improve the convenience of manual focusing and reduce the dependence of focusing accuracy on the user's focusing operation level and vision level.

In a first aspect, an embodiment of the present application provides a control method applied to a photographing device, the method including: acquiring a focus information set for at least one object in the focus frame of the current image; obtaining a first operation, wherein the first The operation includes a zoom operation or the first operation is directed to the focus information in the focus information set; and in response to the first operation, the lens of the camera is controlled to automatically focus based on the focus information set.

In the second aspect, the embodiment of the present application provides a shooting control device, which includes: a lens for capturing images; one or more processors; a storage device for storing one or more computer programs, and the computer programs are stored in When executed by the processor, the above method is realized.

In a third aspect, an embodiment of the present application provides a photographing system, including: the above photographing device; and a supporting mechanism, which supports the photographing device.

In a fourth aspect, the embodiment of the present application provides a readable storage medium, on which a computer program is stored; when the computer program is executed, the control method as in the embodiment of the present application in the first aspect is implemented.

In a fifth aspect, the embodiment of the present application provides a computer program, including executable instructions, and when the executable instructions are executed, implement the above method.

In the embodiment of the present application, after the focus frame for the current image is determined, in-focus information of one or more objects included in the focus frame is acquired. In this way, the photographing device can automatically zoom to a focal length corresponding to certain focus information according to the focus information and the zoom trigger operation. Unlike the manual zooming in the related art that needs to be manually adjusted from the current focal length to the target focal length, in the embodiment of the present application, the in-focus information of the object included in the focus frame is automatically determined, so that only the zoom trigger operation or for a certain The zoom trigger operation of the in-focus information can realize autofocus based on the in-focus information. During the process of manual assisted focusing, the user is less dependent on human judgment whether the zoom has reached the in-focus focal length corresponding to a certain object. On the premise of meeting the user's manual focus requirements, the embodiment of the present application uses the automatic zoom technology to provide assistance to the manual zoom operation, which improves the convenience of manual focus and reduces the impact of focus accuracy on the user's focus operation level and vision level. Dependency, effectively improving the user's manual zoom experience.

Advantages of additional aspects of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of drawings

The above and other objects, features and advantages of the embodiments of the present application will become more comprehensible through the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application will be described by way of example and non-limitation, wherein:

FIG. 1 is an application scenario of a control method, a photographing device, and a photographing system provided in an embodiment of the present application;

FIG. 2 is a block diagram of a photographing device provided in an embodiment of the present application;

Fig. 3 is a schematic diagram of adjusting the focal length by changing the distance between the lenses provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of a lens provided in an embodiment of the present application;

FIG. 5 is a schematic flowchart of a control method provided in an embodiment of the present application;

Fig. 6 is a schematic diagram of the focal length and focus distance of the lens provided by the embodiment of the present application;

FIG. 7 is a schematic diagram of the focus frame, object and focus information provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of the process of manual assisted focusing based on the zoom ring provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of the process of manual assisted focusing based on the zoom ring provided by another embodiment of the present application;

FIG. 10 is a schematic diagram of the process of manually assisting focusing based on focus information provided by the embodiment of the present application;

Fig. 11 is a schematic diagram of the in-focus intensity curve provided by the embodiment of the present application;

FIG. 12 is a schematic diagram of the processed in-focus intensity curve provided by the embodiment of the present application;

FIG. 13 is a schematic diagram of a process of manually assisting focusing based on focus information provided by another embodiment of the present application;

FIG. 14 is a schematic diagram of the preset key state and the corresponding display image on the display screen provided by the embodiment of the present application;

FIG. 15 is a schematic flowchart of a control method provided by another embodiment of the present application;

FIG. 16 is a schematic diagram of an image displayed when exiting manual assisted focusing provided by an embodiment of the present application;

FIG. 17 is a schematic flowchart of a control method provided by another embodiment of the present application;

FIG. 18 is a schematic diagram of compensating focus information provided by the embodiment of the present application;

FIG. 19 is a schematic diagram of compensating focus information provided by another embodiment of the present application;

FIG. 20 is a schematic structural diagram of a photographing device provided by an embodiment of the present application;

Fig. 21 is a schematic structural diagram of a photographing device provided by another embodiment of the present application;

FIG. 22 is a schematic structural diagram of a photographing system provided by an embodiment of the present application.

detailed description

Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary, and are intended to explain the present application, and should not be construed as limiting the present application.

It should be noted that when a component is said to be "fixed" to another component, it can be directly on the other component or there can also be an intervening component. When a component is considered to be "carrying" another component, the other component may be carried on a surface of the component, or disposed within the component, or disposed within or on the surface of another component carried by the component. When a component is said to be "connected" to another component, it may be directly connected to the other component or there may be intervening components at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are only for the purpose of describing specific embodiments, and are not intended to limit the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. The terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more features.

With the continuous development of camera technology, various types of cameras have been produced. For example, a single-lens reflex camera, a mirrorless camera, etc., can be configured with a zoom lens or a fixed-focus lens. A zoom lens can have parameters such as follow focus, zoom, and aperture. In the era of digital cameras, manual focus (MF) is a powerful supplement to auto focus (AF). In the digital age, MF is still an indispensable function in some application scenarios.

When the user performs manual focusing, he can manually turn the zoom ring to adjust the focal length of the camera lens, so as to obtain a clear image of the target object. The manual focus method largely depends on the human eye's judgment on the image on the display screen or the sharpness of the object in the viewfinder. In addition, the shooting effect is also related to the photographer's proficiency and even the photographer's vision. Traditional single-lens reflex cameras and rangefinder cameras mostly use manual focus to complete the zoom operation. A variety of quasi-professional and professional digital cameras, as well as SLR digital cameras, are equipped with manual focus functions to meet different shooting needs.

Manual focus can be divided into two types: electronic manual focus and full-time mechanical manual focus. Among them, the electronic manual focus can be matched with a large aperture telephoto lens. For example, the electronic circuit of the electronic manual focus detects the amount of rotation of the zoom ring, and based on the amount of rotation, controls the motor to drive the lens group of the lens to zoom to a focal length corresponding to the amount of rotation. Full-time mechanical manual focus does not require power consumption, and the lens group is driven to move through rollers and swivels different from AF.

When performing manual focus, the target object photographed under in-focus conditions can be clearly presented on the display screen, and the clarity can be judged by the user. In some embodiments, when using a manual focus lens, the camera body can give the user a reference focus indication when the focus condition is met. The focusing screen will provide focusing aids such as Fresnel lens and split image, and can also be connected with an external viewfinder magnifying eyepiece, viewfinder magnifying glass, etc. to improve the accuracy of manual focus. The manual focusing function of some automatic lenses is not realized by pure mechanical rotation transmission force, but has a stepping motor to drive the lens. When the user turns the zoom ring, the motor inside the lens will rotate almost synchronously with the user's rotation to drive the lens to focus. The advantage of this method is that the structure is simple and the interior of the lens barrel is well sealed, but there will be a certain delay. This technology can be used in large-aperture telephoto lenses or other lenses with a zoom ring. The zoom ring only acts as a sensor and is not mechanically connected to the focusing lens group. The simplified structure can make the lens smaller. The lens used in the embodiment of the present application may include the above-mentioned stepping motor to realize manual assisted focusing. Of course, manual assisted focusing can also be achieved through an external motor.

In addition, taking digital cameras as an example, manual focus can also be used when autofocus fails. Specifically, manual focus is required when the contrast of the scene is small, the contrast between the subject and the background is too large, the ambient brightness is low, the ambient brightness is too high, there is interference from a high-brightness light source, it needs to be shot through a transparent barrier such as glass, and the subject is outside the focus area. . Cameras with live view can easily use manual focus by zooming in on the image and achieve accurate focus.

However, there are various problems in the manual focus technology, for example, the user needs to turn the zoom ring precisely to make the image of the target object in the image clear. However, it is not easy for the user to determine whether the target object is in focus at the current focal length, and the user needs to turn the zoom ring back and forth repeatedly to determine the in-focus focal length for the target object, which not only depends on the user's operating experience, but also depends on the user's The level of visual acuity to determine the trend of changes in clarity, etc.

The control method, photographing device, and photographing system provided in the embodiments of the present application can effectively improve the convenience of the user when performing manual focusing operations, reduce the dependence of manual focusing on the user's focusing experience and vision level, and improve user experience.

In addition, in order to meet the user's need for manual focusing in multiple scenes, the control method, shooting device, and shooting system provided in the embodiments of the present application can automatically focus on the object in the focus frame when the positional relationship between the lens and the target object changes. The focus information is compensated to improve the accuracy of the focus information. For example, for a shooting device equipped with a manual zoom lens, the lens can be estimated through the measurement information output by the inertial measurement unit (IMU) (such as the original IMU on the camera or the gimbal or the external IMU), the rangefinder, etc. With respect to the distance change amount between target objects, the in-focus information is compensated according to the distance change amount. Allows novice users to capture clear images of the target object even in complex scenes such as scenes where the user is moving and/or the target object is moving.

It should be noted that the above shooting scenes may be shooting scenes for people shooting, animal shooting, still scene shooting, moving scene shooting, video shooting, movie shooting, TV series shooting and the like. The above scenarios are illustrative only, and should not be construed as limitations on the present application. For example, it can be applied to various scenarios requiring manual zooming operations.

In order to facilitate the understanding of the technical solution of the present application, a detailed description will be given below in conjunction with FIGS. 1 to 22 .

Fig. 1 is an application scenario of a control method, a photographing device and a photographing system provided in an embodiment of the present application.

As shown in FIG. 1 , when a user uses a camera to take pictures, he first needs to find a view. After determining the scene to be shot, he needs to adjust the image definition of the target object through a zoom operation to obtain a satisfactory image for the user. The user can adjust the image definition of the target object through manual focusing, but this process is highly dependent on the user's proficiency in manual focusing operations, the user's vision level, and the like.

In addition, users can also obtain a clear image of the target object through automatic focusing. However, in the auto-focus mode, the target object is automatically determined by the camera through an algorithm, and the image of the target object in the focus frame may not be as clear as the image of the non-target object in the focus frame, and it is not easy to adjust.

The photographing device in FIG. 1 may be a camera, a video camera, or other devices with a photographing function such as an intelligent terminal. The shooting function refers to the function of shooting still images and/or moving images. Specifically, the shooting device may be a manually controlled shooting device, or may be an automatic shooting device, and the automatic shooting device may have a manual focus mode and an automatic focus mode. In the auto-focus mode, the photographing device can automatically adjust its optical components such as the lens. For example, the auto-focus motor can control the rotation of the lens to adjust the focal length of the lens. The manual focus mode is an important supplement to the auto focus mode. For example, when the ambient lighting is too dark, the auto focus motor cannot precisely control the rotation of the lens, and manual focus is required at this time. When the user manually focuses on the camera, the user needs to slowly turn the zoom ring of the camera, and in the process of turning the zoom ring, observe whether the object on the screen is clear through human eyes, when the user observes the image of the object on the screen Stop turning the zoom ring when it is clear enough. In the process of manual focusing, the user needs to continuously rotate the zoom ring to find a better focus position, and cannot quickly and accurately find the focus position when the object is in focus.

The shooting device may be a handheld shooting device, or a shooting device mounted on a movable platform such as a drone.

Taking a user holding a camera to shoot a target person as an example, the user may use a manual assisted focusing method to focus. For example, after the user controls the camera to enter manual mode, the focus frame can be displayed on the camera's display screen by half-pressing the shutter button, and the in-focus information of one or more objects in the focus frame can be obtained, and then respond to the user turning the zoom ring Or click a zoom button to trigger automatic zooming, so that the lens of the camera zooms to the in-focus focal length corresponding to the in-focus information of the target object. Take the user holding the gimbal as an example, the gimbal can carry a camera, and there are mechanical buttons on the gimbal to operate the camera. For example, a camera focus mode selection button, a focus adjustment button, etc. may be provided on the pan/tilt.

Referring to FIG. 1 , the camera's display interface may display a framing image and a focus frame. The focus frame may include one or more objects, and the camera automatically determines focus information corresponding to the one or more objects. The in-focus information can be displayed on the display interface. In the display interface as shown in FIG. 1 , a focus distance bar corresponding to the in-focus information is displayed, and the focus distance bar has one or more in-focus focal lengths. In addition, the display interface of FIG. 1 may also have a current focus mark (refer to the unfilled triangle in FIG. 1 ), an in-focus indicator (refer to the filled triangle in FIG. 1 ), and the like.

In addition, in the embodiment of the present application, inaccurate focus information caused by a change in the distance between the photographed object and the photographing device can also be compensated. For example, the subject and/or the photographing device may move, which will cause the distance between the subject and the photographing device to change, resulting in inaccurate focus information for the subject in the existing focus frame. In this embodiment of the present application, based on the displacement of the shooting object and/or the shooting device, the focus information corresponding to the shooting object can be compensated, so as to improve the focusing accuracy and the convenience of focusing operation.

Taking a photographing device mounted on a mobile platform (such as a drone or a land robot) as an example, the user can control the photographing device on the mobile platform or a remote controller communicatively connected with the mobile platform to perform focusing operations. For example, the display interface of the remote controller can be provided with a plurality of virtual buttons for controlling the camera, and the user can realize at least one of the functions of framing, activating the focus frame, triggering autofocus based on in-focus information, or taking images by operating the virtual buttons. .

For example, the pan-tilt shown in FIG. 1 may be a handheld pan-tilt or a pan-tilt mounted on a mobile platform, and the pan-tilt carries a photographing device, and the photographing device may have a focus adjustment component.

For example, a handheld pan/tilt may include a bracket, a handle, and the like. Wherein, the bracket may include a matching motor and a shaft arm, and the motor is used to drive the rotation of the shaft arm to drive the movement of the photographing device.

The pan/tilt may include but is not limited to a single-axis or multi-axis attitude-adjustable structure for fixing the shooting device on the hand-held part. For example, the gimbal allows the camera to be displaced relative to the hand-held portion, or to rotate along one or more axes. There may be a linkage conversion relationship between the pan-tilt and the handheld part, for example, the first movement (such as movement or rotation) of the handheld part can be converted into the second movement of the pan-tilt. vice versa.

In addition, a sensing system may also be included on the pan/tilt. The sensing system may include one or more sensors to sense spatial orientation, velocity, and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom). The one or more sensors include, but are not limited to, rangefinders, GPS sensors, motion sensors, inertial measurement units, or image sensors. The sensing data provided by the sensing system can be used to control the pose, speed and/or acceleration of the camera, etc.

A communication system may also be included on the cloud platform. The communication system can realize the communication between the pan-tilt and the control terminal with the communication system through wired or wireless signals sent and received. A communication system may include any number of transmitters, receivers, and/or transceivers for wireless communication. Communication can be one-way or two-way. Taking two-way communication as an example, data can be transmitted in two directions between the gimbal and the camera. For example, the control data from the pan/tilt can control the operation of the shooting device or other image capture equipment carried by it (capture still or moving images, zoom, turn on or off, switch focus mode, switch imaging mode, change image resolution, change depth of field, changing exposure time, changing viewing angle or field of view).

For example, the photographing device is detachably carried on the bearing seat of the pan-tilt, and the pan-tilt is connected to the photographing device in communication. Parts such as a display screen, a lever, and a dial can be arranged on the cloud platform so as to realize human-computer interaction between the user and the cloud platform. Wherein, a human-computer interaction interface may be displayed on the display screen. The dial includes but is not limited to a focus wheel and/or a zoom wheel.

In the scene where the mobile platform is equipped with a shooting device, the movable translation can also be connected to the control terminal. Multiple provide control instructions, and receive information (such as the position and/or motion information of the carrier or the shooting device, data sensed by the shooting device, such as image data) from one or more of the pan/tilt and the shooting device. In some embodiments, the control data of the control terminal may include instructions about position, movement, and braking, or control over the pan/tilt and/or the shooting device. For example, control data may result in changes in the position and/or orientation of the gimbal. The control data of the control terminal can control the operation of the shooting device or other image capture equipment (capture still or moving images, zoom, turn on or off, switch focus mode, switch imaging mode, change image resolution, change focal length, change depth of field, change exposure time, changing viewing angle or field of view). In some embodiments, communications to the gimbal and/or camera may include information from one or more sensors. Communication information may include sensory information transmitted from one or more different types of sensors, such as GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors. For example, the control terminal may be a remote controller of a mobile platform, or an intelligent electronic device such as a mobile phone, an iPad, or a wearable electronic device that can be used to control the pan/tilt. The control terminal can be far away from the pan-tilt to realize the remote control of the pan-tilt, and can be fixed or detachably installed on the pan-tilt, which can be set according to specific requirements.

In some embodiments, the mobile platform can communicate with remote devices other than the controlling terminal, or with remote devices other than the controlling terminal. The control terminal can also communicate with another remote device and PTZ. For example, the pan/tilt and/or the control terminal may communicate with another mobile platform or a carrier or a photographing device of another mobile platform. The additional remote device may be a second terminal or other computing device (such as a computer, desktop computer, tablet computer, smart phone, or other mobile device) when desired. The remote device can transmit data to the pan-tilt, receive data from the pan-tilt, transmit data to the control terminal, and/or receive data from the control terminal. Optionally, the remote device can be connected to the Internet or other telecommunication networks, so that the data received from the pan/tilt and/or the control terminal can be uploaded to a website or server.

FIG. 2 is a block diagram of a photographing device provided by an embodiment of the present application.

As shown in FIG. 2 , the imaging device 100 has an imaging unit 102 and a lens unit 200 . For example, the imaging unit 102 includes an image sensor 120 , an imaging control unit 110 , and a memory 130 . The image sensor 120 may include at least one of CCD or CMOS. The image sensor 120 outputs the image data of the optical image formed through the plurality of lenses 210 to the imaging control unit 110 . The imaging control unit 110 may be constituted by a microprocessor such as a CPU or an MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 can control the imaging device 100 according to a user operation from a user, and the user operation can be an operation on the lens unit 200 , the display screen of the imaging unit 102 , or a remote controller. The memory 130 may be a computer-readable recording medium, and may include at least one of flash memories such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 130 stores programs and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be disposed inside the housing of the photographing device 100 . The memory 130 may be configured to be detachable from the casing of the photographing device 100 .

The lens unit 200 may include a plurality of lenses 210 , a lens moving mechanism 212 , and a lens control unit 220 . The plurality of lenses 210 may include at least one of a zoom lens, a variable focal length lens, or a follow focus lens. At least part of the plurality of lenses 210 is configured to be movable along the optical axis. The lens unit 200 may be an interchangeable lens that can be attached or detached from the imaging unit 102 . The lens moving mechanism 212 moves at least part of the plurality of lenses 210 along the optical axis. The lens control unit 220 drives the lens moving mechanism 212 according to the lens control command from the imaging unit 102 to move one or more lenses 210 along the optical axis direction. The lens control command includes but not limited to: at least one of a zoom control command or a focus control command.

FIG. 3 is a schematic diagram of adjusting the focal length by changing the distance between lenses provided by the embodiment of the present application.

As shown in Figure 3, the zoom lens can be composed of multiple groups of lens groups (such as L1, L2 and L3). In the embodiment of the present application, the output torque of the motor can be used to rotate the zoom ring or the focus ring to drive the movement of the lens groups, and the lens groups are relatively A change in position will change the overall equivalent focal length of the lens. According to different lens optical designs, the moving lens group may be different, and the relationship between the moving distance of the lens group and the focal length change may be different. In this case, only the mapping relationship needs to be changed, and this embodiment is also applicable. As shown in Figure 4, the focal length f is changed by moving the concave lens L2 in the middle, and the focal length f and the position x of the lens L2 can be considered as a linear relationship, as shown in formula (1).

x＝k1*f+b1 Formula (1)

Among them, k1 and b1 are undetermined constants related to mechanical structure and lens group parameters. In the scene where the zoom ring or the follow focus ring is driven to rotate by an external motor, the motor drives the zoom ring to rotate or the follow focus ring to rotate through gear meshing. The above undetermined constants may be obtained through calibration and/or calculation.

FIG. 4 is a schematic diagram of a lens provided by an embodiment of the present application.

As shown in FIG. 4 , there are threads inside the lens, and a zoom ring and a focus ring are provided to adjust the front-rear distance of the lens group of the lens. The focus or follow-focus of the lens may be to keep the position of the subject and the position of the focal plane of the camera body of the shooting device unchanged. In the process of focusing, the thread on the inner wall of the lens converts the rotation angle into the forward and backward translation distance of the lens group of the lens, which is equivalent to adjusting the object distance when the real-time distance between the shooting device and the object remains unchanged. d and the image distance f (refer to Figure 6) to bring the shooting picture into focus. In addition, the lens can also include an aperture ring for adjusting the aperture.

The photographing device 100 may perform at least one of autofocus, manual focus, or manual assisted focus according to a mode selected by the user, and obtain a clear image of the target object.

Regarding manual focus, 2 assistance schemes as shown below can be included. For example, the camera control unit inside the camera calculates the collected image information, and displays the points with relatively large contrast between adjacent pixels in the viewfinder in a special color to remind the user of the current focus range. This method can be called peaking focus. For example, when the user rotates the zoom ring, the viewfinder screen is enlarged to a specified ratio (such as 1:1) between the display screen and the screen pixels, so that the user can observe whether the focus is clear.

The above two methods are methods for helping the user to judge whether the focus is achieved. However, both methods are not efficient and convenient to use, and they are still fully manual focusing methods. How to improve the speed and accuracy of manual focusing is an urgent problem to be solved.

FIG. 5 is a schematic flowchart of a control method provided by an embodiment of the present application.

As shown in FIG. 5 , the control method may include operation S502 to operation S506.

In operation S502, a set of in-focus information for at least one object in an in-focus frame (ROI) of a current image is acquired.

In this embodiment, the current image may be the image displayed on the display screen of the camera when the user uses the photographing device to view a view. When the camera is in the manual assisted focus mode, the focus frame is displayed on the display screen of the camera (or the focus frame is displayed on the display screen when the user presses the shutter button halfway), and the focus frame can be automatically obtained for one or more objects included in the focus frame. focus information. The field of view corresponding to the focus frame is small and includes a limited number of objects, which will not result in too much focus information, thereby reducing the difficulty for the user to determine the focus information corresponding to the target object from the excessive focus information. The display position and size of the focus frame on the display screen can be specified by the user, or the display position and size of the focus frame on the display screen can be preset, such as being located in the middle of the display screen. The position of the focus frame on the monitor can change as the subject moves.

The in-focus information may include the distance information between the camera and the object in the focus frame collected by the sensor, or determine the distance between the objects corresponding to the adjacent pixels based on the contrast intensity between adjacent image pixels collected at different focal lengths information. Wherein, the distance information may be an object distance, and for each lens, the object distance has a corresponding focal length.

In some embodiments, the set of focus information includes parameters and focus strength associated with the lens.

Wherein, the in-focus intensity can be stored or displayed by means of an in-focus intensity curve or the like.

In some embodiments, the parameters associated with the lens include a focal length and a focus distance of the lens, wherein the focus distance is the distance between the subject and the lens.

FIG. 6 is a schematic diagram of the focal length and focusing distance of the lens provided by the embodiment of the present application.

As shown in Figure 6, the light irradiated to the subject is reflected to the lens, and the light reaches the sensor of the camera after refraction and so on. Wherein, a sensor and a lens with a specific size (such as a specific length and width) have a corresponding shooting range at a specific focal length f. Wherein, the focusing distance may also be referred to as object distance d, which is the distance between the subject and the lens.

In some embodiments, the in-focus intensity curve can be obtained in at least two ways, the contrast mode or the phase mode, and the in-focus intensity curve is all focusable object focal distances within the focus frame (ROI).

Among them, regarding the contrast mode, in the manual assisted focusing mode, the contrast intensity of the object image in the focus frame is calculated at the current focal length, and the contrast intensity value corresponding to the passed focal length is obtained with the rotation of the zoom ring, and the contrast intensity curve is obtained as a composite focal strength curve.

Regarding the phase mode, in the auxiliary focus mode, through the phase focus information integrated by the camera sensor (sensor) such as a distance sensor, all focusable focal lengths in the ROI are obtained in advance, and the focus intensity curve is obtained.

FIG. 7 is a schematic diagram of a focus frame, an object, and focus information provided by an embodiment of the present application.

As shown in FIG. 7 , a focus frame is displayed at the central position of the display image displayed on the monitor. An image of 4 objects is displayed in the focus frame, and the object located in the middle of the 4 objects is the target object. The distance between each object and the camera can be the same or different. When the distance between each object and the camera is different, each object has a corresponding different in-focus focal length: in-focus focal length 1 to in-focus focal length 4 .

In the related art, it is inconvenient for the user to accurately adjust the focal length of the lens to the in-focus focal length 3 to obtain a clear image of the target object during autofocus: it is inconvenient for the user to intervene in the selection of the target focal length. After the autofocus is completed, if the target If the image of the subject is not clear, autofocus is repeated until the image of the target object is clear. If the attempt fails, the user may abandon the shot or use manual focus to adjust the focus of the lens to the target focal length. In the process of manual focusing, it needs to rely on the user's focusing operation experience and vision level.

In the embodiment of the present application, the camera may automatically determine the focus information of each object in the focus frame, such as focus focal length 1 to focus focal length 4 . The in-focus focal length information can be displayed on the display screen. In addition, in order to facilitate the user to know the current focal length of the lens, a sign of the current focal length can also be displayed on the display screen. The user can conveniently select the focal length of the focal point to automatically zoom the focal length of the lens to the focal length of the focal point 3.

As shown in FIG. 7 , multiple objects may be included in the focus frame of the current image. For example, the target object is only one of the multiple objects, and the camera control unit may calculate the focus of each object in the multiple objects when they are in focus. focal length. Wherein, the in-focus focal length of each object can be different or partly the same when in focus, that is to say, each object can correspond to a different in-focus focal length, or some objects in the plurality of objects correspond to the same In-focus focal length.

In some embodiments, the shooting device includes a display screen for displaying at least one focus information in the focus information set, and the focus information includes a focus intensity peak value.

After the camera control unit calculates the in-focus focal length of the object when it is in focus, the display component can display the in-focus focal length indicator, which is used to indicate the in-focus focal length, which is only a schematic illustration here, and does not limit the in-focus The specific shape of the focal length indicator. It should be noted that the focus focal length may not be displayed on the display screen.

The display component can display multiple in-focus focal length indicator marks, and the multiple in-focus focal length indicator marks are used to indicate the in-focus focal lengths corresponding to the multiple objects. The display component can also display a current focal length mark, which is used to indicate the current focal length. It should be noted that the current focal length may not be displayed on the display screen.

As shown in Figure 7, after the camera control unit calculates the focus focal length of each object in the plurality of objects when in focus, the control display screen displays a plurality of in-focus focal length indicator marks, and the multiple in-focus focal length indicator marks can specifically be In-focus focal length indicator 71, in-focus focal length indicator 72, in-focus focal length indicator 73, in-focus focal length indicator 74 as shown in Figure 7; in-focus focal length indicator 71, in-focus focal length indicator 72, in-focus focus The in-focus indicator mark 73 or the in-focus indicator mark 74 is used to indicate the in-focus focal length of at least one object among the plurality of objects when in focus. In addition, the camera control unit can also control the display screen to display a current focal length mark 70 , and the current focal length mark 70 is used to indicate the current focus value corresponding to the current adjustment position of the focus adjustment assembly 21 . The focal length difference indicated between the current focal length mark 70 and a certain in-focus focal length indicator mark, such as the in-focus focal length indicator mark 71 , is related to the difference between the current focus value and the in-focus focal length indicated by the in-focus focal length indicator mark 71 .

In Fig. 7, at the current moment, the adjustment position of the zoom ring is at position A, the current focal length is smaller than the focal length corresponding to the target object (in-focus focal length 3), the current focal length mark 70 is located on the left side of the in-focus focal length indicator mark 73, and the current The larger the indicated focal distance difference between the focal length mark 70 and the in-focus focal length indicator mark 73 is, the greater the difference between the current focus value and the in-focus focal length indicated by the in-focus focal length indicator mark 73 is.

In some embodiments, after acquiring the focus information set for at least one object in the focus frame of the current image, the above method may further include the following operation: updating the focus information of the focus frame by a sensor. For the scene based on the distance between the object and the shooting device obtained by the sensor, the sensor can update the distance between the two in real time, and then update the focus information set. This helps to improve the problem of inaccurate focus information caused by changes in the positional relationship between the object and the photographing device.

In operation S504, a first operation is obtained, wherein the first operation includes a zoom operation or the first operation is for in-focus information in the in-focus information set.

In this embodiment, the zooming process may be triggered by a first operation.

Taking a camera as an example, the lens of the camera has a zoom ring, and when the user turns the zoom ring, the zooming process can be triggered. For example, the camera has buttons for zooming, such as a button for increasing the focal length and a button for reducing the focal length. When the user presses one of the buttons, the zooming process can be triggered. For example, a zooming component is displayed on the display screen of the camera, and the zooming process can be triggered when the user operates the focusing component. For example, a focus intensity curve is displayed on the display screen of the camera, and when the user selects a certain focus focal length from the focus intensity curve, the zooming process can be triggered.

Take the camera gimbal as an example. For example, the gimbal carrying the camera has buttons for zooming, such as a button to increase the focal length and a button to decrease the focal length. When the user presses one of the buttons, the zooming process can be triggered . For example, a focusing component is displayed on the display screen of the pan/tilt, and the zooming process can be triggered when the user operates the focusing component. For example, a focus intensity curve is displayed on the display screen of the gimbal, and when the user selects a certain focus focal length from the focus intensity curve, the zooming process can be triggered.

Take a mobile platform with a remote control as an example. For example, the remote control has buttons for zooming, such as a button for increasing the focal length and a button for reducing the focal length. When the user presses one of the buttons, it can trigger the The zoom process of the camera on the mobile platform. For example, a focusing component is displayed on the display screen of the remote controller, and the zooming process can be triggered when the user operates the focusing component. For example, a focus intensity curve is displayed on the display screen of the remote controller, and when the user selects a certain focus focal length from the focus intensity curve, the zooming process can be triggered. For example, the user may input the first user operation through a lever or a dial or the like.

In some embodiments, the obtaining of the first operation is that the user rotates the zoom ring or clicks a button for zooming by the user, or the like. For example, if at least one piece of focus information is included in the focus information set, an automatic zoom operation is obtained in response to the rotation of the zoom ring. Specifically, obtaining the first operation includes at least one of the following. For example, if the set of focus information includes at least one piece of focus information, in response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, an automatic zoom operation is obtained. For example, if the set of focus information includes at least one piece of focus information, in response to the duration of rotation of the zoom ring being greater than or equal to a preset duration threshold, an automatic zoom operation is obtained. For example, if the focus information set includes at least one focus information, in response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, and the duration of rotation of the zoom ring is greater than or equal to a preset duration threshold, an automatic zoom operation is obtained . Wherein, the preset rotation speed threshold and the preset duration threshold can be calibrated according to experience or through experiments.

In operation S506, in response to the first operation, the lens of the photographing device is controlled to automatically focus based on the focus information set.

In this embodiment, automatic focusing can be performed based on the determined focus information, without the need for the user to manually adjust the zoom ring to perform manual zooming, which not only helps to increase the zooming speed, but also helps to improve the zooming accuracy and reduce the Eliminates the dependence of manual zoom on the user's operating experience and vision level.

For example, in the process of auto-zooming based on the focus information, since there may be multiple objects in the focus frame, the focal lengths corresponding to each object may be different, and the user can adjust the focal length of the camera to the target focal length by gradually zooming. For example, the target focal length is sequentially approached based on each focus information in the focus information set. For example, the target focal length is approached in segments based on each focus information in the focus information set and the focal length change amount input by the user.

For example, in the process of auto-zooming based on the focus information, when there are multiple focus information in the focus frame, the user can directly select the focus information to increase the zooming speed. For example, the user can realize one-step zooming by directly selecting the focal length of the focus.

In some embodiments, the lens may include a zoom ring. Accordingly, the first operation includes turning the zoom ring in a specific direction.

In response to the first operation, controlling the lens of the camera to automatically focus based on the focus information set may include the following operations: repeating the following operations until the focal length of the lens is changed to a focal length corresponding to the target object: in response to the first operation, controlling The focal length of the lens is changed to the target focal length in the focus information set. The target focal length is the focal length that is determined based on the current focal length of the lens and matched in the focus information set based on the rotation direction information of the zoom ring.

Wherein, the target focal length may include any one of the following: the focal length in the focus information set with the smallest difference with the current focal length of the lens, and the focal length in the focus information set with the largest difference with the current focal length of the lens. In this embodiment, the focal length is first adjusted to the closest or farthest focal length, and then the zooming process is repeated to achieve the target focal point step by step.

In some embodiments, there may be multiple peaks (foreground and foreground) for the locked object in the focus frame, and the user can quickly select and focus among different peaks through manual assisted focusing. The effect of manual assisted focusing is similar to the sticky effect: for example, when moving an object to another object, if certain conditions are met, another object will automatically attract the object to the past, without the need for the user to keep the object Move to this other object to improve the convenience of operation.

FIG. 8 is a schematic diagram of a manual assisted focusing process based on a zoom ring provided in an embodiment of the present application.

As shown in FIG. 8 , there are four in-focus focal lengths for objects in the focus frame: in-focus focal length 1 to in-focus focal length 4 . The current focal length is between in-focus focal length 1 and in-focus focal length 2, that is, the camera is not in focus at the current focal length. The user increases the focal length of the camera by turning the zoom ring, and the camera can automatically zoom with the current focal length as the starting point and the focus focal length 2 as the target focal length. During this period, the user does not need to adjust the focal length of the camera to the focal length 2 through precise zooming operations. The zooming process is performed automatically, and its speed is faster than manual zooming to the focal length 2. After zooming to focal length 2, the camera stops zooming and displays the image captured at focal length 2 on the monitor. The automatic zooming process triggered by the first operation is completed once above.

After the user checks the image on the display screen, if the image of the target object is not clear enough, it indicates that the in-focus focal length 2 is not the in-focus focal length corresponding to the target object. The user can increase the focal length of the camera by turning the zoom ring, and the camera can automatically zoom with the focal length 2 as the starting point and the focal length 3 as the target focal length. During this period, the user does not need to adjust the focal length of the camera to the focal length 3 through precise zooming operations. The zooming process is performed automatically, and its speed is faster than manual zooming to the focal length 3. After zooming to focal length 3, the camera stops zooming and displays the image captured at focal length 3 on the monitor. The automatic zoom process triggered by the first operation twice is completed above. After the user checks the image captured at the in-focus focal length 3 on the display screen, if the image of the target object is clear, the user can perform the shooting operation.

It should be noted that, in order to facilitate the user to know the current focal length and whether the focus is currently in focus, a focus prompt may be given to the user. In this embodiment, there are multiple ways of focusing prompts, for example, focusing prompts in visual form, auditory form, mechanical form, and the like.

Taking the focus prompt in a visual form as an example, the focus prompt may be visually displayed on the display component. For example, the prompt information can be displayed on the viewfinder or screen of the shooting device, or the screen of the remote controller. When the focus prompt is displayed on the screen, the image captured by the shooting device and the focus prompt can be displayed on the screen simultaneously, and the focus prompt can be covered or superimposed on the image displayed on the screen. When the user operates the zoom ring to change the focal length of the shooting device, the user can simultaneously see the image and focus prompts to determine whether the image is in focus.

Referring to FIG. 8 , the display component displays include an in-focus focal length mark, a current focus distance mark, a current focus distance mark (in-focus) and the like. In addition, the display component display may also include a target focal length mark and the like. The current focus mark and the current focus mark (focus) can be represented in different shapes, for example, the current focus mark is represented by a triangle, and the current focus mark (focus) is different from the current focus by a five-pointed star, circle, dot, etc. The flags of the flags are indicated.

The current focal length mark and the current focal length mark (in-focus) can also be represented by the same shape and different states (such as filling state, color, size, etc.). For example, the size of the current focus mark (in focus) is larger than the current focus mark, the current focus mark (in focus) has a fill, the current focus mark (in focus) is yellow and the current focus mark (in focus) is green, etc.

In some embodiments, during or after obtaining the first operation in the above method, obtaining the rotation amount of the zoom ring may further include the following operations. First, the auxiliary focal length is determined based on the current focal length of the lens and the rotation amount of the zoom ring. Then, based on the auxiliary focal length, the target focal length is determined from the focus information set, wherein the target focal length is based on the auxiliary focal length as the initial focal length, and is matched in the focus information set according to the rotation direction information of the zoom ring, and the determined and initial focal length The closest focal length.

FIG. 9 is a schematic diagram of a process of manually assisting focusing based on a zoom ring according to another embodiment of the present application.

As shown in FIG. 9 , there are four in-focus focal lengths for objects in the focus frame: in-focus focal length 1 to in-focus focal length 4 . The current focal length is between in-focus focal length 1 and in-focus focal length 2, that is, the camera is not in focus at the current focal length. The user increases the focal length of the camera by turning the zoom ring. The difference from the embodiment in FIG. 8 is that in this embodiment, the amount of rotation generated by the user's rotation of the zoom ring is also acquired. The user can see the blurring degree of the image of the target object in the preview graphics, if the blurring degree is higher. In order to make the image of the target object clear, the larger the focal length needs to be adjusted. In this way, the user can determine a rough zoom range according to the image blur of the target object. If the image of the target object is adjusted from the current focal length to be clear, the zoom ring needs to be rotated at least 1/4 circle or 3/1 circle or more. In order to avoid too many in-focus focal lengths (such as in-focus focal length 2) between the current focal length and the target focal length, resulting in the need for the user to perform the first operation multiple times when using the zoom process shown in Figure 8, the camera can use the current focal length as the starting point , determine an auxiliary focal length based on the amount of rotation of the zoom ring (refer to the dotted triangle mark in the middle state of Fig. ), and zoom automatically. During this period, the user does not need to adjust the focal length of the camera to focus on focal length 2 and focus on focal length 3 through precise zoom operations. The zoom process is automatically performed, and its speed is faster than manual zooming to focus on focal length 2 and focus on focal length 3. In addition, as shown in FIG. 9 , since there is no need to automatically zoom to the focus focal length 2, one autofocus step is reduced, and the zoom speed is further improved.

The embodiment shown in FIG. 9 may also have focus prompt information, and reference may be made to the content related to the above focus prompt information.

In addition, the indicated focal length difference between the in-focus focal length indicator mark and the current focal length mark changes uniformly as the adjustment position of the zoom ring changes. The rotation angle of the zoom ring can also be referred to as the adjustment amount of the zoom ring, and the indicated focal length difference between the in-focus focal length indicator mark and the current focal length mark changes uniformly with the adjustment amount of the zoom ring. For example, the zoom ring surrounds the optical assembly, and the adjustment amount of the zoom ring may specifically be the rotation angle of the zoom ring, and the rotation angle of the zoom ring refers to the change amount of the angle of the zoom ring when it is rotated. When the zoom ring is turned, the indicated focal length difference between the in-focus focal length indicator mark and the current focal length mark changes accordingly. The variation of the indicated focal length difference is directly proportional to the rotation angle of the zoom ring, that is, the indicated focal length difference between the in-focus focal length indicator mark and the current focal length indicator changes uniformly with the rotation angle of the zoom ring.

In some embodiments, the peak value of the in-focus intensity can be displayed in numerical values, graphs, or line segments.

In some embodiments, the display screen is also used to display input components corresponding to the in-focus information set. The input component can perform information input in response to user operations, such as inputting focus information selected by the user through the input component.

For example, in response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set may include: in response to a user operation on the input component, determining the focal length of the target object corresponding to the target object from the focus information set.

In some embodiments, the above method may further include at least one of the following operations. For example, a current focus indicator is displayed, which corresponds to a point in the in-focus intensity curve. For example, in-focus prompt information is displayed, and the in-focus prompt information includes display state change information of the current focus distance mark.

FIG. 10 is a schematic diagram of a process of manually assisting focusing based on focus information provided by an embodiment of the present application.

As shown in FIG. 10 , it is a framing image displayed on the display screen of the camera. When the user performs manual assisted focusing to focus, the display screen also displays a focus frame (the frame shown by the dotted line). The image within the focus frame includes head image (target object) and leaf image (non-target object). Since the distance between each object in the focusing frame and the shooting device is different, the focal length of each object is also different. In order to facilitate the user to quickly select the in-focus focal length corresponding to the target object, in-focus information, in-focus prompt information, and the like may also be displayed on the display screen.

Referring to FIG. 10 , the left side of the focus frame also displays the in-focus focal length in a bar, wherein each scale in the bar can represent a in-focus focal length. In addition, in order to facilitate the user to know the current focal length, a focal length mark may also be displayed at a position corresponding to the in-focus focal length of the bar, as shown by an unfilled triangle in FIG. 10 . When the user needs to select the in-focus information, he can click on a scale in the bar-shaped in-focus focal length. If the image of the target object collected at the focal length corresponding to the scale is not clear enough, the user can click on another scale to zoom to obtain the zoomed image of the target object, and repeat the above operations until the image of the target object is clear. It should be noted that the bar-shaped in-focus focal lengths shown in FIG. 10 are only exemplary, and may also be displayed as arrow lines with scales, line segments with scales, circles with scales, and the like.

FIG. 10 also shows focus prompt information, such as a filled triangle in FIG. 10 , which helps to improve the user's current focus state. For example, when the current focal length is equal to the in-focus focal length in the in-focus information, in-focus prompt information is displayed. The in-focus prompt information may be prompted by a state change of the current focus distance mark, such as filling, changing color or changing size of the current focus distance mark.

Referring to FIG. 10 , it is also possible to acquire the user's selection operation on any in-focus focal length indicator mark among the plurality of in-focus focal length indicator marks. For example, a user's click operation on any in-focus focal length indicator mark among the plurality of in-focus focal length indicator marks is acquired. Referring to Fig. 10, the user clicks on the in-focus focal length indicator mark on the display screen, the display screen may be a touch screen, and the touch screen can sense the user's click operation on the in-focus focal length indicator mark, and send the click operation to to Camera Control.

In this embodiment, by displaying the in-focus focal length mark for indicating the in-focus focal length, and displaying the current focal length mark for indicating the current focal length, when the in-focus focal length mark and the current focal length mark do not overlap or the current focus distance mark is not in focus, Prompt the user to adjust the focal length of the shooting device to achieve focus, so that the user can quickly and accurately find the focal length when the object is in focus when manually focusing.

In some embodiments, the display screen is also used to display an in-focus intensity curve corresponding to the in-focus information set. The in-focus strength can represent the image pixel contrast value of the object in the focus frame, and the larger the contrast value, the higher the in-focus strength. In addition, the in-focus strength can also represent the probability of an object existing at a certain focal length.

For example, the in-focus intensity curve can be generated as follows. First, determine the in-focus information to be displayed. The in-focus information to be displayed is the focal length information in the in-focus information set that meets the display conditions. The display conditions include that the in-focus intensity corresponding to the in-focus information to be displayed is greater than or equal to a preset intensity threshold. Then, an in-focus intensity curve is generated based on the in-focus information to be displayed.

In some embodiments, the in-focus information is displayed in the form of an in-focus intensity curve, and the two-dimensional coordinate system where the in-focus intensity curve is located includes a first coordinate axis representing focal length and a second coordinate axis representing in-focus intensity. The first coordinate axis and the second coordinate axis may be perpendicular to each other. For example, the first coordinate axis may be the X axis based on the long side of the display screen, and the second coordinate axis may be the Y axis based on the wide side of the display screen.

FIG. 11 is a schematic diagram of an in-focus intensity curve provided by an embodiment of the present application.

As shown in FIG. 11 , a two-dimensional coordinate system is displayed, wherein the horizontal axis represents the focal length, and the vertical axis represents the in-focus intensity.

Specifically, draw a two-dimensional On-Screen display (screen menu adjustment method, referred to as OSD), the abscissa is the focus distance (this distance is obtained through focal length conversion), and the ordinate is the focus intensity; there is also a current focus indicator Indicates the current focusing distance (with a simple scale as a reminder).

In the contrast mode, when a contrast curve peak value is not obtained during the movement of the focus ring, only the obtained contrast intensity curve is drawn; the abscissa of the view is adaptively zoomed to only display the obtained contrast intensity range to highlight the contrast intensity content . Once at least one peak is obtained during the movement of the focus ring (the contrast intensity on both sides is lower than the contrast intensity in the middle), this peak (indicator) needs to be marked.

In phase mode, the intensity curve in the focus frame is directly drawn on the OSD, and all focusable points are marked as peak values.

In a certain embodiment, limited by the small display area of the display screen, in order to better display the in-focus intensity curve, the in-focus intensity curve may be processed such as scaling, stretching, cropping, and deformation.

FIG. 12 is a schematic diagram of the processed in-focus intensity curve provided by the embodiment of the present application.

For example, when there are multiple objects in the focus frame and they are far away from the camera, their in-focus strength will be lower than that of objects in the foreground, and the probability that these objects are target objects is also low. In order to reduce the number of in-focus focal lengths, the in-focus information of these objects can be filtered. Referring to Figure 11 and Figure 12, the first peak and the third peak on the right side in Figure 11 have been filtered in the upper image of Figure 12 because of the low in-focus intensity (such as lower than the dotted horizontal line in Figure 11) Lose.

For example, when there is no in-focus information for certain focal lengths or a certain section of focal length, the in-focus intensity curve may be clipped in order to prevent the in-focus intensity curve from occupying too much space. Referring to Fig. 11 and Fig. 12, there is no in-focus intensity between the two peaks on the left in Fig. 11, which has been filtered out in the upper image of Fig. 12 .

For example, in order to improve the aesthetics of the in-focus intensity curve, the in-focus intensity curve can be abstracted into a scale. Referring to the upper figure of Figure 12, there are three peaks with high in-focus intensity, and these three peaks can be projected onto the scale, as shown in the lower figure of Figure 12. Through the above operations, a relatively regular in-focus intensity curve can be displayed on the display screen, the display space occupied by the in-focus intensity curve can be reduced, and the tidiness of the displayed image can be improved.

In some embodiments, there is a first corresponding relationship between at least one piece of focus information in the focus information set and the sub-regions in the focus frame. The first operation includes a sub-region selection operation.

Correspondingly, in response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set may include the following operations. Firstly, the target focal length corresponding to the selected sub-region is determined from the in-focus information set based on the first correspondence. Then, in response to the target focal length, the lens of the photographing device is controlled to automatically focus based on the target focal length.

Wherein, the first corresponding relationship may be fixed. In addition, the first correspondence may be variable.

FIG. 13 is a schematic diagram of a process of manually assisting focusing based on focus information provided by another embodiment of the present application.

As shown in FIG. 13 , the collected images and focus frames are displayed on the display screen. In the previous embodiments, the camera only acquires the in-focus information included in the focus frame, but the user cannot intuitively determine the correspondence between each in-focus information and each object. For example, there are objects A and B in the focus frame, and it is determined by pixel contrast that there are in-focus information A and in-focus information B, and the in-focus information A and in-focus information B can be displayed on the display screen for the user to select. However, the user needs to determine the correspondence between the object A and the object B and the in-focus information A and the in-focus information B based on experience or logical reasoning. Alternatively, the user may sequentially select focus information A and focus information B to determine focus information corresponding to the target object. The efficiency of manual focus needs to be further improved.

In order to facilitate the user to quickly and accurately select the in-focus information corresponding to the target object, the imaging device of this embodiment further provides a first correspondence so as to select the in-focus information corresponding to the target object.

In some embodiments, before obtaining the first operation, the above method may further include the following operation: displaying the in-focus intensity corresponding to at least part of the sub-regions in the focus frame.

In some embodiments, obtaining the first operation may include the following operations. Firstly, in response to a selection operation on a sub-region in the focus frame, the selected sub-region is determined. Then, the target focal length corresponding to the selected sub-region is determined from the in-focus information set based on the first correspondence.

The first correspondence is variable as an example for illustration. Referring to the left figure of FIG. 13 , the camera can divide the display area corresponding to each object from the focus frame by means of image processing, and determine the in-focus information corresponding to each display area. In this way, the corresponding relationship between each display area and the in-focus information can be established. When the user wants the image of the head in the focus frame to be clear, the selection of in-focus information can be realized by clicking the image of the head. For example, image processing is first performed to determine that there are two object images in the focus frame, and then focus information corresponding to each object image is determined through a contrast mode, and the determined focus information is bound to the corresponding object image. When the user clicks on the head image, the camera can determine corresponding in-focus information based on the head image, and then automatically focus based on the in-focus information.

The first correspondence is fixed as an example for illustration. Referring to the right figure of Fig. 13, the focus frame can be divided into multiple sub-regions in advance, and the position of each sub-region relative to the focus frame is fixed. When the in-focus information for each sub-region is obtained, each sub-region can be established. Correspondence between sub-regions and in-focus information. Since the focus frame is divided into multiple sub-areas, the probability of including multiple objects in one sub-area is greatly reduced, effectively reducing the amount of in-focus information corresponding to one sub-area, so that users can quickly click on a sub-area In-focus information corresponding to the target object is determined. If the image of the target object is displayed in a certain sub-area, the sub-area may be clicked to select in-focus information corresponding to the target object. For example, when the user clicks on the sub-area where the head image is located, the camera may determine corresponding focus information based on the head image, and then perform automatic focusing based on the focus information.

It should be noted that the area of the focus frame may be small. In order to facilitate the user to accurately select a sub-area from the focus frame, an area can be provided on the display interface for zooming in on the image in the focus frame. Ease and accuracy of selecting sub-regions.

In some embodiments, the sticky focus mode may be entered at any time when there is at least one in-focus intensity peak. For example, once the focus frame rotates to a certain extent (or speed) in the direction of the peak value, it will automatically and quickly focus to the nearest focus intensity peak in this direction, and the current focus indicator will change color (for example, from red to green) to show focus.

In order to reduce excessive energy consumption, wear and aging caused by frequent acquisition of focus information sets by the photographing device, the operation of acquiring focus information sets for at least one object in the focus frame of the current image may be performed only when conditions are met.

In some embodiments, acquiring the focus information set for at least one object in the focus frame of the current image may include the following operations.

For example, in response to the second operation indicating to enter the manual assisted focus mode, a set of focus information for at least one object in the focus frame of the current image is acquired. Wherein, the form of the second operation may be the same as the first operation, such as pressing a button corresponding to the function of entering the manual auxiliary focus mode, clicking a virtual button corresponding to the function of entering the manual auxiliary focus mode on the display interface, and the like. It should be noted that one button can be used to realize one or more functions, for example, the shutter button can activate different functions in different states, and it can switch to the manual auxiliary focus function and activate the focus frame function in the half-press state. Fully press down the corresponding shooting function.

For example, in response to the third operation representing the zoom trend, a set of focus information for at least one object in the focus frame of the current image is acquired. In this embodiment, when the user performs a zoom operation, an operation of acquiring a focus information set for at least one object in the focus frame of the current image may be performed. The in-focus information obtained a long time before the manual focus operation may be caused by the shaking or moving of the user during the operation, or the shaking or moving of the target object, resulting in the in-focus information changes happened. In order to improve the above situation, the focus information is obtained after the user starts to perform the zoom operation, which effectively reduces the time interval between performing the zoom operation and obtaining the focus information, and helps to improve the accuracy of the obtained focus information .

In some embodiments, in response to the second operation indicating to enter the manual assisted focus mode, acquiring the focus information set for at least one object in the focus frame of the current image may include the following operations. For example, in response to the second operation, the focus information set for at least one object in the focus frame of the current image is acquired based on the sensor.

Among them, the sensor can be used to measure the distance between the object and the camera (or lens group). The sensor can be set on the shooting device, such as a built-in sensor of the camera or an external sensor. The sensor can be arranged on a bracket carrying the shooting device, such as on a mounting plate on which the shooting device is installed, on a bracket of a pan-tilt, or on a handle assembly of a pan-tilt. The sensor can be set on a mobile platform equipped with a camera, such as a drone or a land robot. The specific setting position of the sensor is not limited.

The sensors include but are not limited to at least one of the following: direct measurement sensors such as laser range finders and infrared range finders, and accumulative measurement sensors such as inertial measurement units (IMUs), speedometers, and acceleration sensors. Wherein, the sensor can be arranged on the photographing device, the pan/tilt equipped with the photographing device, or the mobile platform equipped with the photographing device, so as to determine the attitude information of the main body of the inertial measurement unit. For example, for the pan/tilt, in order to facilitate determining the attitude information of the photographing device, the inertial measurement unit may be arranged on the fixing mechanism of the photographing device for measuring the attitude information of the fixing mechanism. The inertial measurement unit may be at least one of an accelerometer or a gyroscope, and may be used to measure the attitude and acceleration of the photographing device.

The photographing device held by the user is taken as an example for illustration. The monocular (single lens) shooting device is equipped with sensors such as structured light and depth of field detection (also known as time-of-flight, Time of flight, TOF for short), etc., which can realize distance measurement. The binocular photographing device or the multi-purpose photographing device can calculate the distance based on the images captured for the same object under the respective perspectives of the binoculars.

Sensors such as IMU can be used to assist in determining the distance. For example, in a scene where the distance between the target object and the shooting device is known, the displacement and direction of the shooting device itself can be detected based on the IMU, so that the target object can be updated based on the distance and displacement. distance from the camera. IMU is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. A three-axis gyroscope and a three-axis accelerometer can be installed in the IMU, and the pose of the object can be calculated by measuring the angular velocity and acceleration of the object in three-dimensional space.

For example, the inertial measurement unit may include a gyroscope and/or an accelerometer, the gyroscope may be used to determine the angular acceleration information of the gimbal, and the accelerometer may be used to determine the acceleration information of the gimbal. Since the shooting device and the gimbal can be fixed to each other, the angular acceleration information and acceleration information of the gimbal can be used to characterize the attitude information and acceleration information of the shooting device.

For example, in the phase mode, through the phase focus information integrated by the camera sensor, all focusable focal lengths in the ROI are obtained in advance, and the focus intensity curve is obtained.

It should be noted that, in order to ensure the accuracy of the in-focus information: when the recalculation condition is met, output a re-ranging prompt message or forcefully stop using the information output by the IMU to determine the distance between the object and the camera. Due to the limited detection accuracy of the IMU, or the influence of the IMU's own characteristics (such as the detection accuracy for high-acceleration scenarios is lower than that for low-acceleration scenarios), the applicable scenarios for determining the moving distance of the camera based on the IMU after one initialization are limited. . Therefore, the user can be promptly prompted to re-initialize the distance between the object and the photographing device, so as to improve the accuracy of the obtained focus information.

For example, the reinitialization distance condition includes at least one of the following: the inertial measurement unit has experienced a power-off operation, the camera or the gimbal has experienced a power-off operation, the displacement of the camera or gimbal in the power-off state exceeds the recalibration displacement threshold, or the inertial measurement The acceleration of the unit is greater than the set acceleration threshold.

In some embodiments, in response to the second operation indicating to enter the manual assisted focus mode, acquiring the focus information set for at least one object in the focus frame of the current image may include the following operations. For example, in response to the second operation, by adjusting the focal length of the lens, based on the pixel contrast of the at least one object in the focus frame of the current image, the focus information set for at least one object in the focus frame of the current image is acquired.

For example, the distance between the target object and the camera and the corresponding in-focus focal length may be determined by determining the distance based on the amount of blur of multiple images captured when the positional relationship between the lens and the imaging surface is different. Here, this method is referred to as a Bokeh Detection Auto Focus (BDAF for short) method.

For example, the amount of blur (Cost) of an image can be represented by the following equation (2) using a Gaussian function. In Equation (2), x represents a pixel position in the horizontal direction. σ represents the standard deviation value.

By aligning the focus lens to the lens position corresponding to the minimum point of the blur amount curve of the image, it is possible to focus on the target object included in the image.

In some embodiments, acquiring the focus information set for at least one object in the focus frame of the current image based on the sensor may include the following operations.

Firstly, the in-focus focal length of at least one object in the focus frame of the current image is acquired by the sensor according to a preset sampling period, wherein the image captured based on the in-focus focal length satisfies the preset imaging condition.

Then, at least part of the in-focus focal length and the in-focus intensity information corresponding to at least part of the in-focus focal length are used as in-focus information.

In some embodiments, acquiring the focus information set for at least one object in the focus frame of the current image based on the pixel contrast of the at least one object in the focus frame of the current image by adjusting the focal length of the lens may include the following operations.

Firstly, during the process of adjusting the focal length of the lens from the first preset focal length to the second preset focal length, image information of multiple shooting surfaces is obtained. Wherein, the first preset focal length may be any one of the current focal length, the minimum focal length of the lens, or the maximum focal length of the lens. The second preset focal length may be any one of a focal length having a designated focal length difference relative to the current focal length, a maximum focal length of the lens, or a minimum focal length of the lens.

Then, the in-focus shooting surface is determined from the plurality of shooting surfaces, and there are pixels whose adjacent pixel contrast exceeds a preset contrast threshold in the image information of the in-focus shooting surface.

Next, at least part of the in-focus focal length corresponding to the in-focus shooting surface and the corresponding in-focus intensity are used as in-focus information.

It can be realized by the above method: calculate the contrast intensity of the image in the focus frame at the current focal length, and obtain the contrast intensity value corresponding to the passed focal length with the rotation of the zoom ring, and then obtain the contrast intensity curve as the in-focus intensity curve.

Specifically, firstly, when the lens and the imaging plane are in the first positional relationship (for example, the first focal length), the first image is captured and stored in the memory. Next, by moving the focusing lens or the image sensor in the direction of the optical axis of the lens group, the lens and the imaging surface are in a state of a second positional relationship (such as a second focal length). A second image is taken at this second focal length and stored in memory. This determines the amount of image blur for the same object in the first image and the second image. Repeating the above process for several times can obtain: multiple image blurring amounts for the same object at different focal lengths, and the multiple image blurring amounts can form an image blurring amount curve. The focal length corresponding to the minimum point of the image blur amount curve is taken as the target focal length. In addition, the object distance between the target object and the lens can also be obtained based on the target focal length.

For example, like mountain-climbing AF, the focus lens or image sensor can be moved in the direction of the optical axis without exceeding the focus point. The amount of movement of the focus lens or the image sensor can be, for example, 5 μm, 10 μm, 15 μm, or the like.

In a specific embodiment, adjusting the focal length of the lens from the first preset focal length to the second preset focal length includes: adjusting the focal length of the lens from the minimum focal length of the lens to the maximum focal length of the lens.

For example, after activating the focus frame, adjust the focal length of the lens from the minimum focal length of the lens to the maximum focal length of the lens, so that the focal plane and the corresponding focal plane can be determined based on the contrast values of the images collected at each focal length. In-focus focal length. A plane of focus can correspond to a sharp image of one or more objects.

In some embodiments, the in-focus information of the object included in the focus frame may be acquired after the zoom operation is detected. Specifically, the lens includes a zoom ring.

Correspondingly, in response to the third operation representing the zoom trend, acquiring the focus information set for at least one object in the focus frame of the current image may include the following operations.

First, a third operation is acquired, which causes the zoom ring to rotate. Wherein, the operation manner of the third operation may be the same as that of the first operation, which is not limited here.

Then, in response to the third operation, during the process of adjusting the focal length of the lens from the current focal length to the maximum focal length or the minimum focal length, image information of multiple shooting surfaces is obtained.

Next, the in-focus shooting surface is determined from the plurality of shooting surfaces, and there are pixels whose adjacent pixel contrast exceeds a preset contrast threshold in the image information of the in-focus shooting surface.

Then, use at least part of the focus focal length and the corresponding focus strength as the focus information.

In some embodiments, obtaining the second operation may include at least one of the following: receiving an operation on a preset key, receiving an operation on a preset interactive component, or receiving a preset operation gesture.

Wherein, the preset button can be set on: the photographing device, the pan/tilt carrying the photographing device (such as the handle assembly of the pan/tilt), and the control terminal (such as a remote controller, etc.) of the mobile platform carrying the photographing device.

The preset interactive component can be displayed on the display screen of the shooting device, the display screen of the pan/tilt carrying the shooting device, or the display screen of the control terminal of the mobile platform carrying the shooting device.

The operation gestures can be collected by an image processor or a remote controller.

In some embodiments, obtaining the second operation includes: generating the second operation when the preset key is pressed with a first specified displacement, wherein the preset button has at least two specified displacements, and the first specified displacement is at least two specified displacements. One of the displacements. Wherein, the preset button may be a shutter button or the like.

In some embodiments, after the preset key is pressed for a first specified distance, the above method may further include the following operations.

Firstly, a fourth operation is obtained. The fourth operation is a second designated displacement when the preset key is pressed, wherein the first designated displacement and the second designated displacement are respectively one of at least two designated displacements. Then, in response to the fourth operation, image information is acquired.

In some embodiments, in the manual focus mode, once the user presses the shutter button halfway, the user enters the manual assisted focus mode. In this mode, the focus object in the focus frame is locked by IMU and visual means. Even if the lens and/or body shakes, or the lens and/or body moves, the focus frame is still locked to the focus object, which can support Focus first and then compose the picture.

The following takes the second operation and the fourth operation obtained through the shutter button of the camera as an example for illustrative description.

FIG. 14 is a schematic diagram of preset button states and corresponding displayed images on the display screen provided by the embodiment of the present application.

As shown in Figure 14, the state of the shutter button is shown on the left, and the display image corresponding to the state of the shutter button is shown on the right side of the display. The upper figure of FIG. 14 shows a framing image. When the button is half-pressed, the second operation is obtained, and a focus frame is displayed in the framing image. In the lower figure of FIG. 14 , after the user fully presses the shutter button, the image is collected and saved, and the captured image can be displayed on the display screen. After the user fully presses the shutter button and releases the shutter button, options: save and delete buttons may be shown on the display screen for selection by the user.

By integrating the button for entering the manual auxiliary focus function and the button for the shooting function into the same button, there is no need to add new buttons, and the space utilization rate is improved.

In some embodiments, in order to improve the accuracy of the focus information to ensure the clarity of the target object in the captured image, the manual assisted focus mode may be exited after the conditions for exiting the manual assisted focus mode are met. Wherein, when the manual assisted focusing mode condition is met, the existing in-focus information may result in a lower definition of the captured image of the target object.

Fig. 15 is a schematic flowchart of a control method provided by another embodiment of the present application.

As shown in FIG. 15 , the above method may further include operation S1508 after performing operation S306 to obtain the second operation.

In operation S1508, if it is determined that the condition for exiting the manual assisted focusing mode is satisfied, the manual assisted focusing mode is exited.

In some embodiments, the conditions for exiting the manual assisted focusing mode include at least one of the following.

For example, the preset button is in a state corresponding to exiting the manual assisted focusing mode. When the user wishes to exit the manual assisted focus mode, the mode can be exited. For example, the user releases the shutter button, the user selects an auto-focus mode, the user turns off the phone, and so on.

For example, the target object moves out of the focus frame, wherein there are multiple objects in the focus frame, and there is a second corresponding relationship between each object and the focus information in the focus information set. The target object may be an object corresponding to in-focus information determined by the user. Referring to Fig. 8, if the user zooms the lens focal length to the in-focus focal length 3, the shutter button is not released, and the zoom ring is not turned again, however, the image of the object corresponding to the in-focus focal length 3 is already outside the focus frame , it can be determined that the condition for exiting the manual assisted focus mode has been satisfied.

For example, the shooting device can only determine the in-focus information of the focus frame by adjusting the lens, and the target object is displaced in the opposite direction or in the opposite direction relative to the lens, wherein the focus frame includes multiple objects, and each object is combined with the in-focus information There is a second corresponding relationship between the in-focus information. On the premise that the in-focus information has been determined, if the target object is displaced in the opposite direction or in the opposite direction relative to the lens, it is not convenient to update or compensate the in-focus information through the contrast mode. In order to ensure the accuracy of the focus information, it may be determined that the current condition for exiting the manual assisted focus mode is satisfied.

In addition, for the scheme of determining the distance between the object and the shooting device at least based on the information output by the IMU, the conditions for exiting the manual assisted focusing mode may also include at least one of the following: the inertial measurement unit has experienced a power-off operation, the camera or the gimbal After a power-off operation, the displacement of the camera or gimbal in the power-off state exceeds the recalibration displacement threshold, or the acceleration of the inertial measurement unit is greater than the set acceleration threshold. When at least one of these conditions is met, the accuracy of the focus information cannot be guaranteed, and the manual assisted focus mode can be exited.

FIG. 16 is a schematic diagram of an image displayed when exiting manual assisted focusing according to an embodiment of the present application.

As shown in FIG. 16 , the upper left image is the image displayed in the focus frame after the user performs a zoom operation. There is a first distance between the user avatar and the leaves in the image. After completing the manual assisted focusing, the user can perform image acquisition operations.

In the upper right picture of Fig. 16 , after the user performs the zoom operation, the subject moves before shooting, resulting in a second distance between the user's avatar and the leaves in the image, but at this time, the user's avatar and the leaves are still in the image In the focus frame, it can be considered that the distance between the subject and the shooting device has not changed significantly, and autofocus based on the focus information can obtain a clear image of the subject. At this point, you can still shoot without exiting the manual assisted focus mode.

In the lower picture of Figure 16, after the user performs the zoom operation, the subject moves before taking pictures, causing the user's avatar in the image to be outside the focus frame. It can be considered that the distance between the subject and the shooting device has changed. Autofocus based on the existing focus information has a high probability that a clear image of the subject cannot be obtained. At this point, you can exit the manual assisted focus mode, such as the focus frame is no longer displayed. Of course, users can re-enter manual assisted focus mode to focus and shoot.

For example, if at least one of the following situations occurs: the focused object moves out of the screen, the user releases the shutter button, or the focused object moves vertically (in contrast mode), then the manual assisted focusing mode can be exited.

It should be noted that if the focus frame moves due to tracking the subject, after exiting the manual assisted focus mode, the focus frame can return to the user-specified position in the normal preview mode, such as the center of the display screen.

In some embodiments, in order to improve the accuracy of the focus information in the focus information set, the existing focus information may also be updated.

Fig. 17 is a schematic flowchart of a control method provided by another embodiment of the present application.

As shown in FIG. 17 , after acquiring the focus information set for at least one object in the focus frame of the current image, the above method may further include operation S1710.

In operation S1710, the in-focus information set is compensated based on a change amount of the in-focus information in the in-focus information set.

In some embodiments, compensating the focus information set based on the change amount of the focus information in the focus information set may include the following operations. First, determine the pose change information of the lens. Then, the focus information set is compensated based on the pose change information of the lens.

For example, the in-focus information can be compensated according to the position change amount. For example, the in-focus focal length of the object in the focus frame is compensated according to the amount of change in the position of the camera.

In this embodiment, the position change amount can be used as the object distance change amount of the shooting object relative to a certain lens, based on the object distance change amount, the image distance change amount for the certain lens can be determined, and then based on the distance change amount to determine The set of focus information is compensated to ensure the accuracy of the focus information and improve the focus accuracy of the automatic zoom operation based on the focus information.

The object to be photographed may be stationary. For example, when the object to be photographed is stationary, only the IMU may be used to calculate the amount of change in the position of the photographing device to determine the amount of change in the distance between the photographing object and the photographing device. In addition, the photographed object may also be moving. In this case, the change amount of the distance between the photographed object and the photographic device determined in other ways may be optimized based on the position change amount of the photographic device calculated by the IMU.

For example, in the related art, the change amount of the distance between the shooting object and the shooting device may also be determined in the following manner. Taking the focus scene as an example, you can use Time of Flight (TOF) for target distance measurement, screen contrast for focus position search (abbreviated as CDAF), or measure and estimate the diagonal position through PDAF pixels. , to achieve focus control.

A scene in which the subject is moving is taken as an example for illustration.

In some embodiments, determining the pose change information of the lens may include the following operations: determining the pose change information of the lens based on an inertial measurement unit, where the pose change information includes at least one of displacement information and angle change information.

TOF, CDAF and PDAF can obtain the in-focus information of the objects included in the focus frame when entering the manual assisted focus mode, and then use the position change of the shooting device calculated by the IMU to optimize such as TOF, CDAF or PDAF in some The distance measurement effect in the scene.

For example, TOF needs to be equipped with a dedicated distance detection device, which is costly; some gimbals do not have this device, and a corresponding distance detection device needs to be added; the frame rate of TOF is low, such as 10fps, which may cause some key frames The lack of the relative distance change between the camera and the target cannot meet the needs of focus or zoom scenarios.

For example, the CDAF method has the same refresh rate as the screen, but consumes more computing resources; requires multi-frame convergence; and has obvious breathing effects, which cannot well meet the operational requirements for focus or zoom in some scenarios.

For example, the PDAF method has the same refresh rate as the screen, but it has certain requirements for light intensity, which cannot meet the needs of follow-focus or zoom scenes when the light is poor.

Optimizing the distance measurement effect in some scenarios such as TOF, CDAF or PDAF based on the position change of the camera device calculated by the IMU may include the following operations.

First, the focus system obtains the focus information of each object in the focus frame through TOF, CDAF, or PDAF. Assuming that the target does not move, or the movement of the photographer is the main movement factor, at this time, use the IMU or IMU and Visual Inertial Odometry (Visual Inertial Odometry, VIO for short) to obtain the camera position change at a high frame rate.

The calculated camera position change is used to estimate the relative distance change between the camera and the target, and is used to fine-tune the results of the focusing system (such as the upper key frame for missing the relative distance change between the camera and the target) Fine-tune the latest relative distance change in one frame or before to predict the relative distance change between the camera and the target corresponding to the key frame.

The principle of distance calculation is given below. Through vector rotation, the acceleration value measured in the sensor coordinate system is rotated to the geographic coordinate system, as shown in formula (3).

in,

is the rotation matrix from the sensor frame to the geographic frame.

is the sensor measurement vector under the sensor system.

is the sensor measurement vector under the geographic system. When the motion only includes translation, subtract the gravitational acceleration g to obtain the translation acceleration.

When the accelerometer integrates from rest, the initial velocity vel (0) = 0, and the moving velocity is shown in formula (4).

Where acc is the acceleration of translation, vel is the moving speed of the camera, and the distance from the camera to the object can be expressed as shown in formula (5).

d=Δd+d ₀ formula (5)

Among them, Δd is the amount of position change,

d ₀ is the distance from the camera to the subject when the integration starts, and d is the distance from the camera to the subject. The acceleration data can be used to calculate Δd in real time, but the initial distance is related to the shooting scene and can be determined by the methods mentioned above (TOF, etc.).

During the continuous change of the object distance d, the focus information compensation is performed according to the object distance d, so that the image of the photographed object is clear.

The object distance d can be obtained by integrating the acceleration obtained by the IMU twice.

For example, the expression of the image distance f obtained from similar triangles can be shown in formula (6).

Wherein, frame_width is the image width of the subject, and view_width is the width of the subject, as shown in FIG. 6 .

FIG. 18 is a schematic diagram of compensating focus information provided by an embodiment of the present application.

As shown in FIG. 18 , a user holds a camera or a gimbal carrying a camera to shoot a target object. After entering the manual assisted focusing mode, after determining the in-focus focal length of the target object, the composition can be adjusted according to the user's wishes. As shown in Figure 18, the user can hold the pan-tilt to move horizontally and/or vertically relative to the subject to optimize the composition. However, after the user moves relative to the subject, the distance between the camera and the subject changes. If you still use the existing in-focus focal length to shoot, it may cause the image of the subject in the captured image to be unclear. Therefore, the in-focus information may be compensated during the process of the user moving relative to the object to be photographed based on the method described above. For example, if the user is 0.5 meters away from the subject, the IMU will send the detected 0.5 meters to the camera control unit, and the camera control unit will use the 0.5 meters to compensate the existing distance between the user and the subject. Then, the compensated in-focus focal length is determined based on the compensated distance, so as to update the in-focus focal length of the photographed object. It should be noted that the above 0.5 meters is only for the convenience of understanding the solution of this application. In fact, the IMU will output the displacement information of the camera according to the preset cycle.

Through the above method, the focus information such as the focal length of the collection can be updated in time. For example, when the lens body moves horizontally, the peak value calculates the moving distance based on the IMU to track the focus target and compensate the focus strength curve. For example, when the lens body moves relative to the subject, the IMU can be used to calculate the moving distance to compensate the in-focus intensity curve.

In some embodiments, there are multiple objects in the focus frame of the current image, and there is a second corresponding relationship between each object and the focus information in the focus information set.

Correspondingly, compensating the focus information set based on the change amount of the focus information in the focus information set may include the following operations. First, the pose change information of at least one object is determined. Then, based on the pose change information of the at least one object and the second corresponding relationship, the focus information corresponding to each of the at least one object in the focus information set is compensated. Wherein, the pose change information may include: position change information and/or attitude change information.

In some embodiments, determining the pose change information of at least one object may include the following operations: first, determine at least one object included in the focus frame through image recognition, and then determine the position of at least one object included in the focus frame through image processing Posture change information. The image of the object in the focus frame is obtained through image recognition, which facilitates the binding between the object image and the in-focus information, so as to update the in-focus information corresponding to the object image.

In some embodiments, determining the pose change information of at least one object included in the focus frame through image processing may include the following operations. First, calculating displacement information of the sub-image of the at least one object in the current image. Then, based on the respective in-focus information of the at least one object and the displacement information of the at least one object in the current image, the pose change information for the at least one object in the focus frame of the current image is respectively calculated. Wherein, the in-focus information may include in-focus focal length and the like. For example, if the subject moves horizontally, the subject can be tracked visually, and the movement distance can be calculated in combination with lens parameters to compensate for the in-focus intensity curve.

For example, when the subject moves vertically, in the phase mode, the OSD and in-focus intensity curve can be calculated and updated in real time. In the contrast mode, it is impossible to accurately compensate the focus intensity curve distance, and the user can be prompted to switch to the phase mode for compensation, or exit the manual assisted focus mode.

FIG. 19 is a schematic diagram of compensating focus information according to another embodiment of the present application.

As shown in FIG. 19 , after the camera determines the in-focus information corresponding to the target object, the camera does not move, and the target object moves in a vertical direction relative to the direction connecting the target object and the camera. On the premise that the target object does not move out of the focus frame, the in-focus information of the target object can be compensated. For example, if the in-focus focal length of the target object is known, the object distance d can be determined based on the in-focus focal length, and then the real moving distance of the target object can be calculated based on the moving distance of the image of the target object in the focus frame, so that The distance d' between the moving target object and the camera.

Those skilled in the art should be able to understand that although the above embodiments have given the focus information compensation method when the camera moves and the subject does not move, and the focus information compensation method when the subject moves and the camera does not move . However, the same can be applied to the focus information compensation method when both the camera and the subject are moving. For example, by combining the processing results of the above two methods, it can be applied to a scene where both the camera and the subject are moving.

In some embodiments, the focus frame includes multiple objects, and there is a second corresponding relationship between each object and the focus information in the focus information set.

Correspondingly, compensating the focus information set based on the change amount of the focus information in the focus information set may include the following operations.

First, the pose change information of the lens and the pose change information of multiple objects included in the focus frame are determined.

Then, the focus information set is compensated based on the pose change information of the lens and the pose change information of the multiple objects included in the focus frame.

In the embodiment of the present application, the user can not only retain the flexibility of the user to quickly select the subject in the manual assisted focusing mode, but also realize fast and efficient automatic focusing, and can still focus quickly when the camera and the subject are moving , effectively improving the user experience.

In the embodiment of the present application, by rotating the focus ring of the lens or other selection buttons, the user can quickly and automatically find the closest corresponding focal length for zooming according to the existing focus strength information, which effectively improves the convenience of manual zooming .

The embodiment of the present application provides a two-dimensional OSD in-focus peak indication mechanism, which facilitates the user to intuitively determine the relationship between the current focal length and the in-focus focal length of the object in the focus frame, and facilitates the user to select the desired in-focus focal length.

In the embodiment of the present application, when the shooting device and/or the shooting object is in motion, the in-focus strength information of the object in the focus frame can be compensated, so that the user can realize the shooting method of focusing first and then composing the picture, and there is no need to change the composition Then perform the focusing operation again to meet the user's needs for diverse shooting techniques and improve the convenience of the focusing operation.

Another aspect of the present application also provides a shooting control device.

FIG. 20 is a schematic structural diagram of a photographing device provided by an embodiment of the present application.

The photographing device 2000 carries a photographing device and an inertial measurement unit, and the inertial measurement unit is used to measure attitude information of the photographing device. As shown in FIG. 20 , the photographing device 2000 includes: at least one processor 2010 and a memory 2020 . The memory 2020 stores computer-executable instructions.

At least one processor 2010 executes the computer-executed instructions stored in the memory 2020, so that the following steps are implemented when executing the computer-executed instructions:

First, a collection of in-focus information for at least one object in the in-focus frame of the current image is acquired.

Then, a first operation is obtained, wherein the first operation includes a zoom operation or the first operation is directed to focus information in the focus information set.

Next, in response to the first operation, the lens of the photographing device is controlled to automatically focus based on the focus information set.

In order to facilitate the interaction between the user and the pan/tilt, the photographing device may be provided with an input unit and/or an output unit.

For example, the input unit may be used to input the user's operation instructions to the photographing device, and the input unit may include a rotary wheel, buttons, and a touch screen for realizing human-computer interaction.

Specifically, the component for realizing human-computer interaction may include a display screen for displaying an interactive interface, and the user may input control instructions in the interactive interface.

For example, the photographing device may further include a display screen for displaying a user interaction interface.

In addition, the photographing device may further include a state (mode) prompting component. For example, the photographing device may include an indicator light.

It should be understood that the input unit may include other components or parts besides the wheels and keys for realizing human-machine interaction, for example, it may have a switch of the camera, and the like.

A processor may be provided in the input part for processing input control instructions, or sending and receiving signals, and the like. Of course, the processor can also be arranged in the lens.

Optionally, the processor can be a central processing unit (Central Processing Unit, referred to as CPU), and the processor can also be other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (application specific integrated circuit (ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The processing unit can be connected with a non-volatile computer-readable storage medium 2020 . The non-transitory computer-readable storage medium 2020 may store logic, code and/or computer instructions 2021 executed by the processing unit for performing one or more steps. The non-volatile computer-readable storage medium 2020 may include one or more storage units (removable medium or external storage, such as SD card or RAM). In some embodiments, the measurement information of the IMU can be directly transmitted and stored in the storage unit of the non-volatile computer-readable storage medium 2020 . The storage unit of the non-transitory computer-readable storage medium 2020 may store logic, codes and/or computer instructions 2021 executed by the processing unit to perform various embodiments of the various methods described herein. For example, the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to perform the zoom and/or focus functions described above. In some embodiments, the storage unit of the non-volatile computer-readable storage medium 2020 can store the processing results generated by the processing unit.

In some embodiments, the processing unit can be connected with the control module to control the state of the motor.

The processing unit can also be connected with the communication module to transmit and/or receive data with one or more peripheral devices (such as terminals, display devices, or other remote control devices). Any suitable communication method may be utilized here, such as wired communication or wireless communication. For example, the communication module may utilize one or more local area networks, wide area networks, infrared, wireless, Wi-Fi, peer-to-peer (P2P) networks, telecommunications networks, cloud networks, and the like. Optionally, relay stations, such as signal towers, satellites, or mobile base stations, may be used.

The input module on the camera device may include one or more input mechanisms to obtain input generated by the user through operating the input module. Input mechanisms include one or more joysticks, switches, knobs, slide switches, buttons, dials, touch screens, keypads, keyboards, voice controls, gesture controls, inertial modules, and the like. An input module may be used to obtain user input for controlling any aspect such as a camera, lens, or components thereof.

The processing unit can be connected with the memory. The memory includes volatile or non-volatile storage media for storing data, and/or logic, codes, and/or program instructions executable by the processing unit for executing one or more rules or functions. The memory may include one or more storage units (removable media or external memory such as SD card or RAM). In some embodiments, the data input to the module can be directly transferred and stored in the storage unit of the memory. The storage unit of the memory may store logic, code and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein. For example, the processing unit can be used to execute instructions to cause one or more processors of the processing unit to process and display sensing data (such as images) obtained from a camera or motor, and control instructions generated based on user input, including motion instructions and target object information, and cause the communication module to transmit and/or receive data, etc. The storage unit may store sensing data or other data received from an external device such as a mobile platform. In some embodiments, the storage unit of the memory may store the processing results generated by the processing unit.

For specific content, refer to the same part of the previous embodiment, and details are not repeated here.

In some embodiments, the lens includes a zoom ring. The first operation includes turning the zoom ring in a certain direction.

Correspondingly, in response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set includes: repeating the following operations until the focal length of the lens is changed to a focus focal length corresponding to the target object: in response to the first operation, controlling the lens The focal length is changed to the target focal length in the focus information set. The target focal length is the focal length that is determined based on the current focal length of the lens and matched in the focus information set based on the rotation direction information of the zoom ring.

In some embodiments, the target focal length includes any one of the following: the focal length in the focus information set with the smallest difference from the current focal length of the lens, or the focal length in the focus information set with the largest difference from the current focal length of the lens.

In some embodiments, the computer program is processed to: obtain the rotation amount of the zoom ring during or after obtaining the first operation; determine the auxiliary focal length based on the current focal length of the lens and the rotation amount of the zoom ring; The auxiliary focal length determines the target focal length from the focus information set, wherein the target focal length is the auxiliary focal length as the initial focal length, and is matched in the focus information set according to the rotation direction information of the zoom ring, and the determined one is closest to the initial focal length focal length.

In some embodiments, the display screen is also used to display an input component corresponding to the focus information set; in response to the first operation, controlling the lens of the camera to automatically focus based on the focus information set includes: responding to the user inputting the input component The operation is to determine the focal length of the target object corresponding to the target object from the in-focus information set.

In some embodiments, the display screen is also used to display an in-focus intensity curve corresponding to the in-focus information set.

In some embodiments, the in-focus intensity curve is generated by determining the in-focus information to be displayed, the in-focus information to be displayed is the focal length information in the in-focus information set that satisfies the display conditions, and the display conditions include the in-focus information to be displayed The corresponding in-focus intensity is greater than or equal to a preset intensity threshold; an in-focus intensity curve is generated based on the in-focus information to be displayed.

In some embodiments, obtaining the first operation includes at least one of the following: if the focus information set includes at least one focus information, in response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, obtaining an automatic zoom operation ; or if at least one focus information is included in the focus information set, the automatic zoom operation is obtained in response to the rotation duration of the zoom ring being greater than or equal to the preset duration threshold; or if at least one focus information is included in the focus information set, In response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, and the rotation duration of the zoom ring being greater than or equal to a preset duration threshold, an automatic zoom operation is obtained.

In some embodiments, there is a first corresponding relationship between at least one focus information in the focus information set and the sub-regions in the focus frame; the first operation includes a sub-region selection operation; in response to the first operation, control the shooting The automatic focusing of the lens of the device based on the focus information set includes: determining the target focal length corresponding to the selected sub-region from the focus information set based on the first correspondence; and in response to the target focal length, controlling the lens of the shooting device to automatically focus.

In some embodiments, obtaining the first operation includes: determining the selected sub-region in response to the selection operation on the sub-region in the focus frame; and determining from the in-focus information set based on the first correspondence target focal length.

In some embodiments, acquiring the focus information set for at least one object in the focus frame of the current image includes: obtaining the focus information set for at least one object in the focus frame of the current image in response to the second operation indicating to enter the manual assisted focus mode. A focus information set; or in response to a third operation representing a zoom trend, acquiring a focus information set for at least one object in the focus frame of the current image.

In some embodiments, in response to the second operation indicating to enter the manual assisted focus mode, acquiring the focus information set for at least one object in the focus frame of the current image includes: in response to the second operation, acquiring the focus information set of the current image based on the sensor A collection of in-focus information for at least one object in the focus frame; and/or in response to the second operation, by adjusting the focal length of the lens, based on the pixel contrast of the at least one object in the focus frame of the current image, acquire the focus frame of the current image A collection of in-focus information for at least one object.

In some embodiments, by adjusting the focal length of the lens, based on the pixel contrast of the at least one object in the focus frame of the current image, obtaining the focus information set for at least one object in the focus frame of the current image includes: In the process of adjusting the focal length from the first preset focal length to the second preset focal length, the image information of multiple shooting surfaces is obtained; the in-focus shooting surface is determined from the multiple shooting surfaces, and there are adjacent images in the image information of the in-focus shooting surface. A pixel whose pixel contrast exceeds a preset contrast threshold; and at least part of the in-focus focal length corresponding to the in-focus shooting surface and the corresponding in-focus intensity are used as in-focus information.

In some embodiments, adjusting the focal length of the lens from the first preset focal length to the second preset focal length includes: adjusting the focal length of the lens from the minimum focal length of the lens to the maximum focal length of the lens.

In some embodiments, the lens includes a zoom ring; in response to the third operation representing the zoom trend, acquiring the focus information set for at least one object in the focus frame of the current image includes: acquiring the third operation, the third operation causes zooming The ring rotates; in response to the third operation, during the process of adjusting the focal length of the lens from the current focal length to the maximum focal length or the minimum focal length, image information of multiple shooting surfaces is obtained; determining the in-focus shooting surface from the multiple shooting surfaces, and achieving focus In the image information of the shooting surface, there are pixels whose adjacent pixel contrast exceeds a preset contrast threshold; and at least part of the in-focus focal length and corresponding in-focus intensity are used as in-focus information.

In some embodiments, obtaining the second operation includes at least one of the following: receiving an operation on a preset key, receiving an operation on a preset interactive component, or receiving a preset operation gesture.

In some embodiments, obtaining the second operation includes: generating the second operation when the preset key is pressed with a first specified displacement, wherein the preset button has at least two specified displacements, and the first specified displacement is at least two specified displacements. One of the displacements.

In some embodiments, the conditions for exiting the manual assisted focusing mode include at least one of the following: the preset button is in a state corresponding to exiting the manual assisted focusing mode; the target object moves out of the focus frame, wherein there are multiple There is a second corresponding relationship between each object and the focus information in the focus information set; the shooting device can only determine the focus information of the focus frame by adjusting the lens, and the target object is displaced in the opposite direction or relative to the lens. Displacement in the back direction, wherein the focus frame includes a plurality of objects, and there is a second corresponding relationship between each object and the focus information in the focus information set.

In some embodiments, compensating the focus information set based on the change amount of the focus information in the focus information set includes: determining the pose change information of the lens; compensate.

In some embodiments, there are multiple objects in the focus frame of the current image, and there is a second corresponding relationship between each object and the focus information in the focus information set; based on the change amount of the focus information in the focus information set Compensating the focus information set includes: determining the pose change information of at least one object; and based on the pose change information of the at least one object and the second corresponding relationship, focusing information to compensate.

In some embodiments, determining the pose change information of the at least one object includes: determining at least one object included in the focus frame through image recognition; and determining the pose change information of the at least one object included in the focus frame through image processing.

In some embodiments, the focus frame includes multiple objects, and there is a second corresponding relationship between each object and the focus information in the focus information set. Correspondingly, compensating the focus information set based on the change amount of the focus information in the focus information set includes: determining the pose change information of the lens and the pose change information of multiple objects included in the focus frame; The pose change information and the pose change information of multiple objects included in the focus frame are used to compensate the focus information set.

FIG. 21 is a schematic structural diagram of a photographing device provided by another embodiment of the present application.

As shown in FIG. 21 , the photographing device, including a lens and a camera body, is used to capture an object and obtain an image after the motor drives the lens to zoom or focus. The camera body may be a camera body of a handheld SLR or mirrorless camera. A lens is a lens that includes a zoom ring and a follow focus ring, such as a manual lens. The lens in this embodiment can also support manual focus and follow focus.

The photographing device may include a zoom ring coupled to the optical assembly, the optical assembly may include a plurality of lenses, and the zoom ring may be used to adjust the focus of the optical assembly. In some embodiments, the zoom ring may include a mechanical adjustment device. When the user turns the zoom ring, the focus of the optical assembly will change as the zoom ring is turned. In some other embodiments, the zoom ring can also include one or more adjustment buttons or adjustment wheels, which can be set on the optical assembly or the body, and when used to press the adjustment button or turn the adjustment wheel, the shooting The device will generate electrical signals for instructing how to adjust the focus of the optical assembly, eg how to adjust the rotational direction and/or rotational distance of one or more optical elements in the optical assembly. The electrical signal can instruct a motor mounted on the optical assembly or the body to rotate to drive the optical assembly to focus. The adjustment button or adjustment wheel may be a mechanical adjustment button or adjustment wheel, or may be a virtual adjustment button or adjustment wheel displayed on a display screen.

In some embodiments, the photographing device may further include a control device, and the control device may be disposed on the body, or the control device may be a part of the body. In some embodiments, the control device can be independent from the fuselage, or can be coupled with the fuselage. The control device can execute the control methods of the subsequent embodiments. The control device includes a memory and a processor. The memory is used to store program codes. The processor can call the program codes. When the program codes are executed, the processor is used to execute the control methods of the subsequent embodiments.

The memory includes non-random computer-readable storage media such as random access memory (RAM), read-only memory, flash memory, hard disk memory, or optical media, among others. Processor 1084 and/or processor 1164 include any suitable hardware processor, such as a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

The photographing device may also include sensors such as a gyroscope for determining the distance between the camera and the subject. For example, an inertial measurement unit (IMU) is provided in the camera, and the IMU includes an accelerometer, so that the displacement of the camera can be calculated according to the acceleration output by the accelerometer.

Another aspect of the present application also provides a shooting system.

As shown in FIG. 22 , the photographing system may include a photographing device and a supporting structure, and the supporting structure is used to carry the photographing device. The cloud platform can be used to carry the photographing device. For example, the photographing device can be fixed on the platform in a detachable manner through structural fixation (such as being fixed by a photographing device fixing mechanism).

An IMU can be set on the gimbal. For example, the IMU includes an accelerometer, so that the acceleration information collected based on the accelerometer is integrated to obtain velocity information, and the velocity information is integrated to obtain displacement information. The measurement information collected by the IMU may include the amount of position change of the photographing device corresponding to different sampling moments. If the distance difference between the IMU and the photographing device is a fixed value, the distance change between the IMU and the photographing device may be used as the distance change between the subject and the photographing device. For example, the reference plane of the IMU is located below the camera. The calculation processing of the position change amount may be processed by a controller in the IMU or a controller in the motor, which is not limited in this embodiment.

Optionally, the shooting device fixing mechanism of the pan/tilt can fixedly connect the motor to the bottom of the camera body, and the lens is mounted on the camera body through a bayonet.

A focus wheel and/or a zoom wheel can be set on the pan/tilt, and the focus wheel and/or zoom wheel can push the current rotation position and speed data of the focus wheel or the zoom wheel to the shooting device, so as to achieve zooming or focus. In addition, a motor for determining the zoom ring or focus ring can be provided on the gimbal, and the focus wheel and/or zoom wheel can send data such as the current rotational position and rotational speed of the focus wheel or zoom wheel to the motor, so that The motor turns to the specified position corresponding to the focus wheel or the zoom wheel.

A display screen can be set on the pan/tilt, and during the focus or zoom process using the control method provided by the embodiment of the present application, the user can also switch the automatic mode to the manual mode by inputting a control command on the display screen to select a mode, Or switch from manual mode to manual assisted focus mode, and use the focus wheel or zoom wheel to control the position of the motor to achieve focus or zoom.

Those skilled in the art should understand that the foregoing naming of the components of the camera system is for identification purposes only, and should not be construed as limiting the embodiments of the present application.

Wherein, the shooting control device includes: at least one processor and memory.

The memory stores computer-executable instructions.

At least one processor executes the computer-executable instructions stored in the memory, so that at least some of the steps shown above are implemented when the computer-executable instructions are executed.

It can be understood that, in some embodiments, the above steps can also be completed by the cooperation of the pan/tilt and the shooting device, for example:

In one embodiment, the pan/tilt determines the position change amount of the camera carried by the pan/tilt in a specific direction through the measurement information of the accelerometer in the inertial measurement unit; the pan/tilt transmits the position change amount to the camera, and the camera determines the position change amount according to the Update the in-focus information corresponding to the target object, and the lens performs autofocus according to the in-focus focal length.

The communication module of the pan/tilt can be used to transmit and/or receive data from one or more remote devices (such as mobile platforms, base stations, etc.). For example, the communication module can transmit control signals (such as motion signals, target object information, and tracking control instructions) to peripheral systems or devices, such as the above-mentioned centering platform and/or load. The communication module may include a transmitter and a receiver for receiving data from the remote device and transmitting data to the remote device, respectively. In some embodiments, the communication module may include a transceiver, which combines the functions of a transmitter and a receiver. In some embodiments, the transmitter and receiver can communicate with each other and with the processing unit. Communication may utilize any suitable means of communication, such as wired communication or wireless communication.

The gimbal shown in Figure 22 is a handheld gimbal. In addition, the gimbal can be set on a mobile platform.

The images captured by the mobile platform during its movement can be transmitted back from the mobile platform or imaging device to the control terminal or other suitable devices for display, playback, storage, editing or other purposes. Such transmission may occur in real-time or near real-time as the imaging device captures the imagery. Optionally, there may be a delay between image capture and transmission. In some embodiments, the imagery may be stored in the memory of the mobile platform without being transmitted anywhere else. Users can view these images in real time and, if desired, adjust target object information or adjust other aspects of the mobile platform or its components. The adjusted target object information may be provided to the mobile platform, and the iterative process may continue until a desired image is obtained. In some embodiments, images may be transmitted from the camera and/or the control terminal to the remote server. For example, images can be shared on some social networking platforms, such as WeChat Moments or Weibo.

Hereinafter, taking a hand-held camera as an example, the performing subject of each of the above operations will be exemplarily described. For example, each of the above operations can be performed by a hand-held camera. Specifically, the corresponding functions can be realized by an input unit of the camera, a processor installed in the camera, and a driver installed in the lens.

The following takes the hand-held pan/tilt as an example to illustrate the execution subject of the above operations. For example, operations related to driving the lens to move can be performed by the lens of the camera. The rest of the operations can be performed by a camera or a handheld pan/tilt. Specifically, the corresponding functions can be realized by the input unit of the handheld pan/tilt, the processor of the camera, and the like.

For example, the operation of acquiring the focus information set for at least one object in the focus frame of the current image may be performed by the camera. The obtaining first operation may be performed by a camera or a gimbal. Controlling the automatic focusing of the lens of the shooting device based on the focus information set can be performed by the camera.

Hereinafter, taking the photographing device mounted on the mobile platform as an example, the execution subject of each of the above operations will be exemplarily described.

For example, the operation of acquiring the focus information set for at least one object in the focus frame of the current image may be performed by the camera. The obtaining the first operation may be performed by a camera or a remote control of the mobile platform. Controlling the lens of the shooting device to automatically focus based on the focus information set can be performed by the camera.

It should be noted that the execution subject of the above operations is only an example and should not be construed as a limitation to the present application. It can be completed independently by one of the mobile platform, the control terminal, and the shooting device, or several of them cooperate. For example, in the case where the mobile platform is a land robot, a human-computer interaction module (such as a display for displaying the human-computer interaction interface, etc.) can be set on the land robot, and the user can directly obtain user information on the interactive interface displayed on the mobile platform. operations to generate user instructions, identify images of target objects, etc. Wherein, independent completion includes actively or passively, directly or indirectly acquiring corresponding data from other devices to perform corresponding operations.

The embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program includes program code for executing the method provided in the embodiment of the present application. When the computer program product is run on the electronic device, the The program code is used to enable the electronic device to implement the control method provided by the embodiment of the present application.

When the computer program is executed by the processor, the above-mentioned functions defined in the system/device of the embodiment of the present application are performed. According to the embodiments of the present application, the above-described systems, devices, modules, units, etc. may be implemented by computer program modules.

In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program can also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part, and/or installed from a removable medium. The program code contained in the computer program can be transmitted by any appropriate network medium, including but not limited to: wireless, wired, etc., or any appropriate combination of the above.

According to the embodiments of the present application, the program codes for executing the computer programs provided by the embodiments of the present application can be written in any combination of one or more programming languages, specifically, high-level process and/or object-oriented programming language, and/or assembly/machine language to implement these computing programs. Programming languages include, but are not limited to, programming languages such as Java, C++, python, "C" or similar programming languages. The program code can execute entirely on the user computing device, partly on the user device, partly on the remote computing device, or entirely on the remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider). business to connect via the Internet).

The above is the best embodiment of the present application. It should be noted that the best embodiment is only used for understanding the present application, and is not used to limit the protection scope of the present application. Moreover, the features in the optimal embodiment are applicable to the method embodiment and the device embodiment at the same time unless otherwise specified, and the technical features appearing in the same or different embodiments can be used without conflicting with each other. Use in combination.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application. .

Claims

A control method applied to a shooting device, comprising:

Obtain a collection of in-focus information for at least one object in the focus frame of the current image;

Obtaining a first operation, wherein the first operation includes a zoom operation or the first operation is directed to in-focus information in the in-focus information set; and

In response to the first operation, control the lens of the photographing device to automatically focus based on the focus information set.
The method of claim 1, wherein the set of focus information includes parameters and focus strength associated with the lens.
The method according to claim 2, wherein the parameters associated with the lens include a focal length and a focus distance of the lens, wherein the focus distance is a distance between a subject and the lens.
The method of claim 1, wherein the lens includes a zoom ring;

The first operation includes turning the zoom ring in a particular direction;

In response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set includes: repeating the following operations until the focal length of the lens is changed to a focus focal length corresponding to the target object: Response In the first operation, controlling the focal length of the lens to change to the target focal length in the focus information set, the target focal length is based on the current focal length of the lens as the initial focal length, based on the rotation direction of the zoom ring The information is matched with the determined focal length in the in-focus information set.
The method according to claim 4, wherein the target focal length includes any one of the following: the focal length in the focus information set with the smallest difference from the current focal length of the lens, the focal length in the focus information set with The focal length with the largest difference between the current focal lengths of the lens.
The method according to claim 4, wherein the method further comprises: obtaining the rotation amount of the zoom ring during or after the obtaining of the first operation;

determining an auxiliary focal length based on the current focal length of the lens and the rotation amount of the zoom ring;

Determine the target focal length from the focus information set based on the auxiliary focal length, wherein the target focal length is based on the auxiliary focal length as the starting focal length, according to the rotation direction information of the zoom ring in the focus information set Matching is carried out, and the determined focal length is the closest to the initial focal length.
The method according to claim 1, wherein the photographing device includes a display screen for displaying at least one focus information in the focus information set, and the focus information includes a focus intensity peak value.
The method according to claim 7, wherein the display screen is further used to display an input component corresponding to the focus information set;

In response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set includes:

In response to a user operation on the input component, a focal length of a target object corresponding to the target object is determined from the set of in-focus information.
The method according to claim 7, wherein the display screen is further used to display an in-focus intensity curve corresponding to the in-focus information set.
The method according to claim 9, wherein the in-focus intensity curve is generated in the following manner:

Determine the in-focus information to be displayed, the in-focus information to be displayed is the focal length information in the in-focus information set that meets the display conditions, and the display conditions include that the in-focus intensity corresponding to the in-focus information to be displayed is greater than or equal to preset intensity threshold;

The in-focus intensity curve is generated based on the in-focus information to be displayed.
The method of claim 9, further comprising:

displaying a current focus indicator corresponding to a point in the in-focus intensity curve; and

In-focus prompt information is displayed, and the in-focus prompt information includes display state change information of the current focus distance mark.
The method according to claim 4, wherein said obtaining the first operation comprises at least one of the following:

If the focus information set includes at least one focus information, in response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, an automatic zoom operation is obtained; or

If the focus information set includes at least one focus information, in response to the rotation duration of the zoom ring being greater than or equal to a preset duration threshold, an automatic zoom operation is obtained; or

If the focus information set includes at least one focus information, in response to the rotation speed of the zoom ring being greater than or equal to a preset rotation speed threshold, and the rotation duration of the zoom ring being greater than or equal to a preset duration threshold, Get automatic zoom operation.
The method according to claim 1, wherein there is a first corresponding relationship between at least one focus information in the focus information set and the sub-regions in the focus frame; the first operation includes sub-region selection operate;

In response to the first operation, controlling the lens of the photographing device to automatically focus based on the focus information set includes:

determining a target focal length corresponding to the selected sub-region from the in-focus information set based on the first correspondence; and

In response to the target focal length, controlling the lens of the photographing device to automatically focus based on the target focal length.
The method according to claim 13, further comprising: before the acquiring the first operation, displaying the in-focus strength corresponding to at least some sub-regions in the focus frame.
The method of claim 13, wherein said obtaining the first operation comprises:

In response to a selection operation on a sub-region in the focus frame, determine the selected sub-region; and

A target focal length corresponding to the selected sub-region is determined from the focus information set based on the first correspondence.
The method according to claim 1, wherein said obtaining the focus information set for at least one object in the focus frame of the current image comprises:

In response to the second operation indicating to enter the manual assisted focus mode, acquire a focus information set for at least one object in the focus frame of the current image; or

In response to the third operation representing the zoom trend, a set of focus information for at least one object in the focus frame of the current image is acquired.
The method according to claim 16, wherein, in response to the second operation indicating to enter the manual auxiliary focus mode, acquiring the focus information set for at least one object in the focus frame of the current image comprises:

In response to the second operation, acquiring a set of focus information for at least one object in the focus frame of the current image based on the sensor; and/or

In response to the second operation, by adjusting the focal length of the lens, based on the pixel contrast of the at least one object in the focus frame of the current image, a set of focus information for at least one object in the focus frame of the current image is acquired.
The method according to claim 17, wherein the sensor-based acquisition of the focus information set for at least one object in the focus frame of the current image comprises:

Acquiring in-focus focal lengths for at least one object in the focus frame of the current image through the sensor according to a preset sampling period, wherein the image captured based on the in-focus focal length satisfies preset imaging conditions; and

At least part of the in-focus focal lengths and in-focus intensity information corresponding to the at least part of the in-focus focal lengths are used as the in-focus information.
The method according to claim 17, wherein, by adjusting the focal length of the lens, based on the pixel contrast of at least one object in the focus frame of the current image, the focus of at least one object in the focus frame of the current image is acquired Information collections include:

In the process of adjusting the focal length of the lens from the first preset focal length to the second preset focal length, image information of multiple shooting surfaces is obtained;

Determining an in-focus shooting surface from the plurality of shooting surfaces, where there are pixels whose adjacent pixel contrast exceeds a preset contrast threshold in the image information of the in-focus shooting surface; and

The in-focus focal length and the corresponding in-focus intensity at least partly corresponding to the in-focus shooting surface are used as the in-focus information.
The method according to claim 19, wherein said adjusting the focal length of the lens from a first preset focal length to a second preset focal length comprises: adjusting the focal length of the lens from the minimum focal length of the lens to the minimum focal length of the lens maximum focal length.
The method of claim 16, wherein the lens includes a zoom ring;

In response to the third operation of characterizing the zoom trend, acquiring the focus information set for at least one object in the focus frame of the current image includes:

obtaining a third operation that causes the zoom ring to rotate;

In response to the third operation, during the process of adjusting the focal length of the lens from the current focal length to the maximum focal length or the minimum focal length, image information of multiple shooting surfaces is obtained;

Determining an in-focus shooting surface from the plurality of shooting surfaces, where there are pixels whose adjacent pixel contrast exceeds a preset contrast threshold in the image information of the in-focus shooting surface; and

At least part of the focus distance and the corresponding focus strength are used as the focus information.
The method according to claim 16, wherein obtaining the second operation comprises at least one of the following: receiving an operation on a preset key, receiving an operation on a preset interactive component, or receiving a preset operation gesture.
The method according to claim 16, wherein obtaining the second operation comprises: generating the second operation when a preset key is pressed with a first specified displacement, wherein the preset key has at least two specified displacements , the first specified displacement is one of the at least two specified displacements.
The method according to claim 23, further comprising: after the preset key is pressed for a first specified displacement,

Obtaining a fourth operation, the fourth operation is a second specified displacement of the preset button being pressed, wherein the first specified displacement and the second specified displacement are respectively the two specified displacements of the at least two specified displacements. one; and

In response to the fourth operation, image information is acquired.
The method of claim 23, further comprising: after obtaining the second operation,

If it is determined that the condition for exiting the manual assisted focusing mode is met, the manual assisted focusing mode is exited.
The method according to claim 25, wherein the conditions for exiting the manual assisted focusing mode include at least one of the following:

The preset button is in a state corresponding to exiting the manual auxiliary focus mode;

The target object moves out of the focus frame, wherein there are multiple objects in the focus frame, and there is a second corresponding relationship between each object and the focus information in the focus information set;

The shooting device can only determine the in-focus information of the focus frame by adjusting the lens, and the target object is displaced in the opposite direction or in the opposite direction relative to the lens, wherein the focus frame includes a plurality of objects, and each object is related to the There is a second corresponding relationship between focus information in the focus information set.
The method according to claim 1, further comprising: after acquiring the focus information set for at least one object in the focus frame of the current image,

The in-focus information set is compensated based on a change amount of the in-focus information in the in-focus information set.
The method according to claim 27, wherein the compensating the focus information set based on the change amount of the focus information in the focus information set comprises:

determining the pose change information of the lens; and

The focus information set is compensated based on the pose change information of the lens.
The method according to claim 28, wherein said determining the pose change information of the lens comprises:

The pose change information of the lens is determined based on an inertial measurement unit, and the pose change information includes at least one of displacement information and angle change information.
The method according to claim 27, wherein there are multiple objects in the focus frame of the current image, and there is a second corresponding relationship between each object and the focus information in the focus information set;

The compensating the in-focus information set based on the change amount of the in-focus information in the in-focus information set includes:

determining pose change information of at least one of the objects; and

Based on the pose change information of at least one of the objects and the second corresponding relationship, the focus information corresponding to each of the at least one of the objects in the focus information set is compensated.
The method according to claim 30, wherein said determining at least one pose change information of said object comprises:

determining at least one object included in the focus frame through image recognition; and

The pose change information of at least one object included in the focus frame is determined by image processing.
The method according to claim 31, wherein the determining the pose change information of at least one object included in the focus frame through image processing comprises:

calculating displacement information of at least one sub-image of said object in said current image; and

Calculating pose change information for the at least one object in the focus frame of the current image based on the in-focus information of the at least one object and the displacement information of the at least one object in the current image.
The method according to claim 27, wherein the focus frame includes a plurality of objects, and there is a second corresponding relationship between each object and the in-focus information in the in-focus information set;

The compensating the in-focus information set based on the change amount of the in-focus information in the in-focus information set includes:

determining pose change information of the lens and pose change information of a plurality of objects included in the focusing frame; and

The focus information set is compensated based on the pose change information of the lens and the pose change information of a plurality of objects included in the focus frame.
According to the method according to any one of claims 1 to 33, after acquiring the focus information set for at least one object in the focus frame of the current image, further comprising:

The focus information of the focus frame is updated by the sensor.
The method according to any one of claims 1 to 33, wherein the in-focus information is displayed in the form of an in-focus intensity curve, and the two-dimensional coordinate system where the in-focus intensity curve is located includes the first coordinate representing the focal length axis and a second axis representing the in-focus intensity.
A photographing device, comprising:

A lens for collecting images;

one or more processors;

A storage device for storing one or more computer programs which, when executed by the processor, implement the method according to any one of claims 1-35.
A camera system, comprising:

The photographing device as claimed in claim 36; and

A support mechanism supports the photographing device.
A computer-readable storage medium for storing one or more computer programs which, when executed by the processor, implement the method according to any one of claims 1-35.