WO2021168804A1

WO2021168804A1 - Image processing method, image processing apparatus and image processing system

Info

Publication number: WO2021168804A1
Application number: PCT/CN2020/077217
Authority: WO
Inventors: 邹文; 彭亮; 丁硕; 夏斌强
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-02-28
Filing date: 2020-02-28
Publication date: 2021-09-02
Also published as: CN112514366A

Abstract

Provided is an image processing method, comprising: acquiring a first maximum viewing angle area corresponding to a first focal length at the current attitude angle, and obtaining a second maximum viewing angle area corresponding to a second focal length; comparing the first maximum viewing angle area with the second maximum viewing angle area; and determining the selectable range of the area to be processed in a first image according to a comparison result, so as to obtain at least one second image of the area to be processed at the second focal length, wherein the first image is acquired at the first focal length, and the second focal length is greater than the first focal length. According to the present disclosure, the photographable range of a combined lens with multiple focal lengths at multiple focal lengths can be determined, such that a user can understand the photographable range at a specified focal length, thereby improving the user experience.

Description

Image processing method, image processing device and image processing system

Technical field

The present disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing device, and an image processing system.

Background technique

During surveys or inspections, such needs are often encountered. For example, it is necessary to photograph a 100-meter-high iron tower, and it is necessary to check every detail of the detailed parts of the iron tower to see if any screws are rusted, or the steel is out. An abnormal situation.

However, due to the different shooting ranges of combined lenses with multiple focal lengths at different focal lengths, it is difficult for users to determine the shooting ranges of combined lenses with multiple focal lengths under multiple focal lengths through software applications, which makes it difficult for users to use software. The application controls the combined lens to shoot the required image at the specified focal length, causing a poor user experience.

Public content

In view of this, the embodiments of the present disclosure provide an image processing method, an image processing device, and an image processing system, which can determine the shooting range of a combined lens with multiple focal lengths at multiple focal lengths through specific application software, so as to facilitate users Understand the range of shooting under the specified focal length to improve user experience.

In a first aspect, an embodiment of the present disclosure provides an image processing method, the method includes: obtaining a first maximum viewing angle area corresponding to a first focal length at a current attitude angle, and obtaining a second maximum viewing angle area corresponding to a second focal length ; Compare the first maximum viewing angle area with the second maximum viewing angle area; and determine the selectable range of the area to be processed in the first image according to the comparison result, so as to obtain at least one second of the area to be processed under the second focal length Image, the first image is acquired at the first focal length; wherein the second focal length is greater than the first focal length.

In application, since the angle of view area corresponding to the first focal length is different from the angle of view area corresponding to the second focal length, for example, when the posture of the camera is adjusted by the posture adjustment device, the maximum posture adjustment range of the camera is affected by the mechanical of the posture adjustment device. Due to the influence of the structure and the like, the second maximum viewing angle area corresponding to the second focal length may not completely include the first maximum viewing angle area corresponding to the first focal length under the current attitude angle. Also, because the user is not clear about which areas in the first maximum viewing angle area overlap with the second maximum viewing angle area, that is, the user is not clear about which areas in the first image can be photographed with the second focal length to obtain a clearer partial image. Therefore, when the area to be processed selected by the user exceeds the above-mentioned overlapping area, it is impossible to take pictures at the second focal length, resulting in dissatisfaction of the user. In addition, when the area to be processed selected by the user exceeds the overlapped area, the posture adjustment device is likely to be damaged. However, the image processing method provided by the embodiments of the present disclosure can compare the first maximum angle of view area corresponding to the first focal length and the second maximum angle of view area corresponding to the second focal length under the current attitude angle to determine whether the area to be processed is in The selectable range in the first image corresponding to the first maximum viewing angle area, so as to obtain at least one second image of the area to be processed at the second focal length. In this way, determining the selectable range of the area to be processed in the first image not only allows the user to clarify which areas in the first image can be photographed with the second focal length to obtain a clearer partial image, but also can be limited by the selectable range Measures such as user operation and adjustment range of the attitude adjustment device reduce the possibility of damage to the attitude adjustment device.

In a second aspect, embodiments of the present disclosure provide an image processing device, which includes: one or more processors; and a computer-readable storage medium. The computer-readable storage medium is used to store one or more computer programs. When the computer program is executed by the processor, it realizes: obtain the first maximum viewing angle area corresponding to the first focal length under the current attitude angle, and obtain the second focal length The corresponding second largest viewing angle area; comparing the first largest viewing angle area with the second largest viewing angle area; and determining the selectable range of the area to be processed in the first image according to the comparison result, so as to obtain the area to be processed in the second At least one second image at the focal length, the first image is acquired at the first focal length; wherein the second focal length is greater than the first focal length.

The image processing device provided by the embodiment of the present disclosure can automatically determine the selectable range of the area to be processed in the first image, thereby facilitating the user to determine the area in the first image that can be photographed with the second focal length, so as to obtain a clearer part The image helps to improve the user experience and helps reduce the possibility of damage to the posture adjustment device.

In a third aspect, an embodiment of the present disclosure provides an image processing system. The image processing system includes a photographing device and a control terminal communicatively connected to the photographing device. The control device is used to obtain the area to be processed, and the photographing device is used to obtain the photographing device. The first maximum angle of view area corresponding to the first focal length under the current attitude angle, and the second maximum angle of view area corresponding to the second focal length is obtained; the first maximum angle of view area and the second maximum angle of view area are compared; and according to the comparison result Determine the selectable range of the area to be processed in the first image, so as to control the photographing device to acquire at least one second image of the area to be processed at the second focal length. The first image is acquired at the first focal length, where the second The focal length is greater than the first focal length.

In the image processing system provided by the embodiments of the present disclosure, the photographing device can automatically determine the selectable range of the area to be processed in the first image, thereby facilitating the user to determine the photographable area in the first image that can be photographed with the second focal length, so that The user controls the photographing device to obtain a clearer partial image of the photographable area at the second focal length, which helps to improve the user experience and helps reduce the possibility of damage to the posture adjustment device.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium that stores executable instructions that, when executed by one or more processors, can cause one or more processors to execute the above Methods.

The advantages of the additional aspects of the present disclosure will be partially given in the following description, and some will become obvious from the following description, or be understood through the practice of the present disclosure.

Description of the drawings

Through the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the embodiments of the present disclosure will become easier to understand. In the drawings, a number of embodiments of the present disclosure will be described in an exemplary and non-limiting manner, in which:

FIG. 1 is an application scenario of an image processing method, image processing device, and image processing system provided by embodiments of the disclosure;

2 is an application scenario of an image processing method, image processing device, and image processing system provided by another embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the disclosure;

4 is a schematic diagram of a vertical range of a second maximum viewing angle area corresponding to a second focal length provided by an embodiment of the present disclosure;

5 to 10 are schematic diagrams of the display interface of the control terminal provided by the embodiments of the disclosure;

FIG. 11 is a schematic diagram of the division of to-be-processed areas provided by an embodiment of the present disclosure;

FIG. 12 is a schematic diagram of an area to be photographed according to an embodiment of the disclosure;

FIG. 13 is a schematic diagram when the number of shots provided by an embodiment of the disclosure is an odd number;

14 is a schematic diagram when the number of shots provided by an embodiment of the disclosure is an even number;

15-17 are schematic diagrams of display images provided by embodiments of the disclosure;

FIG. 18 is a schematic structural diagram of an image processing device provided by an embodiment of the disclosure;

FIG. 19 is a schematic structural diagram of an image processing system provided by an embodiment of the disclosure;

FIG. 20 is a schematic structural diagram of a control terminal provided by an embodiment of the disclosure; and

FIG. 21 is a schematic diagram of the structure of a photographing device provided by an embodiment of the disclosure.

Detailed ways

The embodiments of the present disclosure are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are intended to explain the present disclosure, but should not be construed as limiting the present disclosure.

Fig. 1 is an application scenario of an image processing method, an image processing device, and an image processing system provided by embodiments of the disclosure. As shown in FIG. 1, the UAV 10 can rotate around up to three orthogonal axes, such as a first pitch axis (X1), a first heading axis (Y1), and a first roll axis (Z1). Rotating around three axes can refer to pitch rotation, heading rotation and roll rotation. The angular velocity of the UAV 10 around the first pitch axis, the first yaw axis, and the first roll axis can be expressed as ωX1, ωY1, and ωZ1. The unmanned aerial vehicle 10 can perform movement in translation along a first pitch axis, a first yaw axis, and a first roll axis. The linear velocity of the unmanned aerial vehicle 10 along the first pitch axis, the first yaw axis, and the first roll axis can be denoted as VX1, VY1, and VZ1, respectively. It should be noted that the unmanned aerial vehicle 10 is only an exemplary description of the application scenario, and cannot be understood as a limitation on the application scenario of the present disclosure. The present disclosure can also be applied to a variety of movable platforms. For example, the movable platform may also be a pan-tilt cart, a handheld pan-tilt, a robot, etc., these movable platforms can rotate around one axis or two axes, or cannot rotate around an axis.

As shown in FIG. 1, the imaging device 20 is mounted on the unmanned aerial vehicle 10 via a carrier. The carrier can make the imaging device 20 move around and/or along up to three orthogonal axes (in other embodiments, non-orthogonal axes) relative to the UAV 10, such as the second pitch axis (X2 ), the second yaw axis (Y2) and the second roll axis (Z2) move. The second pitch axis, the second yaw axis, and the second roll axis may be parallel to the first pitch axis, the first yaw axis, and the first roll axis, respectively. In some possible embodiments, the load is an imaging device including an optical module, and the second roll axis may be substantially parallel to the optical path and the optical axis of the optical module. The optical module can be connected with an image sensor to capture images. The carrier can send a control signal according to an actuator connected to the carrier, such as a motor, so that the carrier can rotate around up to three orthogonal axes, such as the second pitch axis, the second yaw axis, and the second roll axis. . Rotation around the three axes can be referred to as pitch rotation, heading rotation and roll rotation. The angular velocity of the camera 20 around the second pitch axis, the second yaw axis, and the second roll axis can be expressed as ωX2, ωY2, and ωZ2, respectively. The carrier can make the imaging device 20 move relative to the UAV 10 along the second pitch axis, the second yaw axis, and the second roll axis. The linear velocity of the camera 20 along the second pitch axis, the second yaw axis, and the second roll axis can be denoted as VX2, VY2, VZ2, respectively.

In some possible embodiments, the carrier may only allow the camera 20 to rotate relative to the UAV 10 about a subset of the three axes (the second pitch axis, the second yaw axis, and the second roll axis). For example, the carrier may only allow the camera 20 to rotate around the second pitch axis, the second yaw axis, and the second roll axis or a combination of any of them, but not allow the camera 20 to move in translation along any axis. For example, the carrier may allow the camera 20 to rotate around only one of the second pitch axis, the second yaw axis, and the second roll axis. The carrier may allow the camera 20 to rotate around only two of the second pitch axis, the second yaw axis, and the second roll axis. The carrier may allow the camera device 20 to rotate around the three axes of the second pitch axis, the second yaw axis, and the second roll axis.

In some possible embodiments, the carrier may only allow the camera 20 to move along the second pitch axis, the second yaw axis, and the second roll axis or a combined translational movement of any of them, but not allow the camera 20 to move around. Any axis rotation movement. For example, the carrier may allow the camera 20 to move along only one of the second pitch axis, the second yaw axis, and the second roll axis. The carrier may allow the camera 20 to move along only two of the second pitch axis, the second yaw axis, and the second roll axis. The carrier may allow the camera 20 to move along the three axes of the second pitch axis, the second yaw axis, and the second roll axis.

In some possible embodiments, the carrier may allow the camera 20 to perform rotational and translational motions relative to the UAV 10. For example, the carrier may allow the camera 20 to move along and/or around one, two, or three of the second pitch axis, the second yaw axis, and the second roll axis.

In some possible embodiments, the imaging device 20 may be directly mounted on the UAV 10 without a carrier, or the carrier does not allow the imaging device 20 to move relative to the UAV 10. In this case, the posture, position, and/or direction of the photographing device 20 are fixed relative to the unmanned aerial vehicle 10.

In some possible embodiments, the adjustment of the posture, direction and/or position of the photographing device 20 may be achieved individually or collectively by appropriate adjustments to the UAV 10, the carrier and/or the photographing device 20. For example, the camera 20 can be rotated 80 degrees around a designated axis (such as a heading axis) by the following method: the UAV 10 rotates 80 degrees alone, the load is rotated 80 degrees relative to the UAV 10 through actuation of the carrier, and no The human aircraft 10 rotates 50 degrees while the load rotates 30 degrees relative to the unmanned aircraft 10.

Similarly, the translational movement of other loads can be achieved by proper adjustment of the UAV 10 and the carrier. Further, the adjustment of the operating parameters of the load can also be completed by one or more of the UAV 10, the carrier, or the control terminal of the UAV 10. Among them, the operating parameters of the load may include, for example, the zoom degree or the focal length of the imaging device.

For example, the unmanned aerial vehicle 10 may include an unmanned aerial vehicle, and the photographing device 20 may include an imaging device.

In an embodiment, the control terminal 30 may also be included in the application scenario. Among them, the movable platform can be an aircraft (such as an unmanned aerial vehicle), a pan-tilt cart, a handheld pan-tilt, a robot, and so on. The control terminal 30 may be a device such as a mobile phone or a tablet computer, and may also have a remote control function to realize remote control of a movable platform such as the unmanned aerial vehicle 10. The unmanned aerial vehicle 10 may include an attitude adjustment device (such as the carrier described above) such as a pan/tilt, and the imaging device 20 may be mounted on the pan/tilt. There may be one photographing device 20. The photographing device 20 may include a first lens and a second lens for performing different photographing tasks. Of course, the photographing device 20 may also include two or more. The first lens and the second lens are located at Different camera devices. The first lens and the second lens may be lenses corresponding to the conventional shooting function of the camera. For example, the first lens and the second lens may be a wide-angle lens and a zoom lens, respectively. Among them, a wide-angle lens can be used to obtain a wider and complete picture, a zoom lens can be used to obtain high-definition details, a pan-tilt can be used to adjust the shooting angle of the camera 20, and the UAV 10 can be used to ensure smooth movement and no drift. The control terminal 30 can be used to control the movement of the unmanned aerial vehicle 10. The control terminal 30 can also obtain the shooting screen returned by the shooting device 20 for the user to view. In addition, the control terminal 30 can also obtain a user's control instruction on the photographing device 20, and send the control instruction to the photographing device. The control instruction may be, for example, an instruction to control the zoom factor of the zoom lens 30 of the photographing device 20, or may be an instruction to control the photographing range of the zoom lens of the photographing device 20, or the like.

The unmanned aerial vehicle 10 may include various types of UAV (Unmanned Aerial Vehicle, unmanned aerial vehicle), such as a quadrotor UAV, a hexarotor UAV, and the like. The gimbal included can be a three-axis gimbal, that is, the attitude of the gimbal can be controlled on the three axes of the pitch axis, roll axis, and yaw axis, so as to determine the corresponding attitude of the gimbal so that it can be mounted on The photographing device 20 on the pan/tilt can complete corresponding photographing tasks.

In the embodiment of the present disclosure, the unmanned aerial vehicle 10 may establish a communication connection with the above-mentioned control terminal 30 through a wireless connection method (for example, a wireless connection method based on WiFi or radio frequency communication, etc.). The control terminal 30 may be a controller with a joystick and a display screen, and the UAV 10 is controlled by the amount of the stick. The control terminal 30 can also be a smart device such as a smart phone or a tablet computer. It can control the unmanned aerial vehicle 10 to fly automatically by configuring the flight trajectory on the user interface UI, or control the unmanned aerial vehicle 10 to fly automatically by means of body sensing, or through After the flight trajectory is recorded during the flight of the UAV 10 in advance, the UAV 1 is controlled to automatically fly along the recorded flight trajectory.

In the embodiment of the present disclosure, the photographing device 20 may also establish a communication connection with the above-mentioned control terminal 30 through a wireless connection method (for example, a wireless connection method based on WiFi or radio frequency communication, etc.). For example, the photographing device 20 establishes a wireless communication connection with the aforementioned control terminal 30 through a wireless communication module of a movable platform. For another example, the photographing device 20 may establish a communication connection with the aforementioned control terminal 30 by itself, wherein, in some embodiments, the photographing device 20 is an independent photographing device, such as a sports camera. The control terminal 30 can install application software for controlling the shooting device 20, and the user can view the shooting screen returned by the shooting device on the display interface of the control terminal 30 through the application software, and provide the user with the interaction between the control terminal 30 and the shooting device 20 interface.

The imaging device 20 includes charge-coupled devices (CCD), and uses complementary metal-oxide-semiconductor (CMOS), or N-type metal-oxide-semiconductor (N-type metal-semiconductor). oxide-semiconductor, NMOS, Live MOS), or other types of sensors. Specifically, the sensor may be a part of the imaging equipment (such as a camera) carried by the unmanned aerial vehicle. The imaging equipment may be carried on the unmanned aerial vehicle through a pan/tilt, which can adjust the attitude angle of the photographing device 20.

The image data captured by the camera 20 can be stored in a data storage device. The data storage device may be based on semiconductor, magnetic, optical or any suitable technology, and the data storage device may include flash memory, USB drive, memory card, solid state drive, hard disk drive, floppy disk, optical disk, magnetic disk, etc. For example, data storage devices include removable storage devices that can be detachably connected to imaging devices, such as memory cards of any suitable format, such as personal computer cards, micro flash memory cards, SM cards, and Memory Stick Memory Stick, Memory Stick Duo Memory Stick, Memory Stick PRO Duo Memory Stick, Mini Memory Card, Multimedia Memory Card, Micro Multimedia Memory Card, MMCmicro Card, PS2 Card, SD Card, SxS Memory Card, UFS Card, Micro SD Card, MicroSD card, xD card, iStick card, serial flash memory module, NT card, XQD card, etc. The data storage device may also include an external hard disk drive, optical disk drive, magnetic disk drive, floppy disk drive, or any other suitable storage device connected to the imaging device.

The image captured by the camera 20 can be transmitted to the control terminal 30 through the wireless communication module. In some possible embodiments, the image data may be compressed or subjected to other processing before being transmitted by the wireless communication module. In other embodiments, the image data may not undergo compression or other processing before transmission. The transmitted image data can be displayed on the control terminal 30 so that the user who operates the user terminal can see the image data and/or interact with the control terminal 30 based on the image data.

The photographing device 20 may preprocess the captured image data. In the process of preprocessing the image data, hardware, software or a combination of the two can be used. The hardware may include Field-Programmable Gate Array (FPGA) and so on. Specifically, after the image data is captured, the unprocessed image data can be pre-processed to perform noise removal, contrast enhancement, scale space representation, and the like.

Fig. 2 is an application scenario of an image processing method, image processing device, and image processing system provided by another embodiment of the present disclosure.

As shown in FIG. 2, this scene may include a photographing device 20, a control terminal 30, and a posture adjustment device 40. Wherein, the posture adjusting device 40 may be a device for adjusting the posture of the photographing device 20, such as a pan/tilt, etc., to adjust the posture of the photographing device 20. The control terminal 30 may be an electronic device such as a mobile phone, a tablet computer, a notebook, a desktop computer, a remote control, etc., to realize remote control of the posture adjustment device 40 and the photographing device 20. For example, the imaging device 20 may be mounted on a posture adjustment device 40 which is a pan/tilt. The photographing device 20 may include a zoom lens to perform photographing tasks. The zoom lens may be a lens corresponding to the conventional shooting function of the camera. For example, a zoom lens can be used to obtain a more complete picture in the environment at a low focal length, and a high focal length of the zoom lens can be used to obtain high-definition details. The movement of the device 20 is smooth and without drift. The control terminal 30 can be used to control the movement of the posture adjustment device 40. The control terminal 30 can also obtain the shooting screen returned by the shooting device 20 for the user to view. In addition, the control terminal 30 can also obtain a user's control instruction on the photographing device 20, and send the control instruction to the photographing device. The control instruction may be, for example, an instruction to control the zoom factor of the zoom lens of the photographing device 20, or may be an instruction to control the photographing range of the zoom lens of the photographing device 20, or may be an instruction to view a partially clear photo, or the like.

Take the security scene as an example to introduce. The user may set a monitoring task on the control terminal 30, and the monitoring task may include the task of acquiring images in a specified time period and a specified area using multiple focal lengths. For example, at 9 o'clock in the morning, an image of the scene as shown in FIG. 2 (such as an image acquired at a low focal length), and a partial image of the area where the key monitoring object is located in the image (such as an image acquired at a high focal length) is acquired at 9 o'clock in the morning. In this way, it is convenient for the user to perform state monitoring and the like based on the image acquired by the photographing device 20. It can be understood that the design application is not limited to security scenes. The security scenes are shooting in a fixed range. In addition, they can also be shooting in a moving range, for example, shooting in different scenes through a movable platform.

In addition, users can use multiple focal lengths to obtain images at the current point in time and in a specific area at any time. For example, when the user finds that there may be an abnormality in the scene as shown in FIG. 2, he can control the photographing device 20 to photograph the area where the abnormality may exist in time for state tracking and the like.

In the embodiment of the present disclosure, the photographing device 20 may also establish a communication connection with the above-mentioned control terminal 30 through a wired connection or a wireless connection (for example, a wireless connection based on WiFi or radio frequency communication, etc.). The control terminal 30 can install application software for controlling the shooting device 20, and the user can view the shooting screen returned by the shooting device on the display interface of the control terminal 30 through the application software, and provide the user with the interaction between the control terminal 30 and the shooting device 20 interface. The photographing device 20 includes an image sensor to capture image information.

Next, an exemplary description of the image processing method will be given with reference to FIGS. 3-17.

FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the disclosure.

As shown in FIG. 3, the image processing method may include operations S301 to S305. It should be noted that the above operations can be performed on a variety of electronic devices with computing capabilities, for example, can be performed on at least one of an unmanned aerial vehicle, a camera, a pan/tilt, a pan/tilt camera, and a control terminal. Among them, the pan/tilt camera refers to the equipment including the pan/tilt and the camera, and the pan/tilt and the camera are connected by a non-quick release.

The following is an example of a system with a pan/tilt camera (shooting terminal) and a control terminal (control terminal). In one embodiment, operations S301 to S305 are performed on the pan-tilt camera. In another embodiment, operations S301 to S305 are performed on the control terminal. In another embodiment, at least part of operations S301 to S305 are performed on the pan-tilt camera, and the rest are performed on the control terminal. For the scenarios where parts of operations S301 to S305 are performed on different electronic devices, different electronic devices can communicate with each other the data required for the operation, user operations, intermediate operation results, and final operation results, etc. .

In the process of performing the above operations S301 to S305, the required data and user operations can be collected by a specific sensor, such as image data collected by an image sensor, and user operations collected by a touch screen.

In operation S301, a first maximum viewing angle area corresponding to the first focal length under the current attitude angle is obtained, and a second maximum viewing angle area corresponding to the second focal length is obtained.

In this embodiment, the second maximum viewing angle area corresponding to the second focal length is determined based on the maximum rotatable angle of the posture adjustment device (such as a pan-tilt). The first maximum viewing angle area corresponding to the first focal length under the current attitude angle is related to the pose of the camera. Take a camera installed on the drone as an example for illustrative description. The camera is installed on the drone through a pan-tilt, and the attitude of the camera is adjusted through the pan-tilt.

In one embodiment, since the deflection range of the pitch angle of the gimbal in the normal downward installation includes the first range, such as (-90, 30), the deflection range of the yaw angle includes the second range, such as (-300, 300). In order to avoid the “meter” arrangement of the picture when the gimbal controls the camera to shoot downwards, it is necessary to limit the adjustable range of the gimbal's yaw angle and pitch angle when taking pictures with the second focal length. For example, the maximum adjustable range can be determined by the maximum rotation angle of the pan/tilt.

4 is a schematic diagram of a vertical range of a second maximum viewing angle area corresponding to a second focal length provided by an embodiment of the disclosure. As shown in FIG. 4, if the deflection range of the pitch angle of the pan/tilt at the second focal length is set to include the first range being (MIN, MAX), it can be determined that the second maximum viewing angle area corresponding to the second focal length includes:

in,

Is the vertical size of the viewing area corresponding to the second focal length. Correspondingly, setting the deflection range of the pitch angle of the pan/tilt at the first focal length includes the second range (MIN, MAX) (because the camera device corresponding to the first focal length and the camera corresponding to the second focal length can be controlled through the same pan/tilt In the photographing device, the first range and the second range may be the same), it can be determined that the first maximum viewing angle area corresponding to the first focal length includes:

in,

Is the vertical size of the viewing area corresponding to the first focal length.

When the distance is at a certain current attitude angle (cur_pitch), the first maximum viewing angle area corresponding to the first focal length can be expressed as

The first maximum viewing angle area corresponding to the first focal length at the current attitude angle may be the first viewing angle area corresponding to the first focal length of the camera at the current attitude angle, and the camera may capture an image of the first viewing angle area at the first focal length .

It should be noted that the second focal length includes any of the following: fixed focal length (such as the focal length of a camera with a fixed focal length), preset focal length (such as the focal length specified in the preset shooting task), and the focal length determined based on the focal length algorithm ( For example, determined based on the historical focal length data used by the user), based on the focal length determined by the sensor and the focal length selected by the user (for example, the focal length selection component is displayed on the display interface, and the user clicks the button, slides the focal length selection bar, enters text, etc. Select focal length). Wherein, the focal length determined based on the sensor includes: the focal length determined based on distance information, and the distance information is determined by a laser rangefinder. For example, a laser rangefinder can be set on a camera, pan/tilt or drone to determine the applicable focal length.

In operation S303, the first maximum viewing angle area and the second maximum viewing angle area are compared. For example, comparing whether the first maximum viewing angle area includes the second maximum viewing angle area, or comparing whether the second maximum viewing angle area includes the first maximum viewing angle area. For another example, compare the upper limit of the first maximum angle of view area and the upper limit of the second maximum angle of view area, compare the lower limit of the first maximum angle of view area and the lower limit of the second maximum angle of view area, compare the left limit of the first maximum angle of view area and the second maximum For the left limit of the viewing angle area, compare the right limit of the first maximum viewing angle area and the right limit of the second maximum viewing angle area.

In operation S305, the selectable range of the area to be processed in the first image is determined according to the comparison result, so as to obtain at least one second image of the area to be processed under the second focal length, the first image being obtained under the first focal length . Wherein, the second focal length is greater than the first focal length.

In this embodiment, the selectable range of the area to be processed in the first image corresponding to the first maximum viewing angle area may be determined based on the comparison result, and the area in the first image corresponding to the selectable range may be based on the second focal length Take a shot.

In one embodiment, if the first maximum angle of view area corresponding to the first focal length is included in the second maximum angle of view area corresponding to the second focal length, it is determined that the first maximum angle of view area can be photographed based on the second focal length . That is, the upper and lower boundaries of the selectable range are the upper and lower boundaries of the first maximum viewing angle area, and the value of the selectable range is (0.0, 1.0) (for example, the upper boundary of the selectable range and the upper boundary of the first maximum viewing angle area are taken The value ratio is 0% to 100%, and the value ratio of the lower boundary of the selectable range to the lower boundary of the first maximum viewing angle area is 0% to 100%).

In another embodiment, if the upper boundary of the first maximum viewing angle area exceeds the upper boundary of the second maximum viewing angle area, the upper limit of the selectable range needs to be calculated.

In another embodiment, if the lower boundary of the first maximum viewing angle area exceeds the lower boundary of the second maximum viewing angle area, the lower limit of the selectable range needs to be calculated.

In another embodiment, if the left boundary of the first maximum viewing angle area exceeds the left boundary of the second maximum viewing angle area, the left limit of the selectable range needs to be calculated.

In another embodiment, if the right boundary of the first maximum viewing angle area exceeds the right boundary of the second maximum viewing angle area, the right limit of the selectable range needs to be calculated.

In another embodiment, in order for the user to intuitively see the relationship between the first maximum angle of view area and the second maximum angle of view area, and then select an appropriate area to be processed, when the method is applied to the control end of the shooting end, it can also The following operations are included to display at least one of the multiple boundaries of the selectable range. For example, one or more boundaries of the selectable range can be displayed in the display interface. For another example, when the boundary of the area to be processed exceeds the selectable range, the boundary of the exceeded selectable range may be displayed on the display interface. For another example, the boundary beyond the selectable range among the boundaries of the area to be processed may be displayed on the display interface. For another example, the non-selectable range and the selectable range can be displayed differently, such as by transparency or color.

Figures 5 to 10 are schematic diagrams of the display interface of the control terminal provided by the embodiments of the present disclosure.

As shown in FIG. 5, determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If it is determined according to the comparison result that the second maximum viewing angle area S2 includes the first maximum viewing angle area S1, then determining the area to be processed The selectable range of the area in the first image is any area in the first image.

For example, if

Then the upper boundary 1t_piont_y=0.0. if

Then the lower boundary rb_piont_y=1.0.

As shown in FIG. 6, determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If it is determined according to the comparison result that the second maximum viewing angle area S2 includes part of the first maximum viewing angle area S1, then determining The selectable range of the processing area in the first image is the overlap area of the first maximum viewing angle area and the second maximum viewing angle area. The shaded area in FIG. 6 is the overlapping area of the first maximum viewing angle area and the second maximum viewing angle area.

As shown in FIG. 7, the current attitude angle is the current pitch angle, and determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If the first upper limit of the first maximum viewing angle area S1 is determined according to the comparison result Beyond the second upper limit of the second maximum viewing angle area S2, based on the first upper limit, the second upper limit, and the vertical viewing angle area corresponding to the first focal length, the selectable range is determined to be the upper limit (upper boundary) of the first maximum viewing angle area. ).

For example, if

Upper boundary

As shown in Figure 8, the current attitude angle is the current pitch angle, and determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds The first lower limit of the first maximum viewing angle area, based on the second lower limit, the first lower limit, and the vertical viewing angle area corresponding to the first focal length, determine the selectable lower limit (lower boundary) of the selectable range in the first maximum viewing angle area .

For example, if

Lower boundary

As shown in Figure 9, the current attitude angle is the current yaw angle, and determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If the limit exceeds the second left limit of the second maximum angle of view area, based on the first left limit, the second left limit and the horizontal angle of view area corresponding to the first focal length, the selectable range is determined to be the selectable left limit of the first maximum angle of view area. limit.

For example, the algorithm for the left limit (optional left limit) of the selectable range can refer to the algorithm for calculating the lower limit of the selectable range.

Replace with the horizontal size of the field of view corresponding to the first focal length

will

Replaced with the horizontal size of the viewing area corresponding to the second focal length

That's it.

As shown in Figure 10, the current attitude angle is the current yaw angle, and determining the selectable range of the area to be processed in the first image according to the comparison result may include the following operations. If the limit exceeds the first right limit of the first maximum viewing angle area, based on the second right limit, the first right limit, and the horizontal viewing angle area corresponding to the first focal length, determine the selectable range in the first maximum viewing angle area. Right limit.

For example, the algorithm for the right limit of the selectable range can refer to the above-mentioned algorithm for calculating the upper limit of the selectable range.

will

That's it.

In another embodiment, after determining the selectable range of the area to be processed in the first image according to the comparison result, the method may further include the following operations.

First, determine the area to be processed, where the second largest viewing angle area includes the area to be processed. For example, the to-be-processed area input by the user may be received at the control terminal, or the to-be-processed area may be determined based on the image recognition algorithm at the photographing device or the control terminal. Among them, the area to be processed should be included in the second maximum viewing angle area to avoid multiple second images obtained by taking pictures of the area to be processed under the second focal length, which cannot cover the complete image of the area to be processed (such as after stitching) Form a "m" font image).

In a specific implementation, the area to be processed is determined by way of user input as an example for description. When the user inputs the area to be processed in the control terminal, the control terminal or camera device (the control terminal sends the area to be processed by the user to the camera device) is required to determine whether the candidate area to be processed input by the user exceeds the selectable range. The area to be processed may be a frame-like area input by the user from the display interface of the control terminal, an area formed by multiple coordinates input by the user, or an area formed by multiple points input by the user.

For example, before determining the area to be processed, the method may further include the following operations. The control terminal or camera device can obtain the candidate to-be-processed area (such as the person image area, the scenic spot image area, etc. obtained through the algorithm, or the key monitoring area determined according to the preset task, etc.), and accordingly, determine the to-be-processed area Including: if it is determined that the candidate to-be-processed area does not exceed the selectable range, then the candidate to-be-processed area is determined to be the to-be-processed area. In addition, in order to facilitate the user to input a suitable candidate to-be-processed area, such as to prevent the candidate to-be-processed area from exceeding the selectable range, a prompt message may be output on the control terminal. For example, the selectable range is displayed as the prompt information on the display interface of the control terminal. In addition, after obtaining the candidate to-be-processed area, the method may further include: if it is determined that the candidate to-be-processed area is beyond the selectable range, outputting prompt information at the control terminal. For example, when the candidate to-be-processed area input by the user exceeds the selectable range, prompt information is output on the control terminal. The prompt information includes but is not limited to: image information, text information, sound information, and somatosensory information, such as when the candidate to-be-processed area When the lower limit of the selectable range exceeds the lower limit of the selectable range, the lower limit of the selectable range is displayed on the display interface of the control terminal. Among them, it can be understood that when the selectable range is displayed on the display interface of the control terminal, other areas on the first image in the display interface other than the selectable range cannot be selected, that is, the user is restricted to only select within the selectable range. That is, in this case, the obtained candidate to-be-processed area is the to-be-processed area, and there is no need to further determine whether the candidate to-be-processed area exceeds the selectable range.

In a specific implementation, the determination of the area to be processed based on the image recognition algorithm is taken as an example. The determination of the area to be processed based on the image recognition algorithm may include: As the area to be processed. Among them, a variety of suitable image recognition technologies can be used in the above-mentioned image recognition process, such as CAD-like object recognition methods (edge detection, primal sketch), and appearance (apprearance-based) recognition methods (such as edge matching, divide-conquer search algorithm, grayscale matching, gradient matching, historgrams) of receptive field response, or large model (1arge model bases, etc.), feature-based recognition methods (such as interpretation tree, hypothesizing-testing), pose consistency ), pose clustering, invariance, geometric hashing, scale-invariant feature transform (SIFT), based on fast and robust features (speeded up robust) feature, SURF), genetic algorithms (genetic algorithms), etc.

In another specific implementation, the area to be processed is determined based on a preset task and an image recognition algorithm as an example for description. With reference to the security scenario shown in Figure 2, the user can set a preset task through the control terminal in advance, such as an image collection task based on a preset period. When the preset task is triggered, the camera device will be controlled to take photos based on the preset task. Among them, the preset task can include a variety of task parameters, such as task period, shooting pose sequence information (the shooting pose sequence information can be input by the user, or it can be automatically controlled by the control terminal according to the key monitoring area and focal length information input by the user. Generated), alarm-related information, etc., where the shooting pose sequence information may include the number of photos taken and the pose information of each shot, focal length information, sensitivity information, shutter information, and the like. Alarm related information includes but is not limited to: alarm trigger conditions, alarm mode, etc. The recognition result of the recognition by the image recognition algorithm is compared with the alarm triggering conditions to determine whether to alarm. For example, if it is determined that there are flammable and explosive items in a certain location monitored through image recognition, an alarm is required. For another example, in the field of equipment condition monitoring, monitored objects such as meters and fasteners are usually fixed at designated locations, and the status information of the monitored object is determined based on preset tasks and image recognition algorithms, and then whether the monitored object is determined based on the status information of the monitored object Make an alarm.

Then, the photographing related information is determined based on the second focal length and the to-be-processed area, so as to acquire at least one second image of the to-be-processed area at the second focal length based on the photographing-related information.

In this embodiment, the photographing related information may include: photographing pose sequence information, camera photographing related information, camera type information, and the like. Among them, the shooting pose sequence information may include posture adjustment device associated information. For example, the associated information of the attitude adjustment device includes but is not limited to at least one of the following: attitude angle sequence information, time sequence information, and the like. Camera shooting related information includes but is not limited to at least one of the following: exposure duration sequence information, sensitivity sequence information, denoising mode sequence information, output format sequence information, compression mode sequence information, etc. (It should be noted that the camera shooting related information Each sequence information may also be a single value, for example, the shooting parameters for shooting under each shooting pose of the shooting pose sequence information are the same). For example, the shooting pose sequence information is determined based on the shooting combination information, and the shooting combination information is determined based on the second focal length and the area to be processed.

Specifically, the photographing combination information may include: the number of photographs and the area to be photographed for each photograph. Correspondingly, determining the photographing associated information based on the second focal length and the area to be processed includes: determining the number of photographs and the area to be photographed for each photograph based on the second focal length and the area to be processed. Determine the shooting pose sequence information based on the area to be photographed. Wherein, each shooting pose in the shooting pose sequence information corresponds to a to-be-photographed area, so that under each shooting pose of the shooting pose sequence information, a second image corresponding to the to-be-photographed area is captured at the second focal length.

In a specific embodiment, based on the second focal length and the area to be processed, determining the number of photographs and the area to be photographed for each photograph may include the following operations. First, the viewing angle area corresponding to the second focal length is determined based on the size of the image sensor and the second focal length. Then, the number of photographs and the area to be photographed for each photograph are determined based on the area to be processed and the viewing angle area corresponding to the second focal length. Among them, the area formed by multiple areas to be photographed includes the area to be processed.

FIG. 11 is a schematic diagram of the division of a to-be-processed area provided by an embodiment of the disclosure.

As shown in Figure 11, the size of the camera's image sensor is fixed. Under the second focal length in a specific posture (the size of the image obtained under the long focal length is smaller than the size of the image obtained under the short focal length), the image sensor can capture The size of the image on the display interface is: width W1, length L1. The size of the area to be processed on the display interface is: width W2, length L2. Then the area to be processed can be divided into:

Regions, of which,

Indicates rounding up. In this way, the number of photographs that need to be taken can be determined, so that the multiple second images taken at the second focal length can completely cover the image of the area to be processed in the first image taken at the first focal length. In addition, each divided area can be used as an area to be photographed.

In order to ensure that the multiple second images taken at the second focal length can completely cover the image of the area to be processed in the first image taken at the first focal length, a certain redundant area may be set for the area to be photographed. For example, determining the number of photos taken based on the area to be processed and the viewing angle area corresponding to the second focal length, and the area to be photographed for each photo may include: based on the area to be processed, the viewing angle area corresponding to the second focal length and the area to be photographed. The image overlap ratio determines the number of photographs and the area to be photographed for each photograph. Wherein, the image overlap ratio may be a preset value, the image overlap ratio may be for one or more side lengths (for example, including at least one of the width overlap ratio and the length overlap ratio), or it may be for area (for example, Area overlap ratio).

FIG. 12 is a schematic diagram of an area to be photographed according to an embodiment of the disclosure.

As shown in FIG. 12, the illustrated area to be processed is divided into 9 areas to be photographed, and there is a specific image overlap ratio between adjacent areas to be photographed. Since the horizontal length and the vertical length of the area to be photographed can be different, the horizontal image overlap ratio (overlap _x ) and the vertical image overlap ratio (overlap _y ) can be set. Of course, the same image overlap ratio can also be set for the horizontal and vertical directions of the area to be photographed. In other embodiments, if the area to be photographed is a square, an image overlap ratio with the same horizontal and vertical overlap ratio can be set.

In another embodiment, a camera with two lens groups is used as an example for description. If the first focal length is used for focusing by the first lens group, the second focal length is used for focusing by the second lens group. The baseline deviation always exists, and the baseline deviation needs to be corrected, and the optical axis of the second lens group needs to be aligned with the optical axis of the original first lens by adjusting the posture adjustment device to take an image of a designated area. Specifically, determining the photographing pose sequence information based on the area to be photographed may include: determining the photographing pose corresponding to the area to be photographed for each photograph based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, and The set of all the determined shooting poses is used as the shooting pose sequence information.

In another embodiment, the baseline deviation and the image overlap ratio can also be considered at the same time to improve the accuracy of the shooting posture of the second image. For example, determining the shooting pose sequence information based on the area to be photographed may include: based on the current pose of the camera at the first focal length, the angular position deviation of the second focal length, and the image overlap between the viewing angle area corresponding to the second focal length and the area to be photographed The ratio determines the shooting pose corresponding to the area to be photographed for each shot, and the set of all the determined shooting poses is used as the shooting pose sequence information.

Wherein, the deviation of the second focal length angle position may be determined based on the first deviation, the second deviation, and the third deviation. Specifically, the first deviation between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length is determined, for example, the designated position may be the center point. The second deviation between the designated position of the area to be processed and the designated position of the viewing angle area corresponding to the first focal length is determined. Determine the third deviation between the designated position of the area to be photographed and the designated position of the area to be processed. In this way, the second focal length angular position deviation between the area to be photographed and the designated position of the viewing angle area corresponding to the first focal length can be determined based on the first deviation, the second deviation, and the third deviation. In this way, by controlling the pan-tilt so that the camera deviates from the current attitude angle of the above-mentioned second focal length angle position deviation, it is possible to take pictures of the area to be photographed.

In the following, a specific embodiment is used to illustrate how to adjust the posture of the pan-tilt to realize the photographing of the area to be processed.

According to the focal length of the current telephoto camera and the size information of the sensor, the horizontal angle of view (fov _zx ) corresponding to the second focal length and the size corresponding to the second focal length are calculated using formulas (1) and (2). The size of the vertical viewing angle (fov _{zy ).}

Among them, W and H are the width and height of the sensor, and focal_length is the second focal length.

Assuming that it is necessary to take a picture of the area to be processed at the second focal length, the horizontal angle of view at the first focal length corresponding to the area to be processed is (fov _wx ), and the angle of view at the first focal length corresponding to the area to be processed The vertical viewing angle of the area is (fov _wy ), in order to ensure that the image taken at the second focal length (such as through a telephoto lens) can completely cover the area to be processed, the conditions shown in equations (3) and (4) can be satisfied .

fov _zx +(1-overlap _x )*fov _zx *(m-1)≥fov _wx formula (3)

fov _zy +(1-overlap _y )*fov _zy *(n-1)≥fov _wy formula (4)

Among them, overlap _x is the image overlap ratio in the horizontal direction, and overlap _y is the image overlap ratio in the vertical direction, as shown in FIG. 12, that is, the length overlap ratio and the width overlap ratio. m and n are the number of photos that need to be taken horizontally and vertically respectively. In actual use, overlap _x can be specified by the user or calculated by an algorithm through special rules.

In order to eliminate the baseline deviation as shown above, the internal parameters, rotation matrix and translation matrix between the first lens group and the second lens group can be calibrated to calculate the specific target distance (in general, it is assumed to be infinity). Then, calculate the first deviation from the center of the viewing angle area corresponding to the second focal length to the center of the viewing angle area corresponding to the first focal length. Specifically, it can be as shown in formula (5) to formula (7).

Assuming that the position of the center of the area to be processed is (a, b), and the center of the view area corresponding to the first focal length is (wide_center_x, wide_center_y), the center of the area to be processed can be calculated relative to the view area corresponding to the first focal length The second deviation (offset_2 _x , offset_2 _y ) of the center of, can be specifically expressed by equations (8) and (9).

offset_2 _x = wide_center_x-a formula (8)

offset_2 _y = wide_center_y-b equation (9)

_{After m and n are determined, the third deviation (offset_3 x} , offset_3 _y ) of the first area to be photographed in the upper left corner relative to the center of the area to be processed can be calculated.

FIG. 13 is a schematic diagram when the number of shots provided by an embodiment of the disclosure is an odd number, and FIG. 14 is a schematic diagram when the number of shots provided by an embodiment of the disclosure is an even number.

The third deviation (offset_3 _x , offset_3 _y ) can be calculated using equations (10) and (11).

Among them, fov _x -overlap _x is the horizontal viewing angle after removing the overlap, and fov _y -overlap _y is the vertical viewing angle after removing the overlap.

The sum of these offsets obtains the angular position deviation of the previously calculated m*n area to be photographed in the upper left corner of the area to be photographed relative to the first lens group in the current pose. Among them, the current pose may refer to the current shooting pose angle of the camera. The PTZ can be controlled to adjust the posture of the camera based on the angular position deviation, so that the camera can take a picture of the area to be photographed in the upper left corner of the m*n areas to be photographed under the second focal length.

Specifically, the angular position deviation of the area to be photographed in the upper left corner with respect to the first lens group in the current pose can be determined by formulas (12) and (13).

offset _x =offset_1 _x +offset_2 _x +offset_3 _x formula (12)

offset _y =offset_1 _y +offset_2 _y +offset_3 _y formula (13)

For example, the first focal length is realized by a wide-angle camera (including the first lens group), and the second focal length is realized by a telephoto camera (including the second lens group). Assuming that the current wide-angle camera position is (yaw _current , pitch _current ), the position corresponding to the area to be photographed in the upper left corner of the wide-angle camera is (yaw _target (1, 1), pitch _target (1, 1)), which can be obtained by (14) and formula (15) for calculation.

yaw _target (1, 1) = yaw _current + offset _x equation (14)

pitch _target (1, 1) = pitch _current + offset _y formula (15)

_{After obtaining the position (yaw target} , pitch _target ) corresponding to the photo in the upper left corner of the wide-angle camera, combine the current wide-angle camera's field of view (that is, the horizontal field of view corresponding to the first focal length (fov _zx , fov _zy )) and (overlap _x , Overlap _y ), the positions of other areas to be photographed in m*n areas to be photographed can be obtained. For example, it can be calculated by formula (16) and formula (17).

yaw _target (m,n)=yaw _target (1,1)+(1-overlap _x )*fov _zx *(m-1)

Formula (16)

pitch _target (m, n) = pitch _target (1, 1) + (1-overlap _y )*fov _zy *(n-1)

Formula (17)

Through the above operations, the angular position deviation of each area to be photographed relative to the current posture of the camera can be determined, and the photographing pose sequence information can be obtained.

In another embodiment, after the shooting pose sequence is determined based on the area to be photographed, the method may further include the following operations: controlling the shooting end to shoot at the second focal length under each shooting pose in the shooting pose sequence information. The second image corresponding to the photographing area. It should be noted that this operation can be performed by the control terminal or by the camera terminal. For example, the control terminal may send a shooting instruction to the shooting terminal. For another example, after obtaining the shooting pose sequence information, the shooting end automatically performs shooting based on the shooting pose sequence information, which is not limited here.

For example, the shooting end includes a shooting device, and controlling the shooting end to shoot the second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information may include the following operations: controlling the shooting device through the attitude adjustment device In each shooting pose in the shooting pose sequence information, the shooting device is controlled to shoot the second images respectively corresponding to the area to be photographed at the second focal length in each shooting pose. For example, the attitude adjustment device is set on a movable platform, the attitude adjustment device can be a multi-axis pan/tilt, and the movable platform can be an unmanned aerial vehicle.

In a specific embodiment, the first image and the second image are captured by a zoom camera at the first focal length and the second focal length, respectively. Referring to the security scene shown in FIG. 2, the camera has a zoom lens, and the first image and the second image are obtained by adjusting the focal length of the zoom lens and adjusting the posture of the camera.

In another specific embodiment, the first image is taken by a first photographing device, and the second image is taken by a second photographing device. Wherein, the focal length of the first photographing device is at least partially different from the focal length of the second photographing device. For example, the first photographing device and the second photographing device are two independent photographing devices, which can be set on the same pan-tilt, so that the first photographing device and the second photographing device can move synchronously. For another example, the distance between the first camera and the second camera is fixed, and the first camera is set on the first platform, the second camera is set on the second platform, and the first camera and the second camera are The pan/tilt can move synchronously. For another example, the first imaging device and the second imaging device are packaged in a housing and share an image sensor, and the optical axis of the lens group of the first imaging device and the optical axis of the lens group of the second imaging device are kept parallel.

In another specific embodiment, the focal length of at least one of the first camera and the second camera is adjustable. For example, the first photographing device is a fixed focal length camera, and the second photographing device is an adjustable focal length camera. For another example, the first imaging device is an adjustable focal length camera, and the second imaging device is a fixed focal length camera. For another example, both the first imaging device and the second imaging device are cameras with adjustable focal lengths. Using two shooting devices for shooting is compared to using a shooting device with an adjustable focal length (such as a high-power zoom lens with a larger focal length adjustment range). When applied to a movable platform (such as a drone), there are It helps to reduce the influence of the change of the center of gravity of the lens group due to the adjustment of the focal length, which in turn leads to the change of the center of gravity of the drone. In addition, it can also effectively reduce the high cost caused by the use of high-power zoom lenses.

In another specific embodiment, the first photographing device is a wide-angle camera, and the second photographing device is a telephoto camera. Among them, the telephoto camera may be a camera with an adjustable focal length. The adjustable focal length range of the telephoto camera can be determined according to the cost of the camera and so on. In addition, the adjustable focus range of the telephoto camera may be smaller than the adjustable focus range of the above-mentioned high-power zoom lens. For example, the adjustable focal length range of a telephoto camera is located on the larger focal length side of the adjustable focal range of a high-power zoom lens.

It should be noted that the first photographing device described above may include a first lens group, and the second photographing device may include a second lens group.

In addition, in order to facilitate the user to refer to the first image and the second image, after controlling the shooting end to shoot the second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information, the method also It may include the following operations: storing the first image and the second image in association. For example, the SD card stores the HTML file and the first image and the second image, so that the SD card is coupled to the personal computer (PC) side, and the first image and the second image can be displayed on the PC side based on the HTML file. Wherein, the HTML file may include the mapping relationship between the first image and the second image.

Specifically, storing the first image and the second image in association may include the following operations. First, determine the first mapping relationship between the second image and the area to be processed, and determine the second mapping relationship between the area to be processed and the first image. Then, the first image, the area to be processed, the second image, the first mapping relationship, and the second mapping relationship are stored. Among them, the area to be processed may be expressed as range information, etc., such as a range determined based on multiple coordinates.

In a specific embodiment, when applied to the shooting terminal, the second image is an unprocessed shooting image. For example, the second image is the original captured image. In this way, it is convenient for users to refer to the original photographed images on terminals such as computers. It should be noted that, in order to improve the display effect, the original captured image may undergo pre-processing such as noise reduction, object recognition, and recognition of object region marking.

In a specific embodiment, when applied to the control terminal of the shooting terminal, the second image is a processed captured image, and the resolution of the second image is smaller than the resolution of the unprocessed captured image. For example, image compression may be performed on the second image to reduce the storage occupied by the second image. Alternatively, the network resources occupied by the transmission of the second image can be reduced. For example, in the communication process between the drone and the control terminal, the drone side transmits a thumbnail of the second image. When the control terminal needs to consult the specific second image, it needs to be downloaded from the drone side. After the transmission is compressed The second image is helpful to reduce network resource consumption, and helps to improve the transmission rate and user command response speed. The process of image compression on the second image may occur after the second image is captured, after receiving a user instruction for viewing the second image, or after receiving a user for compressing the second image After the instruction, there is no restriction here.

In addition, after adjusting the zoom factor, the zoom lens can obtain the shooting image according to the new zoom factor (for example, when the user changes the second focal length by adjusting the magnification adjustment component in the display interface of the control terminal), the camera device can real-time capture the shots acquired by the zoom lens The picture is sent to the control terminal, so that the control terminal displays the picture obtained by the zoom lens, so that the user can intuitively check whether the sharpness of the picture obtained by the zoom lens meets their own needs.

In another embodiment, the above method may further include the following operation: if an operation instruction for viewing the second image is received, obtaining an unprocessed photographed image corresponding to the second image from the photographing terminal. Then, the unprocessed captured image corresponding to the second image is displayed. For example, after the camera captures the second image at the second focal length, in order to facilitate the user to confirm the shooting effect, the drone side may send the compressed image of the second image to the control terminal side for display. If the user wants to view the original photographed image of the second image, he can send an operation instruction for viewing the second image to the side of the drone (or the photographing device on it) through the control terminal side. In response to the operation instruction for viewing the second image, the drone side sends the original captured image of the second image to the control terminal side. In this way, the information interaction speed between the control terminal side and the drone side can be satisfied at the same time, and the multiple needs of the user for the second image can also be met.

In addition, in order to further improve the convenience for users to view the image for the area to be processed, the control camera is used to shoot at the second focal length and the area to be photographed (which may include multiple areas to be photographed) under each shooting position of the shooting position sequence. ) After the corresponding second image, the method may further include the following operations. First, a third image is acquired, which is synthesized based on the second image. Then, the first image and the third image are stored in association. Wherein, the third image may be synthesized using multiple second images, for example, based on image stitching technology. For the overlapping image parts between the multiple second images that are stitched, interpolation can be used to make the stitching more natural.

Next, the first image is captured through the wide-angle lens and the second image is captured through the telephoto lens, combined with the display interface of the control terminal, to illustrate the viewing of the photographing result.

Figures 15-17 are schematic diagrams of display images provided by embodiments of the disclosure.

As shown in FIG. 15, the content displayed on the display interface 1501 of the control terminal includes: the first image captured at the first focal length under the current posture of the first camera. The user selects the area to be processed 1502 (included in the second maximum viewing angle area corresponding to the second focal length) from the first image, and sets the second focal length. The photographing terminal or the controlling terminal divides the area to be processed 1502 based on the second focal length, determines four areas to be photographed, and determines photographing related information of the four areas to be photographed. In this way, the shooting end can respectively take pictures of the four areas to be photographed at the second focal length based on the photographing related information, so as to obtain four super-analyzed second images. Then, the first image and the second image can be associated and stored, such as stored in an SD card.

In some possible embodiments, the display interface 1501 further includes a focal length adjustment component 1503, focal length information, etc., so that the user can adjust the second focal length. The current second focal length (ZOOM) is 10 times the focal length. In addition, the display interface 1501 may also include the number of shots and/or the shooting duration of the zoom lens. The shooting information changes as the shooting parameters of the zoom lens change. As a result, the picture parameters of the wide-angle lens will change accordingly, and the lens parameters will change accordingly.

As shown in FIG. 16, after coupling a readable storage medium, such as an SD card, to a terminal device such as a computer, a display interface 1601 of the terminal's display is shown. When the first image is displayed, annotation information 1602 may be displayed, such as the range mark corresponding to the area to be processed in FIG. 15, the text annotation information of the second image stored in association, and the like. Of course, the image identification and format information (DJI00001.jpeg) of the first image can also be displayed. The user can view the second image stored in association by clicking the annotation information 1602 or a specific functional component. For example, the terminal can determine the second image that the user wants to view based on the HTML file and the annotation information 1602, and display it. In addition, the user can hide the annotation information 1602 through a specific operation to ensure the viewing effect. The display interface 1601 of the terminal's display may also include an operation button 1603, so that the user can also perform operations such as editing, deleting, and sharing the first image.

It should be noted that when the user needs to print the first image or the second image, the label information may not be printed. For example, when sending the first image or the second image to the printing device on the terminal or drone side, the HTML file may not be sent.

As shown in FIG. 17, it is a display interface 1601 of the display of the terminal. Among them, the relationship between the currently displayed second image 1701 and the first image can be displayed. The second image 1701 shown in FIG. 17 is the first (1 of 4) of the four second images corresponding to the to-be-processed area of the first image. The user can also edit, delete, and share the second image 1701.

The foregoing describes the methods of the embodiments of the present application in detail. In order to facilitate better implementation of the above solutions of the embodiments of the present application, correspondingly, related devices for cooperating with the implementation of the foregoing solutions are also provided below.

FIG. 18 is a schematic structural diagram of an image processing device provided by an embodiment of the disclosure.

As shown in FIG. 18, the image processing device 1800 includes:

One or more processors 1810; and

The computer-readable storage medium 1820 is configured to store one or more computer programs 1821. When the computer program 1821 is executed by the processor 1810, it realizes:

Obtaining a first maximum viewing angle area corresponding to the first focal length under the current attitude angle, and obtaining a second maximum viewing angle area corresponding to the second focal length;

Comparing the first largest viewing angle area with the second largest viewing angle area; and

Determine the selectable range of the area to be processed in the first image according to the comparison result, so as to obtain at least one second image of the area to be processed at the second focal length, where the first image is obtained at the first focal length;

Wherein, the second focal length is greater than the first focal length.

In some possible embodiments, the second maximum viewing angle area corresponding to the second focal length is determined based on the maximum rotatable angle of the posture adjustment device.

In some possible embodiments, determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the second maximum viewing angle area includes the first maximum viewing angle area, it is determined that the selectable range of the area to be processed in the first image is any area in the first image.

If it is determined according to the comparison result that the second maximum viewing angle area includes part of the first maximum viewing angle area, the selectable range of the area to be processed in the first image is determined to be the overlapping area of the first maximum viewing angle area and the second maximum viewing angle area.

In some possible embodiments, the current attitude angle is the current pitch angle;

The selectable range of the region to be processed in the first image is determined according to the comparison result, including:

If it is determined according to the comparison result that the first upper limit of the first maximum viewing angle area exceeds the second upper limit of the second maximum viewing angle area, the selectable range is determined based on the first upper limit, the second upper limit, and the vertical viewing angle area corresponding to the first focal length Optional upper limit in the first maximum viewing angle area.

If it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds the first lower limit of the first maximum viewing angle area, then based on the second lower limit, the first lower limit, and the vertical viewing angle area corresponding to the first focal length, determine the optional The range is the optional lower limit of the first maximum viewing angle area.

In some possible embodiments, the current attitude angle is the current yaw angle;

If it is determined according to the comparison result that the first left limit of the first maximum angle of view area exceeds the second left limit of the second maximum angle of view area, then based on the first left limit, the second left limit, and the horizontal angle of view area corresponding to the first focal length, The selectable range is determined to be the selectable left limit of the first maximum viewing angle area.

If it is determined according to the comparison result that the second right limit of the second maximum angle of view area exceeds the first right limit of the first maximum angle of view area, based on the second right limit, the first right limit, and the horizontal direction angle of view area corresponding to the first focal length To determine the selectable range at the selectable right end of the first maximum viewing angle area.

In some possible embodiments, when applied to the control end of the shooting end, the computer program 1821, when being executed by the processor 1810, is also used to implement:

At least one of the multiple boundaries of the selectable range is displayed.

In some possible embodiments, after the selectable range of the area to be processed in the first image is determined according to the comparison result, when the computer program 1821 is executed by the processor 1810, it is further used to implement:

Determining the area to be processed, where the second largest viewing angle area includes the area to be processed; and

The photographing associated information is determined based on the second focal length and the area to be processed, so as to acquire at least one second image of the area to be processed at the second focal length based on the photographing associated information.

In some possible embodiments, before determining the area to be processed, when the computer program 1821 is executed by the processor 1810, it is also used to implement:

Obtain candidate areas to be processed;

Determine the area to be processed, including:

If it is determined that the candidate to-be-processed area does not exceed the selectable range, the candidate to-be-processed area is determined to be the to-be-processed area.

In some possible embodiments, after obtaining the candidate to-be-processed area, when the computer program 1821 is executed by the processor 1810, it is further used to implement:

If it is determined that the candidate to-be-processed area exceeds the selectable range, a prompt message is output.

In some possible embodiments, determining the area to be processed includes:

Determine the area to be processed based on the image recognition algorithm.

In some possible embodiments, determining the area to be processed based on the image recognition algorithm includes:

Perform image recognition on the first image to determine the object to be processed; and

Use the area where the object to be processed is located as the area to be processed.

The area to be processed is determined from the first image based on the preset task and the image recognition algorithm.

In some possible embodiments, the photographing related information includes: photographing pose sequence information.

In some possible embodiments, the shooting pose sequence information is determined based on the shooting combination information, and the shooting combination information is determined based on the second focal length and the area to be processed.

In some possible embodiments, the photographing combination information includes: the number of photographs and the area to be photographed for each photograph;

Determining photographing related information based on the second focal length and the area to be processed includes:

Based on the second focal length and the area to be processed, determine the number of photographs and the area to be photographed for each photograph; and

Determine the shooting pose sequence information based on the area to be photographed;

Wherein, each shooting pose in the shooting pose sequence information corresponds to a to-be-photographed area, so that under each shooting pose of the shooting pose sequence information, a second image corresponding to the to-be-photographed area is captured at the second focal length.

In some possible embodiments, based on the second focal length and the area to be processed, determining the number of photographs and the area to be photographed for each photograph includes:

Determining a viewing angle area corresponding to the second focal length based on the size of the image sensor and the second focal length;

Determine the number of photographs and the area to be photographed for each photograph based on the area to be processed and the viewing angle area corresponding to the second focal length;

Wherein, the area formed by a plurality of areas to be photographed includes the area to be processed.

In some possible embodiments, the determination of the number of photographs based on the area to be processed and the viewing angle area corresponding to the second focal length, and the area to be photographed for each photograph include:

The number of photographing times and the area to be photographed for each photograph are determined based on the image overlap ratio between the area to be processed, the viewing angle area corresponding to the second focal length, and the area to be photographed.

In some possible embodiments, determining the shooting pose sequence information based on the area to be photographed includes:

The shooting pose corresponding to the area to be photographed each time is determined based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, and the set of all the determined shooting poses is used as the shooting pose sequence information.

In some possible embodiments, the angular position deviation of the second focal length is determined in the following manner:

Determining the first deviation between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length;

Determining the second deviation between the designated position of the area to be processed and the designated position of the viewing angle area corresponding to the first focal length;

Determine the third deviation between the designated position of the area to be photographed and the designated position of the area to be processed; and

Based on the first deviation, the second deviation, and the third deviation, the second focal length angular position deviation between the to-be-photographed area and the designated position of the viewing angle area corresponding to the first focal length is determined.

In some possible embodiments, after the photographing pose sequence is determined based on the area to be photographed, the computer program 1821, when being executed by the processor 1810, is also used to implement:

The shooting end is controlled to shoot the second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information.

In some possible embodiments, the shooting end includes a shooting device, and controlling the shooting end to shoot the second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information includes:

The posture adjustment device controls the shooting device to sequentially be in each shooting pose in the shooting pose sequence information, and controls the shooting device to shoot the second images corresponding to the area to be photographed at the second focal length under each shooting pose.

In some possible embodiments, the posture adjustment device is provided on a movable platform.

In some possible embodiments, the computer program 1821 is executed by the processor 1810 after controlling the shooting end to shoot the second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information. When, it is also used to achieve:

The first image and the second image are stored in association.

In some possible embodiments, storing the first image and the second image in association includes:

Determine the first mapping relationship between the second image and the area to be processed, and determine the second mapping relationship between the area to be processed and the first image; and

Store the first image, the area to be processed, the second image, the first mapping relationship, and the second mapping relationship.

In some possible embodiments, when applied to the shooting end, the second image is an unprocessed shooting image.

In some possible embodiments, applied to the control terminal of the shooting terminal, the second image is a processed captured image, and the resolution of the second image is smaller than the resolution of the unprocessed captured image.

In some possible embodiments, when the computer program 1821 is executed by the processor 1810, it is also used to implement:

If an operation instruction for viewing the second image is received, obtaining an unprocessed photographed image corresponding to the second image from the photographing terminal;

Display the unprocessed captured image corresponding to the second image.

In some possible embodiments, after controlling the shooting end to shoot the second image corresponding to the area to be photographed at the second focal length in each shooting position of the shooting position sequence, the computer program 1821 is executed by the processor 1810. , Also used to achieve:

Acquire a third image, the third image is synthesized based on the second image; and

The first image and the third image are stored in association.

In some possible embodiments, the first image and the second image are captured by the zoom camera at the first focal length and the second focal length, respectively.

In some possible embodiments, the first image is taken by a first photographing device, and the second image is taken by a second photographing device; wherein the focal length of the first photographing device is at least partially different from the focal length of the second photographing device.

In some possible embodiments, the focal length of at least one of the first camera and the second camera is adjustable.

In some possible embodiments, the first camera is a wide-angle camera, and the second camera is a telephoto camera.

In some possible embodiments, the second focal length includes any one of the following: a fixed focal length, a preset focal length, a focal length determined based on a focal length algorithm, a focal length determined based on a sensor, and a focal length selected by the user.

In some possible embodiments, the focal length determined based on the sensor includes a focal length determined based on distance information, and the distance information is determined by a laser rangefinder.

FIG. 19 is a schematic structural diagram of an image processing system provided by an embodiment of the disclosure.

As shown in FIG. 19, the image processing system 1900 may include:

A photographing device 1910 and a control terminal 1920 communicatively connected with the photographing device 1910;

The photographing device 1910 is used to obtain the first maximum viewing angle area corresponding to the first focal length under the current attitude angle of the photographing device 1910, and to obtain the second maximum viewing angle area corresponding to the second focal length; The viewing angle area is compared; and the selectable range of the area to be processed in the first image is determined according to the comparison result, so as to control the photographing device 1910 to obtain at least one second image of the area to be processed at the second focal length, and the first image is in the first image. Obtained at a focal length; where the second focal length is greater than the first focal length.

In some possible embodiments, the photographing device 1910 is specifically configured to determine that the selectable range of the area to be processed in the first image is in the first image if it is determined according to the comparison result that the second maximum angle of view area includes the first maximum angle of view area. Any area of.

In some possible embodiments, the photographing device 1910 is specifically configured to determine that the second maximum viewing angle area includes part of the first maximum viewing angle area according to the comparison result, and then determining that the selectable range of the to-be-processed area in the first image is the first maximum viewing angle area. The area of overlap between the viewing angle area and the second largest viewing angle area.

In some possible embodiments, the photographing device 1910 is specifically used when the current attitude angle is the current pitch angle. If it is determined according to the comparison result that the first upper limit of the first maximum viewing angle area exceeds the second upper limit of the second maximum viewing angle area, it is based on The first upper limit, the second upper limit, and the vertical viewing angle area corresponding to the first focal length determine the optional upper limit of the selectable range in the first maximum viewing angle area.

In some possible embodiments, the photographing device 1910 is specifically used when the current attitude angle is the current pitch angle. If it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds the first lower limit of the first maximum viewing angle area, it is based on The second lower limit, the first lower limit, and the vertical viewing angle area corresponding to the first focal length determine the optional lower limit of the selectable range in the first maximum viewing angle area.

In some possible embodiments, the photographing device 1910 is specifically used when the current attitude angle is the current yaw angle, if it is determined according to the comparison result that the first left limit of the first maximum angle of view area exceeds the second left limit of the second maximum angle of view area , Based on the first left limit, the second left limit, and the horizontal viewing angle area corresponding to the first focal length, the selectable range is determined to be the selectable left limit of the first maximum viewing angle area.

In some possible embodiments, the photographing device 1910 is specifically used when the current attitude angle is the current yaw angle, if it is determined according to the comparison result that the second right limit of the second maximum viewing angle area exceeds the first right limit of the first maximum viewing angle area , Based on the second right limit, the first right limit, and the horizontal viewing angle area corresponding to the first focal length, the selectable range is determined to be the selectable right limit of the first maximum viewing angle area.

In some possible embodiments, the control terminal 1920 includes a display interface, and the control terminal 1920 is further configured to display at least one of multiple boundaries of the selectable range through the display interface.

In some possible embodiments, the photographing device 1910 is further configured to determine the area to be processed after determining the selectable range of the area to be processed in the first image according to the comparison result, where the second largest viewing angle area includes the area to be processed; And determining photographing related information based on the second focal length and the area to be processed, so as to control the photographing device 1910 to acquire at least one second image of the area to be processed at the second focal length based on the photographing associated information.

In some possible embodiments, the photographing device 1910 is further configured to obtain candidate regions to be processed before determining the regions to be processed;

And it is specifically used to determine that the candidate to-be-processed area is the to-be-processed area if it is determined that the candidate to-be-processed area does not exceed the selectable range.

In some possible embodiments, the photographing device 1910 is further configured to output prompt information if it is determined that the candidate to-be-processed area is beyond the selectable range after acquiring the candidate to-be-processed area.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the area to be processed based on an image recognition algorithm.

In some possible embodiments, the photographing device 1910 is specifically configured to perform image recognition on the first image, determine the object to be processed, and use the area where the object to be processed is located as the area to be processed.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the area to be processed from the first image based on a preset task and an image recognition algorithm.

In some possible embodiments, the photographing combination information includes the number of photographs and the area to be photographed for each photograph;

The photographing device 1910 is specifically configured to determine the number of photographs and the area to be photographed for each photograph based on the second focal length and the area to be processed; and to determine the photographing pose sequence information based on the area to be photographed; wherein, each of the photographing pose sequence information The shooting pose corresponds to a region to be photographed, so that the photographing device 1910 shoots the second image corresponding to the region to be photographed at the second focal length under each shooting pose of the shooting pose sequence information.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the viewing angle area corresponding to the second focal length based on the size of the image sensor and the second focal length; and to determine the number of photographs based on the area to be processed and the viewing angle area corresponding to the second focal length, And the area to be photographed each time the photograph is taken; wherein the area formed by multiple areas to be photographed includes the area to be processed.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the number of photographs and the area to be photographed for each photograph based on the area to be processed, the viewing angle area corresponding to the second focal length, and the image overlap ratio between the area to be photographed.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the photographing pose corresponding to the area to be photographed for each photograph based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, and determine The collection of all shooting poses is used as the shooting pose sequence information.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the first deviation between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length; and determining the area to be processed The second deviation between the designated position of the area and the designated position of the viewing angle area corresponding to the first focal length; and the third deviation between the designated position of the area to be photographed and the designated position of the area to be processed; and based on the first deviation, The second deviation and the third deviation determine the second focal length angular position deviation between the area to be photographed and the designated position of the viewing angle area corresponding to the first focal length.

In some possible embodiments, the photographing device 1910 is specifically configured to photograph the second image corresponding to the area to be photographed at the second focal length under each photographing pose of the photographing pose sequence information.

In some possible embodiments, the photographing device 1910 includes a posture adjusting device and a photographing device. The photographing device 1910 is specifically used to control the photographing device to be in each photographing pose in the photographing pose sequence information sequentially through the pose adjusting device, and to control the photographing device In each shooting posture, a second image corresponding to the area to be photographed is shot at the second focal length.

In some possible embodiments, the photographing device 1910 is further configured to store the first image and the second image corresponding to the area to be photographed at the second focal length under each photographing pose of the photographing pose sequence information. The second image.

In some possible embodiments, the photographing device 1910 is specifically configured to determine the first mapping relationship between the second image and the area to be processed, and determine the second mapping relationship between the area to be processed and the first image; and store the first mapping relationship between the area to be processed and the first image. An image, an area to be processed, a second image, a first mapping relationship, and a second mapping relationship.

In some possible embodiments, the second image is an unprocessed captured image and stored in the photographing device.

In some possible embodiments, the second image is a processed captured image and stored in the control terminal, and the resolution of the second image is smaller than the resolution of the unprocessed captured image.

In some possible embodiments, the control terminal further includes a display interface, and the control terminal 1920 is further configured to obtain an unprocessed image corresponding to the second image from the photographing device 1910 if an operation instruction for viewing the second image is received. Capturing an image; and displaying the unprocessed captured image corresponding to the second image through the display interface.

In some possible embodiments, the photographing device 1910 is also used to control the photographing end to photograph the second image corresponding to the area to be photographed at the second focal length under each photographing position of the photographing position sequence, and then acquire the third image. , The third image is synthesized based on the second image; and the first image and the third image are stored in association.

In some possible embodiments, the photographing device 1910 includes a zoom photographing device, which is used to photograph the first image and the second image at the first focal length and the second focal length, respectively.

In some possible embodiments, the photographing device 1910 includes a first photographing device and a second photographing device;

The first photographing device is used to photograph a first image;

The second photographing device is used to photograph a second image;

Wherein, the focal length of the first photographing device is at least partially different from the focal length of the second photographing device.

In some possible embodiments, the photographing device 1910 includes a laser rangefinder, and the laser rangefinder is used to measure distance information to determine the second focal length based on the distance information.

It should be noted that the image processing system shown above is only exemplary and cannot be construed as a limitation of the present disclosure. Among them, the operations implemented by the photographing device 1910 in the above-mentioned image processing system may be at least partially executed by the control terminal 1920. In addition, the operations implemented by the photographing device 1910 in the above-mentioned image processing system can be at least partially performed by an unmanned aerial vehicle.

For example, the control terminal 1920 may obtain the first maximum angle of view area corresponding to the first focal length of the photographing device 1910 at the current attitude angle, and obtain the second maximum angle of view area corresponding to the second focal length; combine the first maximum angle of view area with the first Compare the two regions with the largest viewing angle; and determine the selectable range of the region to be processed in the first image according to the comparison result, so as to control the photographing device 1910 to obtain at least one second image of the region to be processed at the second focal length, where the first image is Obtained at a first focal length; wherein the second focal length is greater than the first focal length.

For another example, the control terminal 1920 may be configured to determine that the selectable range of the area to be processed in the first image is any area in the first image if it is determined according to the comparison result that the second maximum angle of view area includes the first maximum angle of view area.

It should be noted that the photographing device 1910 described above may be any device with a photographing function, and may refer to a camera, a pan-tilt camera, a movable platform including a camera, and the like.

The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more for realizing the specified logic function. Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by It is realized by a combination of dedicated hardware and computer instructions.

FIG. 20 is a schematic structural diagram of a control terminal provided by an embodiment of the disclosure. As shown in FIG. 20, the control terminal 70 may include: at least one processor 701, such as a CPU, at least one network interface 704, a user interface 703, a memory 705, at least one communication bus 702, and a display screen 706. Among them, the communication bus 702 is used to implement connection and communication between these components. Among them, the user interface 703 may include a touch screen, a keyboard, a mouse, a joystick, and so on. The network interface 704 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface), and a communication connection with the server and the camera 20 can be established through the network interface 704. The memory 705 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 705 includes the flash in the embodiment of the present invention. Optionally, the memory 705 may also be at least one storage system located far away from the foregoing processor 701. As shown in FIG. 20, the memory 705 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

It should be noted that the network interface 704 can be connected to an acquirer, a transmitter, or other communication modules. Other communication modules can include but are not limited to a WiFi module, a Bluetooth module, etc. It should be understood that the control terminal 70 in the embodiment of the present application may also include acquiring Receivers, transmitters and other communication modules.

The processor 701 may be used to call program instructions stored in the memory 705 and perform the following operations:

Wherein, the second focal length is greater than the first focal length.

It is understandable that the function of the control terminal 70 in this embodiment can be specifically implemented according to the method in the above method embodiment, and will not be repeated here.

FIG. 21 is a schematic diagram of the structure of a photographing device provided by an embodiment of the disclosure. As shown in FIG. 21, the photographing device 80 may include: at least one processor 801, such as a CPU, at least one network interface 804, a zoom lens 803, a memory 805, a wide-angle lens 806, and at least one communication bus 802. Among them, the communication bus 802 is used to implement connection and communication between these components. Among them, the network interface 20 port 804 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface), and a communication connection with the control terminal 70 can be established through the network interface 804. The memory 805 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 805 includes the flash in the embodiment of the present invention. Optionally, the memory 805 may also be at least one storage system located far away from the aforementioned processor 801. As shown in FIG. 21, the memory 805 as a computer storage medium may include an operating system, a network communication module, and program instructions.

It should be noted that the network interface 804 can be connected to an acquirer, a transmitter, or other communication modules. Other communication modules can include but are not limited to a WiFi module, a Bluetooth module, etc. It should be understood that the flight trajectory recording device in the embodiment of the present invention can also include Obtainers, transmitters and other communication modules, etc.

The processor 801 may be used to call the program instructions stored in the memory 805 and perform the following operations:

Wherein, the second focal length is greater than the first focal length.

It is understandable that the functions of the photographing device 80 in this embodiment can be specifically implemented according to the methods in the foregoing method embodiments, and details are not described herein again.

The embodiment of the present application also provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, and when it runs on a computer or a processor, the computer or the processor executes any one of the above methods. Or multiple steps. If each component module of the above-mentioned signal processing device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be sent from a website site, computer, server, or data center to another website site via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) , Computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The above-mentioned program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments.

Among them, the aforementioned storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random storage memory RAM, etc. In the case of no conflict, the technical features in this embodiment and the implementation can be combined arbitrarily.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, but not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure .

Claims

An image processing method, characterized in that the method includes:

Obtaining a first maximum viewing angle area corresponding to the first focal length under the current attitude angle, and obtaining a second maximum viewing angle area corresponding to the second focal length;

Comparing the first maximum viewing angle area and the second maximum viewing angle area; and

Determine the selectable range of the area to be processed in the first image according to the comparison result, so as to obtain at least one second image of the area to be processed at the second focal length, where the first image is in the first image. Obtained at focal length;

Wherein, the second focal length is greater than the first focal length.
The method according to claim 1, wherein the second maximum viewing angle area corresponding to the second focal length is determined based on the maximum rotatable angle of the posture adjustment device.
The method according to claim 1, wherein the determining the selectable range of the area to be processed in the first image according to the comparison result comprises:

If it is determined according to the comparison result that the second maximum viewing angle area includes the first maximum viewing angle area, it is determined that the selectable range of the area to be processed in the first image is any area in the first image.
The method according to claim 1, wherein the determining the selectable range of the area to be processed in the first image according to the comparison result comprises:

If it is determined according to the comparison result that the second maximum viewing angle area includes part of the first maximum viewing angle area, it is determined that the selectable range of the area to be processed in the first image is the first maximum viewing angle area and the second maximum viewing angle area. The overlapping area of the viewing angle area.
The method according to claim 1, wherein the current attitude angle is a current pitch angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the first upper limit of the first maximum angle of view area exceeds the second upper limit of the second maximum angle of view area, then based on the first upper limit, the second upper limit, and corresponding to the first focal length The vertical viewing angle area is determined to determine the selectable upper limit of the selectable range in the first maximum viewing angle area.
The method according to claim 1, wherein the current attitude angle is a current pitch angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds the first lower limit of the first maximum viewing angle area, then based on the second lower limit, the first lower limit, and the first focal length Corresponding to the vertical viewing angle area, it is determined that the selectable range is within the selectable lower limit of the first maximum viewing angle area.
The method according to claim 1, wherein the current attitude angle is a current yaw angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the first left limit of the first maximum viewing angle area exceeds the second left limit of the second maximum viewing angle area, then based on the first left limit, the second left limit and the The horizontal viewing angle area corresponding to the first focal length determines that the selectable range is the selectable left limit of the first maximum viewing angle area.
The method according to claim 1, wherein the current attitude angle is a current yaw angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the second right limit of the second maximum viewing angle area exceeds the first right limit of the first maximum viewing angle area, then based on the second right limit, the first right limit, and the The horizontal viewing angle area corresponding to the first focal length determines that the selectable range is at the selectable right limit of the first maximum viewing angle area.
The method according to claim 1, wherein the control terminal applied to the camera terminal further comprises:

At least one of the multiple boundaries of the selectable range is displayed.
The method according to claim 1, wherein after the determining the selectable range of the area to be processed in the first image according to the comparison result, the method further comprises:

Determining the area to be processed, wherein the second maximum viewing angle area includes the area to be processed; and

Determine photographing related information based on the second focal length and the to-be-processed area, so as to acquire at least one second image of the to-be-processed area at the second focal length based on the photographing-related information.
The method according to claim 10, characterized in that, before said determining the area to be processed, further comprising:

Obtain candidate areas to be processed;

The determining the area to be processed includes:

If it is determined that the candidate to-be-processed area does not exceed the selectable range, it is determined that the candidate to-be-processed area is the to-be-processed area.
The method according to claim 11, characterized in that, after said obtaining the candidate to-be-processed area, the method further comprises:

If it is determined that the candidate to-be-processed area exceeds the selectable range, output prompt information.
The method according to claim 10, wherein the determining the area to be processed comprises:

The region to be processed is determined based on an image recognition algorithm.
The method according to claim 13, wherein the determining the area to be processed based on an image recognition algorithm comprises:

Perform image recognition on the first image to determine the object to be processed; and

The area where the object to be processed is located is used as the area to be processed.
The method according to claim 13, wherein the determining the area to be processed based on an image recognition algorithm comprises:

The region to be processed is determined from the first image based on a preset task and an image recognition algorithm.
The method according to claim 10, wherein the photographing related information comprises: photographing pose sequence information.
The method according to claim 16, wherein the photographing pose sequence information is determined based on photographing combination information, and the photographing combination information is determined based on the second focal length and the area to be processed.
The method according to claim 17, wherein the photographing combination information comprises: the number of photographs and the area to be photographed for each photograph;

The determining the photographing associated information based on the second focal length and the area to be processed includes:

Based on the second focal length and the area to be processed, determining the number of photographs and the area to be photographed for each photograph; and

Determining the photographing pose sequence information based on the area to be photographed;

Wherein, each shooting pose in the shooting pose sequence information corresponds to one of the to-be-photographed areas, so that in each shooting pose of the shooting pose sequence information, shooting with the second focal length The second image corresponding to the area to be photographed.
The method according to claim 18, wherein the determining the number of photographs and the area to be photographed for each photograph based on the second focal length and the area to be processed comprises:

Determining a viewing angle area corresponding to the second focal length based on the size of the image sensor and the second focal length;

Determining the number of photographs and the area to be photographed for each photograph based on the area to be processed and the viewing angle area corresponding to the second focal length;

Wherein, the area formed by a plurality of the to-be-photographed areas includes the to-be-processed area.
The method of claim 19, wherein the determining the number of photographs based on the area to be processed and the viewing angle area corresponding to the second focal length, and the area to be photographed for each photograph, comprises:

The number of photographing times and the area to be photographed for each photograph are determined based on the image overlap ratio between the area to be processed, the viewing angle area corresponding to the second focal length, and the area to be photographed.
The method according to claim 18, wherein the determining the photographing pose sequence information based on the area to be photographed comprises:

Determine the shooting pose corresponding to the area to be photographed each time based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, and use the set of all the determined shooting poses as the shooting pose Sequence information.
The method according to claim 21, wherein the second focal length angular position deviation is determined in the following manner:

Determining the first deviation between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length;

Determining a second deviation between the designated position of the area to be processed and the designated position of the viewing angle area corresponding to the first focal length;

Determine the third deviation between the designated position of the area to be photographed and the designated position of the area to be processed; and

Determine a second focal length angular position deviation between the to-be-photographed area and the designated position of the viewing angle area corresponding to the first focal length based on the first deviation, the second deviation, and the third deviation.
The method according to claim 18, characterized in that, after the determination of the shooting pose sequence based on the area to be photographed, the method further comprises:

Control the shooting end to shoot a second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information.
22. The method according to claim 23, wherein the shooting end comprises a shooting device, and the controlling the shooting end is to shoot at the second focal length under each shooting pose of the shooting pose sequence information. The second image corresponding to the area to be photographed includes:

Through the posture adjustment device, the camera is controlled to be in each shooting pose in the shooting pose sequence information in sequence, and the camera is controlled to shoot with the second focal length in each of the shooting poses. The second image corresponding to the area to be photographed.
The method according to claim 24, wherein the posture adjustment device is arranged on a movable platform.
The method according to claim 23, characterized in that, under each shooting pose of the shooting pose sequence information, the shooting end is controlled to shoot the second focal length corresponding to the area to be photographed. After the second image, it also includes:

The first image and the second image are stored in association.
The method according to claim 26, wherein said storing said first image and said second image in association comprises:

Determining a first mapping relationship between the second image and the area to be processed, and determining a second mapping relationship between the area to be processed and the first image; and

The first image, the region to be processed, the second image, the first mapping relationship, and the second mapping relationship are stored.
The method according to claim 26, wherein when applied to a shooting end, the second image is an unprocessed shooting image.
The method according to claim 26, wherein the second image is a processed captured image, and the resolution of the second image is smaller than that of an unprocessed captured image. Rate.
The method according to claim 29, further comprising:

If an operation instruction for viewing the second image is received, obtaining an unprocessed photographed image corresponding to the second image from the photographing terminal;

The unprocessed captured image corresponding to the second image is displayed.
22. The method according to claim 23, characterized in that, in each of the shooting poses of the shooting pose sequence at the control shooting end, the second focal length corresponding to the area to be photographed is shot at the second focal length. After the image, it also includes:

Acquiring a third image, the third image being synthesized based on the second image; and

The first image and the third image are stored in association.
The method according to claim 1, wherein the first image and the second image are taken by a zoom camera at the first focal length and the second focal length, respectively.
The method according to claim 1, wherein the first image is taken by a first photographing device, and the second image is taken by a second photographing device; wherein the focal length of the first photographing device is at least The focal length of part is different from that of the second camera.
The method according to claim 33, wherein the focal length of at least one of the first photographing device and the second photographing device is adjustable.
The method of claim 33, wherein the first camera is a wide-angle camera, and the second camera is a telephoto camera.
The method according to claim 1, wherein the second focal length comprises any one of the following: a fixed focal length, a preset focal length, a focal length determined based on a focal length algorithm, a focal length determined based on a sensor, and a focal length selected by a user.
The method according to claim 36, wherein the focal length determined based on the sensor comprises: a focal length determined based on distance information, the distance information being determined by a laser rangefinder.
An image processing device, characterized in that the device includes:

One or more processors; and

A computer-readable storage medium is used to store one or more computer programs, and when the computer programs are executed by the processor, they realize:

Obtaining a first maximum viewing angle area corresponding to the first focal length under the current attitude angle, and obtaining a second maximum viewing angle area corresponding to the second focal length;

Comparing the first maximum viewing angle area and the second maximum viewing angle area; and

Determine the selectable range of the area to be processed in the first image according to the comparison result, so as to obtain at least one second image of the area to be processed at the second focal length, where the first image is in the first image. Obtained at focal length;

Wherein, the second focal length is greater than the first focal length.
The device according to claim 38, wherein the second maximum viewing angle area corresponding to the second focal length is determined based on the maximum rotatable angle of the posture adjustment device.
The device according to claim 38, wherein the determining the selectable range of the area to be processed in the first image according to the comparison result comprises:

If it is determined according to the comparison result that the second maximum viewing angle area includes the first maximum viewing angle area, it is determined that the selectable range of the area to be processed in the first image is any area in the first image.
The device according to claim 38, wherein the determining the selectable range of the area to be processed in the first image according to the comparison result comprises:

If it is determined according to the comparison result that the second maximum viewing angle area includes part of the first maximum viewing angle area, it is determined that the selectable range of the area to be processed in the first image is the first maximum viewing angle area and the second maximum viewing angle area. The overlapping area of the viewing angle area.
The device according to claim 38, wherein the current attitude angle is a current pitch angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the first upper limit of the first maximum angle of view area exceeds the second upper limit of the second maximum angle of view area, then based on the first upper limit, the second upper limit, and corresponding to the first focal length The vertical viewing angle area is determined to determine the selectable upper limit of the selectable range in the first maximum viewing angle area.
The device according to claim 38, wherein the current attitude angle is a current pitch angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds the first lower limit of the first maximum viewing angle area, then based on the second lower limit, the first lower limit, and the first focal length Corresponding to the vertical viewing angle area, it is determined that the selectable range is within the selectable lower limit of the first maximum viewing angle area.
The device according to claim 38, wherein the current attitude angle is a current yaw angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the first left limit of the first maximum viewing angle area exceeds the second left limit of the second maximum viewing angle area, then based on the first left limit, the second left limit and the The horizontal viewing angle area corresponding to the first focal length determines that the selectable range is the selectable left limit of the first maximum viewing angle area.
The device according to claim 38, wherein the current attitude angle is a current yaw angle;

The determining the selectable range of the area to be processed in the first image according to the comparison result includes:

If it is determined according to the comparison result that the second right limit of the second maximum viewing angle area exceeds the first right limit of the first maximum viewing angle area, then based on the second right limit, the first right limit, and the The horizontal viewing angle area corresponding to the first focal length determines that the selectable range is at the selectable right limit of the first maximum viewing angle area.
The device according to claim 38, wherein, when applied to the control end of the shooting end, when the computer program is executed by the processor, it is further used to realize:

At least one of the multiple boundaries of the selectable range is displayed.
The device according to claim 38, characterized in that, after the selectable range of the area to be processed in the first image is determined according to the comparison result, when the computer program is executed by the processor, the computer program is further used for accomplish:

Determining the area to be processed, wherein the second maximum viewing angle area includes the area to be processed; and

Determine photographing related information based on the second focal length and the to-be-processed area, so as to acquire at least one second image of the to-be-processed area at the second focal length based on the photographing-related information.
The device according to claim 47, wherein, before the determining the area to be processed, the computer program is further used to implement:

Obtain candidate areas to be processed;

The determining the area to be processed includes:

If it is determined that the candidate to-be-processed area does not exceed the selectable range, it is determined that the candidate to-be-processed area is the to-be-processed area.
The device according to claim 48, wherein, after the candidate area to be processed is obtained, when the computer program is executed by the processor, it is further used to implement:

If it is determined that the candidate to-be-processed area exceeds the selectable range, output prompt information.
The device according to claim 47, wherein said determining the area to be processed comprises:

The region to be processed is determined based on an image recognition algorithm.
The device of claim 50, wherein the determining the area to be processed based on an image recognition algorithm comprises:

Perform image recognition on the first image to determine the object to be processed; and

The area where the object to be processed is located is used as the area to be processed.
The device of claim 50, wherein the determining the area to be processed based on an image recognition algorithm comprises:

The region to be processed is determined from the first image based on a preset task and an image recognition algorithm.
The device according to claim 47, wherein the photographing related information comprises: photographing pose sequence information.
The device according to claim 53, wherein the photographing pose sequence information is determined based on photographing combination information, and the photographing combination information is determined based on the second focal length and the area to be processed.
The device according to claim 54, wherein the photographing combination information comprises: the number of photographs and the area to be photographed for each photograph;

The determining the photographing associated information based on the second focal length and the area to be processed includes:

Based on the second focal length and the area to be processed, determining the number of photographs and the area to be photographed for each photograph; and

Determining the photographing pose sequence information based on the area to be photographed;

Wherein, each shooting pose in the shooting pose sequence information corresponds to one of the to-be-photographed areas, so that in each shooting pose of the shooting pose sequence information, shooting with the second focal length The second image corresponding to the area to be photographed.
The device according to claim 55, wherein the determining the number of photographs and the area to be photographed for each photograph based on the second focal length and the area to be processed comprises:

Determining a viewing angle area corresponding to the second focal length based on the size of the image sensor and the second focal length;

Determining the number of photographs and the area to be photographed for each photograph based on the area to be processed and the viewing angle area corresponding to the second focal length;

Wherein, the area formed by a plurality of the to-be-photographed areas includes the to-be-processed area.
The device according to claim 56, wherein the determining the number of photographs based on the area to be processed and the viewing angle area corresponding to the second focal length, and the area to be photographed for each photograph, comprises:

The number of photographing times and the area to be photographed for each photograph are determined based on the image overlap ratio between the area to be processed, the viewing angle area corresponding to the second focal length, and the area to be photographed.
The device according to claim 55, wherein the determining the shooting pose sequence information based on the area to be photographed comprises:

Determine the shooting pose corresponding to the area to be photographed each time based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, and use the set of all the determined shooting poses as the shooting pose Sequence information.
The device according to claim 58, wherein the deviation of the second focal length angle position is determined in the following manner:

Determining the first deviation between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length;

Determining a second deviation between the designated position of the area to be processed and the designated position of the viewing angle area corresponding to the first focal length;

Determine the third deviation between the designated position of the area to be photographed and the designated position of the area to be processed; and

Determine a second focal length angular position deviation between the to-be-photographed area and the designated position of the viewing angle area corresponding to the first focal length based on the first deviation, the second deviation, and the third deviation.
The device according to claim 55, wherein, after the photographing pose sequence is determined based on the area to be photographed, when the computer program is executed by the processor, it is further configured to implement:

Control the shooting end to shoot a second image corresponding to the area to be photographed at the second focal length under each shooting pose of the shooting pose sequence information.
The device according to claim 60, wherein the photographing terminal comprises a photographing device, and the controlling photographing terminal takes a photograph with the second focal length under each photographing pose of the photographing pose sequence information. The second image corresponding to the area to be photographed includes:

Through the posture adjustment device, the camera is controlled to be in each shooting pose in the shooting pose sequence information in sequence, and the camera is controlled to shoot with the second focal length in each of the shooting poses. The second image corresponding to the area to be photographed.
The device according to claim 61, wherein the posture adjustment device is arranged on a movable platform.
The device according to claim 60, wherein the shooting end is controlled to shoot the second focal length corresponding to the area to be photographed under each shooting pose of the shooting pose sequence information. After the second image, when the computer program is executed by the processor, it is also used to realize:

The first image and the second image are stored in association.
The device according to claim 63, wherein said storing said first image and said second image in association comprises:

Determining a first mapping relationship between the second image and the area to be processed, and determining a second mapping relationship between the area to be processed and the first image; and

The first image, the region to be processed, the second image, the first mapping relationship, and the second mapping relationship are stored.
The device according to claim 63, wherein when applied to a shooting end, the second image is an unprocessed shooting image.
The device according to claim 63, wherein the second image is a processed captured image, and the resolution of the second image is smaller than that of an unprocessed captured image. Rate.
The device according to claim 66, wherein the computer program, when being executed by the processor, is further used to implement:

If an operation instruction for viewing the second image is received, obtaining an unprocessed photographed image corresponding to the second image from the photographing terminal;

The unprocessed captured image corresponding to the second image is displayed.
The device according to claim 60, characterized in that, in each shooting pose of the shooting pose sequence at the control shooting end, the second focal length corresponding to the area to be photographed is shot at the second focal length. After the image, when the computer program is executed by the processor, it is also used to realize:

Acquiring a third image, the third image being synthesized based on the second image; and

The first image and the third image are stored in association.
The device according to claim 38, wherein the first image and the second image are taken by a zoom camera at the first focal length and the second focal length, respectively.
The device according to claim 38, wherein the first image is taken by a first photographing device, and the second image is taken by a second photographing device; wherein the focal length of the first photographing device is at least The focal length of part is different from that of the second camera.
The device according to claim 70, wherein the focal length of at least one of the first photographing device and the second photographing device is adjustable.
The device of claim 70, wherein the first photographing device is a wide-angle camera, and the second photographing device is a telephoto camera.
The device according to claim 38, wherein the second focal length comprises any one of the following: a fixed focal length, a preset focal length, a focal length determined based on a focal length algorithm, a focal length determined based on a sensor, and a focal length selected by a user.
The device according to claim 73, wherein the focal length determined based on the sensor comprises: a focal length determined based on distance information, and the distance information is determined by a laser rangefinder.
An image processing system, characterized in that the image processing system includes:

A photographing device and a control terminal communicatively connected with the photographing device;

The control device is used to obtain the area to be processed;

The photographing device is used to obtain a first maximum viewing angle area corresponding to the first focal length of the photographing device under the current attitude angle, and obtaining a second maximum viewing angle area corresponding to the second focal length; A maximum viewing angle area and the second maximum viewing angle area are compared; and the selectable range of the to-be-processed area in the first image is determined according to the comparison result, so as to control the photographing device to obtain that the to-be-processed area is in the second At least one second image at a focal length, the first image being acquired at the first focal length; wherein the second focal length is greater than the first focal length.
The system according to claim 75, wherein the second maximum viewing angle area corresponding to the second focal length is determined based on the maximum rotatable angle of the posture adjustment device.
The system according to claim 75, wherein the photographing device is specifically configured to determine that the area to be processed is in the first image if it is determined according to the comparison result that the second maximum viewing angle area includes the first maximum viewing angle area. The selectable range in is any area in the first image.
The system according to claim 75, wherein the photographing device is specifically configured to determine that the area to be processed is in the first area if it is determined according to the comparison result that the second maximum angle of view area includes part of the first area of maximum angle of view. The selectable range in the image is the overlapping area of the first maximum viewing angle area and the second maximum viewing angle area.
The system according to claim 75, wherein the photographing device is specifically used when the current attitude angle is the current pitch angle, if it is determined according to the comparison result that the first upper limit of the first maximum viewing angle area exceeds the The second upper limit of the second maximum viewing angle area is determined based on the first upper limit, the second upper limit, and the vertical viewing angle area corresponding to the first focal length to determine that the selectable range is within the first maximum viewing angle The optional upper limit of the area.
The system according to claim 75, wherein the photographing device is specifically used when the current attitude angle is the current pitch angle, if it is determined according to the comparison result that the second lower limit of the second maximum viewing angle area exceeds the The first lower limit of the first maximum viewing angle area is determined based on the second lower limit, the first lower limit, and the vertical viewing angle area corresponding to the first focal length. The optional lower limit of the viewing angle area.
The system according to claim 75, wherein the photographing device is specifically used when the current attitude angle is the current yaw angle, if it is determined according to the comparison result that the first left limit of the first maximum viewing angle area exceeds The second left limit of the second maximum viewing angle area is determined based on the first left limit, the second left limit, and the horizontal viewing angle area corresponding to the first focal length. The optional left limit of the first maximum viewing angle area.
The system according to claim 75, wherein the photographing device is specifically used when the current attitude angle is the current yaw angle, if it is determined according to the comparison result that the second right limit of the second maximum viewing angle area exceeds The first right limit of the first maximum viewing angle area is determined based on the second right limit, the first right limit, and the horizontal viewing angle area corresponding to the first focal length. The optional right limit of the first maximum viewing angle area.
The system according to claim 75, wherein the control terminal comprises a display interface, and the control terminal is further configured to display at least one of the multiple boundaries of the selectable range through the display interface.
The system according to claim 75, wherein the photographing device is further configured to determine the area to be processed after the selectable range of the area to be processed in the first image is determined according to the comparison result, Wherein, the second maximum viewing angle area includes the to-be-processed area; and determining photographing related information based on the second focal length and the to-be-processed area, so as to control the photographing device to obtain the to-be-processed area based on the photographing related information At least one second image of the processing area at the second focal length.
The system according to claim 84, wherein the photographing device is further configured to obtain a candidate to-be-processed area before the determination of the to-be-processed area; Beyond the selectable range, it is determined that the candidate to-be-processed area is the to-be-processed area.
The system according to claim 85, wherein the photographing device is further configured to output prompt information if it is determined that the candidate to-be-processed area exceeds the selectable range after the acquisition of the candidate to-be-processed area.
The system according to claim 84, wherein the photographing device is specifically configured to determine the area to be processed based on an image recognition algorithm.
The system according to claim 87, wherein the photographing device is specifically configured to perform image recognition on the first image to determine the object to be processed; and use the area where the object to be processed is located as the area to be processed .
The system according to claim 87, wherein the photographing device is specifically configured to determine the area to be processed from the first image based on a preset task and an image recognition algorithm.
The system according to claim 84, wherein the photographing related information comprises: photographing pose sequence information.
The system according to claim 90, wherein the photographing pose sequence information is determined based on photographing combination information, and the photographing combination information is determined based on the second focal length and the area to be processed.
The system according to claim 91, wherein the photographing combination information includes the number of photographs and the area to be photographed for each photograph;

The photographing device is specifically configured to determine the number of photographs and the area to be photographed for each photograph based on the second focal length and the area to be processed; and determine the photographing pose sequence information based on the area to be photographed; wherein, Each shooting pose in the shooting pose sequence information corresponds to one of the to-be-photographed areas, so that the photographing device can shoot at the second focal length under each shooting pose in the shooting pose sequence information. The second image corresponding to the area to be photographed.
The system according to claim 92, wherein the photographing device is specifically configured to determine a viewing angle area corresponding to the second focal length based on the size of the image sensor and the second focal length; and based on the to-be-processed area and The viewing angle area corresponding to the second focal length determines the number of photographs and the area to be photographed for each photograph; wherein the area formed by a plurality of the areas to be photographed includes the area to be processed.
The system according to claim 93, wherein the photographing device is specifically configured to determine the image overlap ratio between the area to be processed, the viewing angle area corresponding to the second focal length, and the area to be photographed The number of photos taken, and the area to be taken each time.
The system according to claim 92, wherein the photographing device is specifically configured to determine, based on the current pose of the camera at the first focal length and the angular position deviation of the second focal length, which corresponds to the area to be photographed each time a photograph is taken. The shooting pose, and the set of all the determined shooting poses is used as the shooting pose sequence information.
The system according to claim 95, wherein the photographing device is specifically configured to determine the distance between the designated position of the viewing angle area corresponding to the second focal length and the designated position of the viewing angle area corresponding to the first focal length. And determine the second deviation between the designated position of the area to be processed and the designated position of the viewing angle area corresponding to the first focal length; determine the designated position of the area to be photographed and the area to be processed The third deviation between the designated positions of the area; and the determination based on the first deviation, the second deviation, and the third deviation between the designated positions of the viewing angle area corresponding to the first focal length and the to-be-photographed area The angular position deviation of the second focal length between.
The system according to claim 92, wherein the photographing device is specifically configured to photograph at the second focal length corresponding to the area to be photographed under each photographing pose of the photographing pose sequence information. The second image.
The system according to claim 97, wherein the photographing device comprises a posture adjusting device and a photographing device, and the photographing device is specifically configured to control the photographing device to sequentially be in the photographing position and posture through the posture adjusting device. Each shooting pose in the sequence information is controlled, and the shooting device is controlled to shoot a second image corresponding to the area to be photographed at the second focal length in each of the shooting poses.
The system according to claim 98, wherein the posture adjustment device is arranged on a movable platform.
The system according to claim 97, wherein the photographing device is further configured to photograph at the second focal length corresponding to the area to be photographed under each photographing pose of the photographing pose sequence information. After the second image, the first image and the second image are stored in association.
The system according to claim 100, wherein the photographing device is specifically configured to determine a first mapping relationship between the second image and the area to be processed, and to determine the relationship between the area to be processed and the area to be processed. A second mapping relationship between the first images; and storing the first image, the area to be processed, the second image, the first mapping relationship, and the second mapping relationship.
The system of claim 100, wherein the second image is an unprocessed captured image and is stored in the photographing device.
The system according to claim 100, wherein the second image is a processed captured image and stored in the control terminal, and the resolution of the second image is smaller than that of an unprocessed captured image. Rate.
The system according to claim 103, wherein the control terminal further comprises a display interface, and the control terminal is further configured to: Acquiring an unprocessed captured image corresponding to the second image; and displaying the unprocessed captured image corresponding to the second image through the display interface.
The system according to claim 97, wherein the photographing device is further configured to shoot at the second focal length at each photographing position of the photographing position sequence at the control photographing terminal. After the second image corresponding to the area to be photographed, a third image is obtained, the third image is synthesized based on the second image; and the first image and the third image are stored in association.
The system according to claim 75, wherein the camera device comprises a zoom camera device, the zoom camera device is used to capture the first image and the camera at the first focal length and the second focal length, respectively. The second image.
The system according to claim 75, wherein the photographing device comprises a first photographing device and a second photographing device;

The first photographing device is used to photograph a first image;

The second photographing device is used to photograph a second image;

Wherein, the focal length of the first photographing device is at least partially different from the focal length of the second photographing device.
The system according to claim 107, wherein the focal length of at least one of the first camera and the second camera is adjustable.
The system of claim 107, wherein the first photographing device is a wide-angle camera, and the second photographing device is a telephoto camera.
The system of claim 75, wherein the second focal length comprises any one of the following: a fixed focal length, a preset focal length, a focal length determined based on a focal length algorithm, a focal length determined based on a sensor, and a focal length selected by a user.
The system according to claim 110, wherein the photographing device comprises a laser rangefinder, and the laser rangefinder is used to measure distance information to determine the second focal length based on the distance information.
A computer-readable storage medium, characterized in that it stores executable instructions that, when executed by one or more processors, can cause the one or more processors to execute as claimed in claim 1. The method of any one of claims to 37.