WO2024134644A1

WO2024134644A1 - Scene acquisition from an aircraft using a limited field of view camera operable in a dual mode

Info

Publication number: WO2024134644A1
Application number: PCT/IL2023/051253
Authority: WO
Inventors: Ohad ROZENBERG
Original assignee: Israel Aerospace Industries Ltd.
Priority date: 2022-12-20
Filing date: 2023-12-07
Publication date: 2024-06-27

Abstract

There are provided systems and methods comprising, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of its line of sight, while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern, operating the image acquisition device according to a first acquisition mode, said operating comprising controlling the device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the scene is acquired with a field of view which is smaller than the whole scene at the desired image resolution.

Description

SCENE ACQUISITION FROM AN AIRCRAFT USING A LIMITED FIEED OF VIEW CAMERA OPERABEE IN A DUAL MODE

TECHNOLOGICAL FIELD

The invention is in the field of image acquisition.

BACKGROUND

References considered to be relevant as background to the presently disclosed subject matter are listed below (acknowledgement of the references herein is not to be inferred as meaning that these are in any way relevant to the patentability of the presently disclosed subject matter):

- US 2019/094344;

- DE 19714539;

- WO 2020/092179;

- US 9,641,810; and

- US 2021/173397.

There is now a need to propose new systems and methods of acquisition of a scene from an aerial vehicle.

GENERAL DESCRIPTION

In accordance with certain aspects of the presently disclosed subject matter, there is provided a computer-implemented method comprising, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device: while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operating the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xix) below, in any technically possible combination or permutation: i. the method comprises, for a required motion pattern of the line of sight of the image acquisition device with respect to the scene, selecting the first motion pattern of the image acquisition device, such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by a motion of the aerial vehicle complies with said required motion pattern; ii. the method comprises compensating the motion of the aerial vehicle with the device, to enable a motion of the line of sight of the image acquisition device to comply with the required motion pattern with respect to the scene, despite the motion of the aerial vehicle; iii. a first fraction of the different fractions of the scene acquired by the image acquisition device corresponds to a central area of the scene; iv. in the first acquisition mode, the image acquisition device is moved according to the first motion pattern to acquire a first set of fractions of the scene, until a coverage of the scene by the image acquisition device meets a completion criterion, and once the completion criterion is met, the image acquisition device is moved by the device to acquire a second set of fractions of the scene, wherein the first set of fractions of the scene and the second set of fractions of the scene overlap or are identical; v. the image acquisition device is moved according to the first motion pattern to acquire the second set of fractions of the scene; vi. the completion criterion is met when at least one of (a) or (b) is met: (a) the first set of fractions of the scene acquired by the image acquisition device cover a required area of the scene; (b) a duration during which the first set of fractions of the scene have been acquired is above a threshold; vii. the method comprises, in the first acquisition mode, controlling the device to make the image acquisition device perform a plurality of cycles of acquisition, wherein, at each cycle, most or all of the scene is acquired, wherein a fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle is the same for the plurality of cycles; viii. the fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle corresponds to a central area of the scene; ix. the method comprises controlling the device to make the line of sight of the image acquisition device follow a same pattern of acquisition with respect to the scene for the plurality of cycles; x. the method comprises selecting the first motion pattern of the image acquisition device such that a motion of the line of sight on the scene is a spiral or a square spiral; xi. responsive to detecting that a first criterion is met, switching operation of the image acquisition device from the first acquisition mode to a second acquisition mode, different from the first acquisition mode; xii. the first criterion is informative of a target present in the scene, wherein, in the second acquisition mode, the image acquisition device is controlled to acquire one or more images of the target; xiii. the method comprises, in the first acquisition mode, detecting, in at least one fraction of the scene acquired by the image acquisition device, presence of a target that meets a certain condition, responsive to said detecting or to a command of an operator following said detecting, switching operation of the image acquisition device from the first acquisition mode to a second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target; xiv. in the second acquisition mode, the device is controlled to make the line of sight of the image acquisition device track the target; xv. responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, the method comprises switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode; xvi. the method comprises, responsive to detecting that a second criterion is met, switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode, wherein the second criterion is informative of a target or is informative of a command an operator; xvii. in said second acquisition mode, the aerial vehicle is flying over the scene along a flight path corresponding to a hold pattern; xviii. the method comprises obtaining data informative of a flight path

of the aerial vehicle, obtaining data D_requ ed_LOS_patteni informative of a required motion pattern of the line of sight on the scene, using data Dflight__Path and data Drequiied_LOS_pattem to determine the first motion pattern, such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by the flight path of the aerial vehicle, complies with the required motion pattern, and controlling the device coupled to the image acquisition device according to said first motion pattern; and xix. the aerial vehicle cannot hover.

In accordance with other aspects of the presently disclosed subject matter, there is provided a computer-implemented method comprising, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device, operating the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern, enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode. According to some embodiments, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of the features (i) to (xix) above, in any technically possible combination or permutation.

In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (xx) to (xxvi) below, in any technically possible combination or permutation: xx. the first criterion is informative of a target present in the scene, wherein in the second acquisition mode, the image acquisition device is controlled to acquire one or more images of the target; xxi. detecting that the first criterion is met comprises detecting, in at least one fraction of the scene acquired by the image acquisition device in the first acquisition mode, presence of a target that meets a certain condition, responsive to said detecting of the target or to a command of an operator following said detecting, switching operation of the image acquisition device from the first acquisition mode to the second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target; xxii. detecting the presence of the target comprises using at least one of a Video Motion Detection (VMD) method, or an Automatic Change Detection (ACD) method, or an Automatic Target Recognition (ATR) method; xxiii. said first criterion is met when a command to acquire one or more images of a target of the scene is received, wherein in the second acquisition mode, the device is controlled to make the image acquisition device track the target; xxiv. responsive to detecting that a second criterion is met, the method comprises switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode, wherein the second criterion is informative of a target or is informative of a command an operator; xxv. the method comprises switching operation of the image acquisition device from the first acquisition mode to the second acquisition mode in response to detection of a target in one or more images acquired by the image acquisition device or to a command of an operator following said detection, and switching operation of the image acquisition device from the second acquisition mode to the first acquisition mode in response to a detection of an absence of the target in one or more images acquired by the image acquisition device or to a command of an operator following said detection; and xxvi. for a given duration, a total area of the scene acquired in the second acquisition mode is smaller than in the first acquisition mode.

In accordance with other aspects of the presently disclosed subject matter, there is provided a system adapted to be mounted on an aerial vehicle, the system comprising an image acquisition device, and a processor and memory circuitry (PMC) operatively coupled to a device enabling a modification of an orientation of the image acquisition device, wherein the PMC is configured to, while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operate the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

In addition to the above features, the system according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (xxvii) to (xxxviii) below, in any technically possible combination or permutation: xxvii. the PMC is configured to, for a required motion pattern of the line of sight of the image acquisition device with respect to the scene, select the first motion pattern of the image acquisition device such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by a motion of the aerial vehicle complies with said required motion pattern; xxviii. a first fraction of the different fractions of the scene acquired by the image acquisition device corresponds to a central area of the scene; xxix. the system is configured to, in the first acquisition mode, control the device to make the image acquisition device perform a plurality of cycles of acquisition, wherein, at each cycle, most or all of the scene is acquired, wherein a fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle is the same for the plurality of cycles; xxx. the system is configured to control the device such that the fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle corresponds to a central area of the scene; xxxi. the system is configured to control the device to make the line of sight of the image acquisition device follow a same pattern of acquisition with respect to the scene for the plurality of cycles; xxxii. in the first acquisition mode, the system is configured to trigger a motion of the image acquisition device according to the first motion pattern to acquire a first set of fractions of the scene, until a coverage of the scene by the image acquisition device meets a completion criterion, wherein a first fraction of the scene acquired by the image acquisition device in the first set corresponds to a given fraction of the scene, once the completion criterion is met, trigger a motion of the image acquisition device by the device to acquire a second set of fractions of the scene, wherein a first fraction of the scene acquired by the image acquisition device in the second set also corresponds to said given fraction of the scene; xxxiii. the completion criterion is met when at least one of (a) or (b) is met: (a) the first set of fractions of the scene acquired by the image acquisition device cover a required area of the scene; (b) a duration during which the first set of fractions of the scene have been acquired is above a threshold; xxxiv. the system is configured to select the first motion pattern of the image acquisition device, such that a motion of the line of sight on the scene is a spiral or a square spiral; xxxv. the system is configured to, responsive to detecting that a first criterion is met, switch operation of the image acquisition device from the first acquisition mode to a second acquisition mode, different from the first acquisition mode, wherein the first criterion is informative of a target present in the scene; xxxvi. the system is configured to, in the first acquisition mode, detect, in at least one fraction of the scene acquired by the image acquisition device, presence of a target that meets a certain condition, responsive to said detecting or to a command of an operator following said detecting, switch operation of the image acquisition device from the first acquisition mode to a second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target; xxxvii. said first criterion is met when a command of an operator to acquire one or more images of a target of the scene is received, wherein, in the second acquisition mode, the device is controlled to make the image acquisition device track the target; and xxxviii. in the second acquisition mode, the system is configured to perform (a) or (b): (a) controlling the device to make the line of sight of the image acquisition device track the target; (b) controlling the device to make the line of sight of the image acquisition device track the target, and, responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode.

According to some embodiments, the system according to this aspect of the presently disclosed subject matter can optionally include one or more of the features (i) to (xix) described with respect to the method above, and/or one or more the features (xx) to (xxvi) described with respect to the method above, in any technically possible combination or permutation.

In accordance with other aspects of the presently disclosed subject matter, there is provided a system adapted to be mounted on an aerial vehicle, the system comprising an image acquisition device, and a processor and memory circuitry (PMC) operatively coupled to a device enabling a modification of an orientation of the image acquisition device, wherein the PMC is configured to control the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, and responsive to detecting that a first criterion is met, operate the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

In addition to the above features, the system according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (xxxix) to (xliii) below, in any technically possible combination or permutation: xxxix. detecting that the first criterion is met comprises detecting, in at least one fraction of the scene acquired by the image acquisition device in the first acquisition mode, presence of a target that meets a certain condition wherein the system is configured to, in response to said detecting of the target, or to command of an operator following said detecting, switch operation of the image acquisition device from the first acquisition mode to the second acquisition mode, wherein, in the second acquisition mode, the system is configured to operate the image acquisition device to acquire one or more images of the target; xl. the first criterion is informative of a presence of a target in the scene, wherein, in the second acquisition mode, the PMC is configured to control the device to make the image acquisition device acquire one or more images of the target; xli. in the second acquisition mode, the system is configured to control the device to make the line of sight of the image acquisition device track the target; xlii. responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, the system is configured to switch back operation of the image acquisition device from said second acquisition mode to said first acquisition mode; and xliii. responsive to detecting that a second criterion is met, the system is configured to switch back operation of the image acquisition device from said second acquisition mode to said first acquisition mode.

According to some embodiments, the system according to this aspect of the presently disclosed subject matter can optionally include (or execute/perform) one or more of the features (i) to (xxvi) described with respect to the methods above, and/or one or more of the features (xxvii) to (xxxviii), in any technically possible combination or permutation.

In accordance with other aspects of the presently disclosed subject matter, there is provided an aerial vehicle comprising an image acquisition device, a device enabling a modification of an orientation of the image acquisition device, and a processor and memory circuitry (PMC) configured to, while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operate the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

According to some embodiments, the system according to this aspect of the presently disclosed subject matter can optionally include (or execute/perform) one or more of the features (i) to (xliii), in any technically possible combination or permutation.

In accordance with other aspects of the presently disclosed subject matter, there is provided an aerial vehicle comprising an image acquisition device, a device enabling a modification of an orientation of the image acquisition device, and a processor and memory circuitry (PMC) configured to operate the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

According to some embodiments, the system according to this aspect of the presently disclosed subject matter can optionally include (or execute/perform) one or more of the features (i) to (xliii) in any technically possible combination or permutation.

In accordance with other aspects of the presently disclosed subject matter, there is provided a non-transitory computer readable medium comprising instructions that, when executed by at least one processing circuitry, cause the at least one processing circuitry to perform, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device: while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operating the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

In addition to the above features, the instructions are optionally executable by the processing circuitry to include or to perform one or more of the features (i) to (xxvi) above, in any technically possible combination or permutation:

In accordance with other aspects of the presently disclosed subject matter, there is provided a non-transitory computer readable medium comprising instructions that, when executed by at least one processing circuitry, cause the at least one processing circuitry to perform, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device, operating the image acquisition device according to a first acquisition mode, said operating comprising controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern, enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

According to some embodiments, the proposed solution is able to acquire images of a scene using a simple and low-cost image acquisition device.

According to some embodiments, the proposed solution is able to switch (back and forth) between a first acquisition mode enabling a coverage of a scene and a second acquisition mode enabling tracking of a target in a quick and robust manner.

According to some embodiments, the proposed solution is able to switch (back and forth) between a first acquisition mode enabling coverage of a scene and a second acquisition mode enabling tracking of a target with a single image acquisition device.

According to some embodiments, the proposed solution enables different acquisition modes using a single image acquisition device.

According to some embodiments, the proposed solution reduces the data to be transmitted in order to execute different acquisition modes (since a single image acquisition device can be used to perform the different acquisition modes, it is possible to send commands only to this single image acquisition device, or to a device coupled to this single image acquisition device).

According to some embodiments, the proposed solution reduces the weight of the payload embedded on the aircraft.

According to some embodiments, the proposed solution does not require a complex modification of the equipment embedded on the aircraft. According to some embodiments, the proposed solution improves tracking of a target by an image acquisition device, in particular in a situation in which the target is lost.

According to some embodiments, the proposed solution enables, with a single acquisition device, scanning of a large scene and acquiring limited areas of the scene (such as tracking of a target).

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand the subject matter that is disclosed herein and to exemplify how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:

- Fig. 1 illustrates an architecture of an aerial vehicle which includes a system according to some embodiments of the invention;

Fig. 2A illustrates an aerial vehicle flying over a scene along a flight path corresponding to a (non-limitative) hold pattern;

Fig. 2B illustrates, in the example of Fig. 2A, an acquisition of fractions of the scene, according to a first acquisition mode;

Fig. 2C illustrates an embodiment of a method of acquiring different fractions of a scene, which cover most or all of the scene, according to a first acquisition mode;

Fig. 3A illustrates an embodiment of a method of acquiring, a plurality of times, most or all of a scene, according to a first acquisition mode;

- Fig. 3B illustrates an embodiment of a method of performing a plurality of cycles of acquisition of a scene, wherein the cycles are identical or begin with the same fraction of the scene;

Fig. 3C illustrates a non-limitative example of the method of Fig. 3B;

Fig. 4A illustrates an embodiment of a method of switching operation of the image acquisition device from a first acquisition mode to a second acquisition mode;

Fig. 4B illustrates an embodiment of a method of switching operation of the image acquisition device from a second acquisition mode to a first acquisition mode;

Fig. 5A illustrates an embodiment of a method of switching operation of the image acquisition device from a first acquisition mode to a second acquisition mode in which a target is tracked, in response to a detection of the target in the first acquisition mode;

Fig. 5B illustrates an example of the method of Fig. 5A; and

Fig. 5C illustrates an embodiment of a method of switching operation of the image acquisition device from a first acquisition mode to a second acquisition mode, and, conversely, in response to a detection of a target or a loss of the target during tracking of the target.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods have not been described in detail so as not to obscure the presently disclosed subject matter.

The term "processor and memory circuitry" (PMC) as disclosed herein should be broadly construed to include any kind of electronic device with data processing circuitry, which includes for example a computer processing device operatively connected to a computer memory (e.g., digital signal processor (DSP), a microcontroller, a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC), etc.) capable of executing various data processing operations.

It can encompass a single processor or multiple processors, which may be located in the same geographical zone or may, at least partially, be located in different zones and may be able to communicate together.

It is to be noted that while the present disclosure refers to a processor and memory circuitry 130 being configured to perform various functionalities and/or operations, the functionalities/operations can be performed by the one or more processors of the processor and memory circuitry 130 in various ways. By way of example, the operations described hereinafter can be performed by a specific processor, or by a combination of processors. The operations described hereinafter can thus be performed by respective processors (or processor combinations) in the processor and memory circuitry 130, while, optionally, at least some of these operations may be performed by the same processor. The present disclosure should not be limited to be construed as one single processor always performing all the operations.

Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as “controlling”, “acquiring”, “determining”, “operating”, “switching”, “obtaining”, or the like, refer to the action(s) and/or process(es) of a processor and memory circuitry that manipulates and/or transforms data into other data, said data represented as physical, such as electronic, quantities and/or said data representing the physical objects.

Embodiments of the presently disclosed subject matter are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the presently disclosed subject matter as described herein.

The invention contemplates a computer program being readable by a computer for executing one or more methods of the invention. The invention further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing one or more methods of the invention.

Attention is drawn to Fig. 1.

An aerial vehicle 100 includes an image acquisition device 110 (a camera, such as an observation camera) usable to perform image acquisition from the aerial vehicle 100. The image acquisition device 110 can be, for example, mounted on a wing or on the fuselage of the aerial vehicle 100. This is not limitative.

In some embodiments, the aerial vehicle 100 includes a single image acquisition device 110.

In some embodiments, the aerial vehicle 100 includes a plurality of image acquisition devices 110.

As explained hereinafter, the image acquisition device 110 can be a low-cost camera, with a limited field of view.

The aerial vehicle 100 corresponds e.g., to an aircraft. According to some embodiments, the aerial vehicle 100 is an aerial vehicle which cannot hover (a civil aircraft cannot hover, whereas drones, helicopters, and vertical take-off and landing aircraft are aerial vehicles which can hover). This corresponds in particular to a fixed- wing aerial vehicle. The aerial vehicle 100 further includes a device 120 (also called pointing device 120). The image acquisition device 110 is operatively coupled to the device 120. For example, the image acquisition device 110 is mounted on the device 120. The device 120 enables modifying an orientation of the line of sight of the image acquisition device 110. In particular, it enables moving the image acquisition device 110 along at least two degrees of freedom. The device 120 includes for example a pivoted support that permits rotation of the image acquisition 110 device object about at least two axes, or more.

The device 120 includes e.g., a gimbal or at least two gimbals.

The aerial vehicle 100 further includes a processor and memory circuitry (PMC) 130. The PMC 130 is operable to send one or more command(s) to the device 120, in order to modify the orientation of the line of sight of the image acquisition device 110 over time.

According to some embodiments, the PMC 130 is operatively coupled to the image acquisition device 110. In particular, the PMC 130 can control operation of the image acquisition device 110.

According to some embodiments, the PMC 130 can be a dedicated PMC used to control the device 120 and/or the image acquisition device 110.

According to some embodiments, the PMC 130 corresponds to an existing PMC of the aerial vehicle 100 which executes instructions enabling control of the device 120 and/or of the image acquisition device 110.

According to some embodiments, the aerial vehicle 100 comprises a receiver/transmitter (transceiver) enabling data communication with a central station 150. The central station 150 can be e.g., a ground station. An operator located at the central station 150 can therefore send commands to control the pointing device 120 and/or the image acquisition device 110.

In some embodiments, the central station 150 can also include a processor and memory circuitry which generates commands for the pointing device 120 and/or the image acquisition device 110. This enables remote automatic control of the pointing device 120 and/or the image acquisition device 110.

According to some embodiments, the aerial vehicle 100 includes a payload 105 which is operative to perform a physical interaction (e.g., destructive interaction) with a target. For example, payload 105 includes a laser operative to remove material from the target, a device enabling launch of a projectile to annihilate the target, etc. Attention is now drawn to Fig. 2A.

Assume that the aerial vehicle 100 is flying over a scene 200 along a flight path corresponding to a hold pattern 205. The hold pattern enables the aerial vehicle 100 to be maintained substantially over the same scene during a given period of time.

Examples of hold patterns include e.g., circles, ellipses, squares, etc. These examples are not limitative.

A hold pattern is generally a pattern which repeats itself over time. Note that the pattern which is repeated over time is not necessarily exactly the same, but is generally substantially the same over a plurality of the repetitions. For example, a circle (hold pattern) can be repeated, but the geometry (e.g., radius) of the circle can change between two repetitions of the hold pattern (provided that the aircraft remains over the same scene during these repetitions).

As explained hereinafter, the image acquisition device 110 can be operated in at least two acquisition modes: a first acquisition mode, in which most or all of the scene is acquired, and a second acquisition mode, in which a target of the scene (e.g., only a limited part of the scene) is acquired and/or tracked.

Assume that the aerial vehicle 100 is flying over a scene 200 along a flight path corresponding to a hold pattern 205, during a given period of time.

In the first acquisition mode, the pointing device 120 is controlled (using commands generated by the PMC 130) to point the line of sight of the image acquisition device 110 towards different fractions of the scene at different instants of the given period of time (see operation 280 in Fig. 2C). In some embodiments, the different fractions do not overlap. In other embodiments, the different fraction can partially overlap. The motion of the image acquisition device 110 induced by the pointing device 120 follows a first motion pattern (this first motion pattern can be expressed e.g., as a motion relative to the aerial vehicle). This first motion pattern enables the line of sight to cover most or all of the scene.

This first motion pattern is selected such that the line of sight follows a required motion pattern on the scene (on the ground).

Since the image acquisition device 110 is mounted on an aerial vehicle 100 which is in motion, the motion of the aerial vehicle 100 needs to be compensated.

In particular, the first motion pattern of the image acquisition device 110 is selected, such that a motion of the line of sight with respect to the scene caused by both the first motion pattern and the motion of the aerial vehicle, complies with the required motion pattern.

In some examples, the motion pattern of the line on sight on the scene (on the ground) corresponds to a spiral. In some examples, the motion pattern of the line of sight on the scene (on the ground) corresponds to a square spiral. This is however not limitative.

By virtue of the motion of its line of sight with respect to the scene, the image acquisition device 110 acquires each of the different fractions of the scene (operation 285). The different fractions can cover most or all of the scene.

Note that the acquisition of most or all of the scene is mainly caused by the motion of the image acquisition device 110 induced on purpose by the pointing device 120. As mentioned above, the motion of the aerial vehicle 100 is compensated by the motion of the pointing device 120, to ensure that the line of sight follows the required pattern on the ground, such as (but not limited to) a spiral.

Assume for example that the required pattern of the line of sight on the ground corresponds to a given pattern. The PMC 130 receives data informative of the flight path (e.g., planned flight path and/or current flight path) of the aerial vehicle, and the required pattern for the line of sight on ground. The PMC 130 determines the actual motion pattern of the line of sight of the image acquisition device which is required to obtain the required pattern on ground by compensating the motion of the aerial vehicle 100. This enables sending a command to the pointing device 120 which matches this actual motion pattern.

Assume that the scene has to be acquired with a desired resolution. An example of resolution is 40 centimeters per pixel (this is not limitative). Each fraction of the scene is acquired by the image acquisition device 110 with a field of view (maximal field of view - see 250 in Fig. 2B) which is smaller than the dimensions of the whole scene at the desired image resolution. In other words, the field of view of the image acquisition device 110 enables acquiring only a limited fraction of the scene, and not the whole scene, at the desired resolution.

According to some embodiments, the field of view of the image acquisition device 110 in the first acquisition mode is smaller than 25 degrees, usually much less (any value below 25 degrees), in accordance with the requested resolution. These values are not limitative.

In particular, in some embodiments, the image acquisition device 110 is not a WAMI (Wide-Area Motion Imagery) sensor, which acquires images of the entirety of its coverage area all at once, in real time, but rather an image acquisition device 110 which acquires each time only a fraction/subset of its coverage area (thereby being less complex and less costly).

Fig. 2B illustrates an example in which the first motion pattern of the image acquisition device enables the line of sight of the image acquisition device to follow a spiral (square spiral) covering most or all of the scene. A first fraction 230 of the scene is acquired, then a second fraction 231, a third fraction 232, a fourth fraction 233, a fifth fraction 253, etc.

As mentioned above, the image acquisition device 110 acquires, successively, different fractions of the scene. According to some embodiments, the first fraction of the scene acquired by the image acquisition device 110 is a central area of the scene. This is the case in Fig. 2B, in which the first fraction 230 corresponds to a central area of the scene. In general, the area of the scene which contains the largest amount of information corresponds to the central area of the scene. This is however not limitative.

Attention is now drawn to Fig. 3A.

In some embodiments, the method of Fig. 2C can be repeated over time, to acquire most or all of the scene a plurality of times. This is illustrated in Fig. 3A.

As mentioned above with reference to Fig. 2C, the image acquisition device 110 is moved according to a first motion pattern, enabling its line of sight to be pointed over time towards different fractions (first set of fractions) of the scene. This enables the image acquisition device 110 to acquire the first set of fractions at the desired resolution (operations 280 and 285). Once a coverage of the scene meets a completion criterion (see reference 300), the method can be repeated (see reference 310).

The completion criterion can define e.g., the percentage of the scene that has to be covered and/or can define a minimal period of time during which the scene has to be scanned by the image acquisition device 110. The completion criterion is therefore met when the different fractions of the scene acquired by the image acquisition device cover a required area of the scene and/or when the duration in which the different fractions of the scene have been acquired is above a threshold.

Once the completion criterion is met, the image acquisition device 110 is operated to acquire again different fractions (second set of fractions) of the scene.

In some embodiments, when acquisition of the scene is repeated (see reference 310), the image acquisition device 110 is operated to acquire again different fractions of the scene according to the same pattern (of the line of sight) on ground as the previous pattern (of the line of sight) on ground used at the previous iteration.

For example, assume that most or all of the scene has been acquired using a given pattern on ground (e.g., a spiral). This has been obtained by controlling the motion of the image acquisition device according to a first motion pattern which compensates the motion of the aerial vehicle and enables the line of sight to follow the given pattern on ground.

Once the completion criterion has been met, the image acquisition device 110 can be moved again according to the first motion pattern, in order to move the line of sight on ground according to the given pattern, thereby acquiring (again) different fractions of the scene covering most or all of the scene.

The fractions of the scene which are acquired at the second iteration of the method (designated as second set of fractions of the scene) can overlap with or be the same as the fractions of the scene which have been acquired at the first iteration of the method (designated as first set of fractions of the scene).

More generally, the fractions of the scene which are acquired at the N^th iteration of the method can overlap with or be the same as the fractions of the scene which have been acquired at the N-l^th iteration (previous iteration) of the method.

This is not limitative and, in some embodiments, the fractions of the scene which are acquired at the second iteration of the method (respectively at the N^th iteration of the method) can be different from the fractions of the scene which have been acquired in the first iteration of the method (respectively at the N-l^th iteration of the method).

Note that the method of Fig. 3A, in which most or all of the scene is acquired a plurality of times, can be performed while the aerial vehicle is flying over the scene along a flight path corresponding to a hold pattern.

Assume that at the first iteration of the method (or, more generally, at an iteration number “i”), the first fraction of the scene acquired by the image acquisition device 110 corresponds to a given fraction of the scene (for example, a central area of the scene). In some embodiments, at the second iteration of the method (or at an iteration number “i+1”), the first fraction of the scene acquired by the image acquisition device 110 corresponds to the same given fraction of the scene (for example, the central area of the scene). This enables obtaining more data on the given fraction of the scene (e.g., the central area), since, at each iteration, this given fraction is acquired. To the contrary, for some fractions of the scene which are located near the boundaries of the scene, it can occur that the completion criterion is met before the image acquisition device reaches these boundaries. Therefore, these fractions of the scene are not necessarily acquired at each iteration of the method.

Attention is now drawn to Figs. 3B and 3C.

According to some embodiments, in the first acquisition mode, the image acquisition device 110 is controlled (using the pointing device 120) to repeatedly acquire most (e.g., the majority) or all of the scene, according to a plurality of cycles of acquisition.

At each cycle, most or all of the scene is acquired by the image acquisition device 120, by moving the line of sight of the image acquisition device 110 with respect to the scene.

At each cycle, the image acquisition device 110 can be controlled to acquire first the same fraction of the scene. In other words, at the beginning of each cycle, the same fraction of the scene is acquired (see operations 325 and 326).

In some embodiments, the image acquisition device 110 can be controlled to acquire first (at each cycle) a central area of the scene 380.

In some embodiments, the pattern of acquisition followed by the line of sight of the image acquisition device 110 with respect to the scene to acquire most or all of the scene (or until a completion criterion is met) can be cyclic (repetitive) at each cycle. In other words, this pattern of acquisition is repeated identically for the plurality of cycles.

An example of this method is illustrated in Fig. 3C.

During a first cycle of acquisition, the pointing device 120 is controlled to make the image acquisition device 110 first acquire the central area 330 of the scene 380, then the fraction 331 of the scene, and then the fractions 332, 333, 334, 335, 336, 336, 337 to end up with the fraction 338.

Once the last fraction 338 of the scene 380 has been acquired at the first cycle, the pointing device 120 is controlled to perform, by the image acquisition device 110, a second cycle of acquisition.

In the second cycle of acquisition, the pointing device 120 is controlled to make the image acquisition device 110 to acquire first the central area 330 of the scene 380.

In other words, at each cycle, the acquisition starts with the same fraction of the scene 380 (see arrow 381 which indicates that the image acquisition device 110 comes back to the central area 330 of the scene 380 at cycle N, after completion of cycle N-l - in this example, N = 2).

Then, the other fractions 332, 333, 334, 335, 336, 336, 337 and 338 of the scene 380 are acquired again.

In some embodiments, at each cycle, the pattern of acquisition followed by the line of sight of the image acquisition device 110 with respect to the scene, until acquisition of most or all of the scene has been performed, can be the same. In the example of Fig. 3C, the pattern of acquisition of the line of sight with respect to the scene follows a spiral (square spiral), which is repeated at each cycle. Note that this is not limitative.

In some embodiments, at each cycle, the first fraction of the scene 380 acquired by the image acquisition device 110 is the same. However, the pattern of acquisition of the line of sight with respect to the scene 380 in order to acquire other fractions of the scene (until most or all of the scene is acquired, or until a completion criterion is met), can be different at each cycle.

A plurality of N cycles (N>2) of acquisition of the scene 380 can be performed.

Note that the method of Figs. 3B and 3C, in which most or all of the scene is acquired a plurality of times, can be performed while the aerial vehicle is flying over the scene along a flight path corresponding to a hold pattern, during each of the N cycles of repetitive acquisition of the scene.

Attention is now drawn to Fig. 4A.

As mentioned above (see operations 280 and 285 in Fig. 2C), the image acquisition device 110 can be operated in a first acquisition mode, in which the scene is acquired progressively by the image acquisition device 110 (see operation 400 in Fig. 4A). In some embodiments, in operation 400, in the first acquisition mode, the scene is acquired while the aerial vehicle is flying in a hold mode. This is not mandatory.

According to some embodiments, the method of Fig. 4A can include, when a first criterion is met (see operation 405 in Fig. 4A), controlling the image acquisition device 110 according to a second acquisition mode of the image acquisition device, different from the first acquisition mode (see operation 410 in Fig. 4A). This can include sending a command to the pointing device 120 and/or to the image acquisition device 110.

In other words, operation of the image acquisition device 110 is switched from a first acquisition mode to a second acquisition mode. In some embodiment, this switch is performed immediately (within a very short timeline) after detecting that the first criterion is met.

As explained hereinafter, the first criterion can be informative of a target present in a scene. In particular, the first criterion can be met when a target (which meets a certain condition) is detected in the images of the scene acquired by the image acquisition device 110 in the first acquisition mode.

In some embodiments, the first criterion is met when a command from an operator is received, which instructs the system to switch from the first acquisition mode to the second acquisition mode. The operator can for example detect by himself presence of the target based on the images acquired in the first acquisition mode.

In some embodiments, the target is detected in the images acquired in the first acquisition mode using a computerized method (e.g., image processing algorithm) and data informative of this detection is output to an operator. However, this detection does not trigger an automatic switch to the second acquisition mode. The switch to the second acquisition mode is triggered in response to a command of an operator, which decides whether the system should switch to the second acquisition mode (based on his analysis of the target output by the system).

According to some embodiments, the method (see Fig. 4B) can include switching the image acquisition device 110 from the second acquisition mode to the first acquisition mode. This can include sending a command to the pointing device 120 and/or to the image acquisition device 110.

This switch can be performed when a second criterion is met (see operation 415). Note that the second criterion is generally different from the first criterion. In some embodiment, this switch is performed immediately (within a very short timeline) after detecting that the second criterion is met.

As explained hereinafter, the second criterion can be also informative of a target. In some embodiments, the second criterion is informative of the same target as the first criterion.

For example, the second criterion can be informative of an absence of the target in the images acquired by the image acquisition device 110 in the second acquisition mode, or of other data linked to the target. In some embodiments, the switch from the second acquisition mode to the first acquisition mode is triggered in response to a command of an operator, which decides whether the system should switch to the first acquisition mode.

In some embodiments, the operator can detect manually that the target has been lost (based on the images provided by the image acquisition device).

In some embodiments, absence of the target in the images is detected using a computerized method (e.g., image processing algorithm), and data informative of this detection is output to an operator. Based on this data, the operator can send a command to the system to instruct the system to remain in the second acquisition mode or to switch to the first acquisition mode.

Note that in some embodiments, the first acquisition mode is devoted to an acquisition of most or all of a scene, whereas the second acquisition mode corresponds to an acquisition of a specific target or area within the scene. In particular, for a given duration, a total area of the scene acquired in the second acquisition mode is smaller than in the first acquisition mode.

The switch between the first acquisition mode and the second acquisition mode can be performed back and forth as many times as required.

Figs. 5A and 5B illustrate a particular embodiment of the method of Fig. 4.

The method of Fig. 5A includes operating (operation 500) the image acquisition device 110 in the first acquisition mode, in which the scene is acquired progressively by the image acquisition device 110. In some embodiments, in the first acquisition mode, the aerial vehicle is flying in a hold mode. This first acquisition mode has been described with reference to Fig. 2C, in which the pointing device 120 is controlled to move the image acquisition device 110 according to a motion pattern enabling its line of sight to be pointed towards different fractions of the scene, which cover e.g., most or all of the scene.

As mentioned above with reference to Fig. 4A, when the first criterion is met, the method can include controlling the image acquisition device 110 according to a second acquisition mode of the image acquisition device, different from the first acquisition mode. The method of Fig. 5A describes a particular example in which meeting of the first criterion corresponds to a detection of a target.

As visible in Fig. 5A, the method includes detecting (operation 510) a target in at least one fraction of the scene acquired by the image acquisition device 110. For example, assume that it is intended to detect a target (expected target 550) in a scene which meets a certain condition. For example, the certain condition corresponds to a certain type of vehicle (e.g., a vehicle with certain dimensions, or a vehicle of a certain type, etc.). For each fraction of the scene acquired by the image acquisition device 110 operating in the first acquisition mode, operation 510 includes detecting whether the expected target is present. This operation can include using an image processing algorithm, such as a detection algorithm and/or a machine learning model (previously trained to detect the target in the images, using e.g., supervised learning and a training set of images), etc. This operation can be performed by the PMC 130 (or by another PMC).

According to some embodiments, detection of the target can include using a Video Motion Detection (VMD) method, which enables detecting a target in a video.

According to some embodiments, detection of the target can include using an Automatic Change Detection (ACD) method, or an Automatic Change Detection and Classification (ACDC) method.

According to some embodiments, detection of the target can include using an Automatic Target Recognition (ATR) method.

In response to detection of the target 550, the method of Fig. 5A includes controlling (operation 520) operation of the image acquisition device 110 according to a second acquisition mode, different from the first acquisition mode. Note that since the same image acquisition device is used in the first acquisition mode and the second acquisition mode, a quick and efficient transition is enabled, which is particularly beneficial for target racking.

As mentioned above, the switch performed at operation 520 can be performed in response to a command of an operator. As mentioned above, data informative of the detection of the target can be output to the operator, who can then decide whether the system has to switch to the second acquisition mode. In some embodiments, a command of the operator to switch to the second acquisition mode is received, following a manual detection of the target in the images, for example by the operator visualizing the images.

Note that the image acquisition device 110 can be controlled according to the second acquisition mode while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern, enabling the aerial vehicle to be maintained substantially above a same scene during a given period of time. In some embodiments, the aerial vehicle 100 flies above the same scene during the first acquisition mode and the second acquisition mode. This is however not limitative.

According to some embodiments, in the second acquisition mode, the image acquisition device 110 is controlled to acquire an image (one or more images) of the target.

In particular, in the second acquisition mode, the method can include sending a command to the pointing device 120 in order to point the line of sight of the image acquisition device towards the target.

This enables acquiring an image which includes mostly the target. Note that the method can be performed for various targets (which can share similar characteristics, or can have different characteristics), thereby enabling generating a bank of images of targets.

Since the target is detected in the fraction of the scenes acquired by the image acquisition device 110 in the first acquisition mode, the line of sight of the image acquisition device is already pointing towards a location of the scene which is close to the target, thereby enabling a quick transition to the acquisition or tracking of the target. This is due to the fact that the same (single) image acquisition device is used in both acquisition modes.

In some embodiments, it can occur that the line of sight of the image acquisition device 110 is already pointed towards the target itself.

According to some embodiments, in response to the detection (computerized detection) of the target (or to a command of an operator following said computerized detection, or following a command of the operator following a manual detection of the target in the images, for example by the operator), the method can include operating the image acquisition device 110 in the second acquisition mode in which the target is tracked. This tracking can include controlling the pointing device 120 to point the line of sight of the image acquisition device towards the target at different instants of time (or towards an area located in the vicinity of the target). Note that the target can be a mobile target, and therefore the pointing device 120 can guide the line of sight of the image acquisition device towards the target at different instants of time.

As mentioned above, since the aerial vehicle 100 is in motion, tracking of the target can include compensating the motion of the aerial vehicle. In particular, this can include obtaining first data informative of a flight path of the aerial vehicle (current flight path and/or expected flight path), second data informative of a required motion pattern of the line of sight on ground (which enables tracking the target), and using the first data and the second data to determine the motion pattern to be applied to the image acquisition device by the pointing device 120 to enable the line of sight tracking the target. The method can further include controlling the pointing device 120 coupled to the image acquisition device according to this motion pattern.

Note that in some embodiments, the required pattern of the line of sight on ground, which should coincide with the path 560 of the target 550 on ground, can be determined using a prediction/tracking method. This method can include obtaining a prediction of the position of the target at a future time. In some embodiments, the target is tracked, and this track is used to predict the future position of the target (using e.g., Kalman filters or other techniques of the art). In some embodiments, tracking of the target can involve using a device (e.g., a radar) which tracks the target.

The method can include pointing the line of sight of the image acquisition device 110 at a time “t”, depending on the predicted position of the target at this time “t”.

Note that the method enables a flexible and quick switch between a mode in which the whole scene (first acquisition mode) is captured, and another mode in which the target is acquired or tracked over time (second acquisition mode). This switch can be performed from the first acquisition mode to the second acquisition mode, or conversely.

In particular, the same image acquisition device is used to perform (alternatively) both tasks. This reduces the pay load of the aerial vehicle and simplifies the control and the data transmission (since only one image acquisition device, together with its pointing device need to be controlled).

In addition, since the same (single) acquisition device 110 is used for scanning the scene and tracking the target, the delays due to processing are reduced, and the tracking can immediately start upon detection of the target in the first acquisition mode. Tracking of the target is therefore improved.

According to some embodiments, when a second criterion met, the method can include switching back the operation of the image acquisition device 110 from the second acquisition mode to the first acquisition mode (operation 530).

As mentioned above, the switch performed at operation 530 can be performed in response to a command of an operator. Alternatively, the switch can be performed automatically when the second criterion is met. Note that in some embodiments, the method can start by operating the image acquisition device 110 according to the second acquisition mode (for example by tracking a target), and then the method can include switching operation of the image acquisition device 110 to the first acquisition mode (this switch can be performed back and forth as many times as required).

In some embodiments, the second criterion is met when a given duration (in which the image acquisition device 110 is operated according to the first acquisition mode) has elapsed.

In some embodiments, the second criterion is met when a given duration from the switch (from the first acquisition mode to the second acquisition mode), performed at operation 510, has elapsed.

In some embodiments, the second criterion is met when an action has been performed with respect to the target, such as annihilating the target (using e.g., destructive ammunition), acquiring a sufficient number of images of the targets, or loss of the target during tracking.

Attention is drawn to Fig. 5C.

In some embodiments, assume that in the second acquisition mode, the image acquisition device 110 is controlled to track a target. As mentioned above, this can be performed using the pointing device 120. Assume that during the tracking, the target has been lost. This can correspond to a situation in which the target does not appear in a sequence of N frames (with N a threshold which can be predefined) acquired by the image acquisition device 110.

In response to loss of the target (which is an example of the second criterion mentioned at operation 415), the method can include switching from the second acquisition mode to the first acquisition mode (operation 570).

Note that the switch from the second acquisition mode to the first acquisition mode can be performed automatically in response to a detection of a loss of the target in the images.

In some embodiments, this switch is performed in response to a command of an operator. The operator can detect manually the loss of the target in the images, or data informative of the loss of the target (obtained using a computerized method, such as an image processing algorithm) can be output to the operator, who can decide whether to send a command to the system to switch from the second acquisition mode to the first acquisition mode, or not.

As mentioned above, in the first acquisition mode, most or all of the scene is acquired by the image acquisition device 110. Therefore, during acquisition of the scene, it can be attempted to detect (using a computerized method, such as an image processing algorithm), once again, the target which has been previously lost. When the target is detected once again, the method can include switching back the image acquisition mode from its first acquisition mode to its second acquisition mode. In some embodiments, the method can include switching back the image acquisition device from its first acquisition mode to its second acquisition mode in response to a command of an operator. In some embodiments, detection of the target is performed using a computerized method, and data informative of this detection is output to an operator, who can decide whether to instruct the system to switch back to the first acquisition mode or not.

In the second acquisition mode, the pointing device 120 is controlled to resume the tracking: the line of sight of the image acquisition device 110 is oriented towards the position of the target over time, as mentioned above.

This enables an efficient and quick switch from a first acquisition mode in which a global image acquisition is performed, to a second acquisition mode in which a more focused image acquisition is performed. In particular, this enables handling situations in which the target has been lost during the tracking. The switch between the first acquisition mode and the second acquisition mode can be performed dynamically over time, depending on detection of the target in the images acquired by the image acquisition device and/or of a loss of the target in the images.

In embodiments of the presently disclosed subject matter, fewer, more, and/or different stages than those shown in the various methods described above may be executed. In embodiments of the presently disclosed subject matter, one or more stages illustrated in the methods described above may be executed in a different order, and/or one or more groups of stages may be executed simultaneously.

The terms "non-transitory memory" and “non-transitory computer readable medium” used herein should be expansively construed to cover any volatile or nonvolatile computer memory suitable to the presently disclosed subject matter. The terms should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The terms shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the computer and that cause the computer to perform any one or more of the methodologies of the present disclosure. The terms shall accordingly be taken to include, but not be limited to, a read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.

It is to be noted that the various features described in the various embodiments may be combined according to all possible technical combinations.

It is to be understood that the invention is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the presently disclosed subject matter.

Those skilled in the art will readily appreciate that various modifications and changes can be applied to the embodiments of the invention as hereinbefore described without departing from its scope, defined in and by the appended claims.

Claims

CLAIMS:

1. A computer-implemented method comprising, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device: while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operating the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

2. The method of claim 1, comprising, for a required motion pattern of the line of sight of the image acquisition device with respect to the scene, selecting the first motion pattern of the image acquisition device, such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by a motion of the aerial vehicle complies with said required motion pattern.

3. The method of claim 2, comprising compensating the motion of the aerial vehicle with the device, to enable a motion of the line of sight of the image acquisition device to comply with the required motion pattern with respect to the scene, despite the motion of the aerial vehicle.

4. The method of any one of claims 1 to 3, wherein a first fraction of the different fractions of the scene acquired by the image acquisition device corresponds to a central area of the scene.

5. The method of any one of claims 1 to 4, wherein, in the first acquisition mode: the image acquisition device is moved according to the first motion pattern to acquire a first set of fractions of the scene, until a coverage of the scene by the image acquisition device meets a completion criterion, and once the completion criterion is met, the image acquisition device is moved by the device to acquire a second set of fractions of the scene, wherein the first set of fractions of the scene and the second set of fractions of the scene overlap or are identical.

6. The method of claim 5, wherein the image acquisition device is moved according to the first motion pattern to acquire the second set of fractions of the scene.

7. The method of claim 5 or of claim 6, wherein the completion criterion is met when at least one of (i) or (ii) is met:

(i) the first set of fractions of the scene acquired by the image acquisition device cover a required area of the scene;

(ii) a duration during which the first set of fractions of the scene have been acquired is above a threshold.

8. The method of any one of claims 1 to 7, comprising, in the first acquisition mode, controlling the device to make the image acquisition device perform a plurality of cycles of acquisition, wherein, at each cycle, most or all of the scene is acquired, wherein a fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle is the same for the plurality of cycles.

9. The method of claim 8, wherein the fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle corresponds to a central area of the scene.

10. The method of claim 8 or of claim 9, comprising controlling the device to make the line of sight of the image acquisition device follow a same pattern of acquisition with respect to the scene for the plurality of cycles.

11. The method of any one or claims 1 to 10, comprising selecting the first motion pattern of the image acquisition device such that a motion of the line of sight on the scene is a spiral or a square spiral.

12. The method of any one of claims 1 to 11, comprising: responsive to detecting that a first criterion is met, switching operation of the image acquisition device from the first acquisition mode to a second acquisition mode, different from the first acquisition mode.

13. The method of claim 12, wherein the first criterion is informative of a target present in the scene, wherein, in the second acquisition mode, the image acquisition device is controlled to acquire one or more images of the target.

14. The method of any one of claims 1 to 13, comprising: in the first acquisition mode, detecting, in at least one fraction of the scene acquired by the image acquisition device, presence of a target that meets a certain condition, responsive to said detecting or to a command of an operator following said detecting, switching operation of the image acquisition device from the first acquisition mode to a second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target.

15. The method of claim 14, wherein, in the second acquisition mode, the device is controlled to make the line of sight of the image acquisition device track the target.

16. The method of claim 15, wherein, responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, the method comprises switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode.

17. The method of any one of claims 12 to 16, wherein, responsive to detecting that a second criterion is met, the method comprises switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode, wherein the second criterion is informative of a target or is informative of a command an operator.

18. The method of any one of claims 12 to 17, wherein, in said second acquisition mode, the aerial vehicle is flying over the scene along a flight path corresponding to a hold pattern.

19. The method of any one of claims 1 to 18, comprising: obtaining data informative of a flight path of the aerial vehicle,

obtaining data D_required_LOS_pattem informative of a required motion pattern of the line of sight on the scene, using data Dfli_ght__Path and data Drequired_LOS_pattem to determine the first motion pattern, such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by the flight path of the aerial vehicle, complies with the required motion pattern, and controlling the device coupled to the image acquisition device according to said first motion pattern.

20. The method of any one of claims 1 to 19, wherein the aerial vehicle cannot hover.

21. A computer-implemented method comprising, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device, operating the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern, enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

22. The method of claim 21, wherein the first criterion is informative of a target present in the scene, wherein in the second acquisition mode, the image acquisition device is controlled to acquire one or more images of the target.

23. The method of claim 21 or of claim 22, wherein detecting that the first criterion is met comprises detecting, in at least one fraction of the scene acquired by the image acquisition device in the first acquisition mode, presence of a target that meets a certain condition, responsive to said detecting of the target or to a command of an operator following said detecting, switching operation of the image acquisition device from the first acquisition mode to the second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target.

24. The method of claim 23, wherein detecting the presence of the target comprises using at least one of a Video Motion Detection (VMD) method, or an Automatic Change Detection (ACD) method, or an Automatic Target Recognition (ATR) method.

25. The method of any one of claims 21 to 24, wherein, said first criterion is met when a command to acquire one or more images of a target of the scene is received, wherein in the second acquisition mode, the device is controlled to make the image acquisition device track the target.

26. The method of any one of claims 21 to 25, wherein, responsive to detecting that a second criterion is met, the method comprises switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode, wherein the second criterion is informative of a target or is informative of a command an operator.

27. The method of any one of claims 21 to 26, comprising switching operation of the image acquisition device from the first acquisition mode to the second acquisition mode in response to detection of a target in one or more images acquired by the image acquisition device or to a command of an operator following said detection, and switching operation of the image acquisition device from the second acquisition mode to the first acquisition mode in response to a detection of an absence of the target in one or more images acquired by the image acquisition device or to a command of an operator following said detection.

28. The method of any one of claims 21 to 27, wherein, for a given duration, a total area of the scene acquired in the second acquisition mode is smaller than in the first acquisition mode.

29. A system adapted to be mounted on an aerial vehicle, the system comprising: an image acquisition device, and a processor and memory circuitry (PMC) operatively coupled to a device enabling a modification of an orientation of the image acquisition device, wherein the PMC is configured to, while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operate the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

30. The system of claim 29, wherein the PMC is configured to, for a required motion pattern of the line of sight of the image acquisition device with respect to the scene, select the first motion pattern of the image acquisition device such that a motion of the line of sight with respect to the scene caused by the first motion pattern and by a motion of the aerial vehicle complies with said required motion pattern.

31. The system of claim 29 or of claim 30, wherein a first fraction of the different fractions of the scene acquired by the image acquisition device corresponds to a central area of the scene.

32. The system of any one of claims 29 to 31, configured to, in the first acquisition mode, control the device to make the image acquisition device perform a plurality of cycles of acquisition, wherein, at each cycle, most or all of the scene is acquired, wherein a fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle is the same for the plurality of cycles.

33. The system of claim 32, configured to control the device such that the fraction of the scene which is acquired first by the image acquisition device at the beginning of each cycle corresponds to a central area of the scene.

34. The system of claim 32 or of claim 33, configured to control the device to make the line of sight of the image acquisition device follow a same pattern of acquisition with respect to the scene for the plurality of cycles.

35. The system of any one of claims 29 to 34, wherein, in the first acquisition mode, the system is configured to: trigger a motion of the image acquisition device according to the first motion pattern to acquire a first set of fractions of the scene, until a coverage of the scene by the image acquisition device meets a completion criterion, wherein a first fraction of the scene acquired by the image acquisition device in the first set corresponds to a given fraction of the scene, once the completion criterion is met, trigger a motion of the image acquisition device by the device to acquire a second set of fractions of the scene, wherein a first fraction of the scene acquired by the image acquisition device in the second set also corresponds to said given fraction of the scene.

36. The system of claim 35, wherein the completion criterion is met when at least one of (i) or (ii) is met:

37. The system of any one or claims 29 to 36, configured to select the first motion pattern of the image acquisition device, such that a motion of the line of sight on the scene is a spiral or a square spiral.

38. The system of any one of claims 29 to 37, configured to: responsive to detecting that a first criterion is met, switch operation of the image acquisition device from the first acquisition mode to a second acquisition mode, different from the first acquisition mode, wherein the first criterion is informative of a target present in the scene.

39. The system of any one of claims 29 to 38, configured to: in the first acquisition mode, detect, in at least one fraction of the scene acquired by the image acquisition device, presence of a target that meets a certain condition, responsive to said detecting or to a command of an operator following said detecting, switch operation of the image acquisition device from the first acquisition mode to a second acquisition mode, wherein, in the second acquisition mode, the image acquisition device is operated to acquire one or more images of the target.

40. The system of any one of claims 29 to 39, wherein, said first criterion is met when a command of an operator to acquire one or more images of a target of the scene is received, wherein, in the second acquisition mode, the device is controlled to make the image acquisition device track the target.

41. The system of claim 40, wherein, in the second acquisition mode, the system is configured to perform (i) or (ii): (i) controlling the device to make the line of sight of the image acquisition device track the target;

(ii) controlling the device to make the line of sight of the image acquisition device track the target, and, responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, switching back operation of the image acquisition device from said second acquisition mode to said first acquisition mode.

42. A system adapted to be mounted on an aerial vehicle, the system comprising: an image acquisition device, and a processor and memory circuitry (PMC) operatively coupled to a device enabling a modification of an orientation of the image acquisition device, wherein the PMC is configured to: control the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, and responsive to detecting that a first criterion is met, operate the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

43. The system of claim 42, wherein detecting that the first criterion is met comprises detecting, in at least one fraction of the scene acquired by the image acquisition device in the first acquisition mode, presence of a target that meets a certain condition, wherein the system is configured to, in response to said detecting of the target, or to command of an operator following said detecting, switch operation of the image acquisition device from the first acquisition mode to the second acquisition mode, wherein, in the second acquisition mode, the system is configured to operate the image acquisition device to acquire one or more images of the target.

44. The system of claim 42 or of claim 43, wherein the first criterion is informative of a presence of a target in the scene, wherein, in the second acquisition mode, the PMC is configured to control the device to make the image acquisition device acquire one or more images of the target.

45. The system of claim 43 or of claim 44, wherein, in the second acquisition mode, the system is configured to control the device to make the line of sight of the image acquisition device track the target.

46. The system of any one of claims 43 to 45, wherein (i) or (ii) is met:

(i) responsive to a detection of a loss of the target in said tracking, or to a command of an operator following said detection, the system is configured to switch back operation of the image acquisition device from said second acquisition mode to said first acquisition mode;

(ii) responsive to detecting that a second criterion is met, the system is configured to switch back operation of the image acquisition device from said second acquisition mode to said first acquisition mode.

47. An aerial vehicle comprising: an image acquisition device, a device enabling a modification of an orientation of the image acquisition device, and a processor and memory circuitry (PMC) configured to, while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operate the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

48. An aerial vehicle comprising: an image acquisition device, a device enabling a modification of an orientation of the image acquisition device, and a processor and memory circuitry (PMC) configured to operate the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode.

49. A non-transitory computer readable medium comprising instructions that, when executed by at least one processing circuitry, cause the at least one processing circuitry to perform, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device: while the aerial vehicle is flying over a scene along a flight path corresponding to a hold pattern enabling the aerial vehicle to be maintained substantially over said same scene during a given period of time, operating the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution.

50. A non-transitory computer readable medium comprising instructions that, when executed by at least one processing circuitry, cause the at least one processing circuitry to perform, for an aerial vehicle comprising an image acquisition device coupled to a device enabling modification of an orientation of a line of sight of the image acquisition device, operating the image acquisition device according to a first acquisition mode, said operating comprising: controlling the device coupled to the image acquisition device to move the image acquisition device according to a first motion pattern, enabling the line of sight of the image acquisition device to be pointed towards different fractions of the scene at different instants of the given period of time, thereby enabling acquisition of the different fractions covering most or all of the scene at a desired image resolution, wherein each given fraction of the different fractions of the scene is acquired by the image acquisition device with a field of view which is smaller than the whole scene at the desired image resolution, responsive to detecting that a first criterion is met, operating the image acquisition device according to a second acquisition mode, different from the first acquisition mode.