WO2023280653A1 - Unmanned moving object and method of operation - Google Patents
Unmanned moving object and method of operation Download PDFInfo
- Publication number
- WO2023280653A1 WO2023280653A1 PCT/EP2022/067871 EP2022067871W WO2023280653A1 WO 2023280653 A1 WO2023280653 A1 WO 2023280653A1 EP 2022067871 W EP2022067871 W EP 2022067871W WO 2023280653 A1 WO2023280653 A1 WO 2023280653A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- interest
- moving object
- region
- imaging device
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Definitions
- This invention relates to object detection and tracking.
- the invention relates to an unmanned moving object and a method of operating an unmanned moving object to follow a predetermined moving object of interest.
- Object recognition is one of the fundamental challenges in computer vision.
- detecting and tracking of objects such as pedestrians are often required, for example in various traffic situations.
- detecting and tracking of a moving object are often also required, such as in the case of service robots following a person moving about.
- One of the challenges of object tracking is the ability to follow an object of interest. For example, if the object of interest is a face and there are many faces detected, there may be confusion in the object tracked.
- a prior approach uses face detection as well as motion detection to track an object of interest.
- the use of different types of sensors increases the cost of the overall system as well as the cost of computation.
- subsequent images are analysed to detect and track the object of interest again.
- detecting an object of interest in one image is computationally intensive, let alone detecting and tracking the object of interest in multiple images.
- a computer-implemented method of operating an unmanned moving object to follow a predetermined moving object of interest comprising: (i) detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object; (ii) obtaining a region of interest comprising the detected representation; (iii) tracking the region of interest; (iv) detecting whether the representation of the predetermined moving object of interest remains in the region of interest; (v) if the representation of the predetermined moving object of interest is no longer in the region of interest, repeating steps (i) to (iv) with a new image frame obtained by the imaging device.
- the present disclosure provides an efficient object detection and tracking process.
- the disclosed method is also advantageously computationally efficient. Further, the disclosed method advantageously uses memory efficiently since the detecting step (iv) generates parameters only in the region of interest to be stored in memory. Thus, the disclosed method is advantageously lightweight.
- the method disclosed herein enables optimal management of resources on the computer that the method is implemented on, such as memory resources and processor load.
- resources on the computer such as memory resources and processor load.
- the present disclosure optimizes the object detection and tracking processes, thereby enabling the method disclosed herein to be implemented even on platforms that have memory constraints.
- the method disclosed herein therefore solves a technical problem of enabling an unmanned moving object operated by a computer with memory constraints to follow a predetermined moving object of interest.
- the method disclosed herein advantageously decouples the two classical stages of object detection and tracking, in the sense that after the detection stage of detecting an object of interest in an image frame, the tracking stage is undertaken using the region of interest, instead of the whole image frame.
- Input into the detecting step (i), or the classical detection stage is an image frame.
- the image frame may be processed during the detection stage.
- the image frame may be scaled or cropped.
- the image frame may be processed or analysed with one or more machine learning or artificial intelligence models, such as deep learning, using algorithms, such as convolutional neural networks.
- the detecting step (i) may comprise applying one or more feature extractors and one or more classifiers.
- Features of the image in the image frame may be extracted.
- the features or objects extracted from the image may be classified.
- a classification of a feature or object may be compared with the classification of the object of interest. If the classifications match, it may be determined that a representation of the object of interest is detected in the image frame. A region of interest comprising the detected representation in the image frame may be obtained. An output of the detection stage may be the region of interest. An output of the detection stage may be input into the tracking stage. Inputs into a typical tracking stage include every image frame of a video sequence and coordinates for a box bounding detected object(s), which is standard in object tracking.
- a technical problem of object tracking is to handle a situation when the tracked object disappears from or moves outside the image frame or when the object tracked is not the correct one.
- the disclosed solution provides for lightweight tracking, whereby the region of interest is used for tracking.
- the disclosed method may restart on a new or current or most recent image frame.
- Full object detection may advantageously be performed on the new or current or most recent image frame.
- the detecting step (i) which may be computationally intensive and/or requires a large amount of memory resources, may advantageously be used only when required.
- input into the tracking step (iii) of the disclosed method, or a classical tracking stage may include a region of interest within the image frame, instead of the whole image frame.
- the region of interest may be input into the tracking step (iii).
- Consecutive image frames may be input into the tracking step (iii), which processes or analyses the region of interest to track the representation comprised therein.
- the region of interest may be processed or analysed with machine learning or artificial intelligence models, such as deep learning, using algorithms, such as convolutional neural networks.
- the detecting step (iv) may advantageously not require the use of feature extractors.
- the region of interest in subsequent frames of the video sequence may then be used for object tracking.
- the two classical stages of object detection and object tracking are advantageously decoupled.
- the detecting step (iv) may comprise applying one or more classifiers.
- the detecting step (iv) may comprise applying one or more binary classifiers.
- the detecting step (iv) may only comprise applying one or more classifiers.
- the classifier(s) may comprise predicting whether the representation is present in the region of interest or not.
- the detecting step (iv) may not comprise the use of feature extractors or regressors.
- classifier(s) may be more computationally efficient and memory efficient than other object detection processes which may use both classifier(s) and feature extractor(s). If the representation of the predetermined moving object of interest is no longer in the region of interest, the detection stage may be repeated with a new image frame or a current, most recent image frame from the video sequence.
- An output of the tracking stage may be instructions to a control unit of the unmanned moving object to move itself. An output of the tracking stage may be input into the control unit.
- the instructions may cause the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest.
- the method may further comprise causing the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest.
- the instructions or the causing step may comprise providing a location of the predetermined moving object of interest associated with the tracked region of interest.
- the location of the tracked object of interest may be represented in any way, for example by coordinates of the object of interest.
- the location of the tracked object of interest may be in relation to the imaging device.
- the location of the imaging device may represent the location of the unmanned moving object.
- the unmanned moving object may follow the object of interest as it moves.
- the imaging device may comprise a wide-angle lens.
- the imaging device may comprise a fisheye lens.
- the lens may be mounted on or attached to any imaging device.
- a wide-angle or even a 360-degree view or a surround view of the predetermined moving object of interest may be obtained in the image frame.
- the method may advantageously be undertaken using one imaging device.
- the image frame comprising the wide-angle, 360-degree or surround view including the predetermined moving object of interest may then be input into the disclosed method.
- the present disclosure provides flexibility to the unmanned moving object since the predetermined moving object of interest need not be in the line of sight of the unmanned moving object.
- an unmanned moving object comprising an imaging device, a control unit and a computer, wherein: the imaging device is configured to obtain a video sequence; the computer is configured to perform the method as disclosed herein; and the control unit is configured to cause the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer.
- the unmanned moving object may be an unmanned ground vehicle or unmanned aerial vehicle, such as a robot, service robot, shopping trolley, suitcase, personal mobility devices, drones, etc.
- the unmanned ground vehicle may be operable to follow a predetermined moving object of interest, such as a person or other moving object.
- the predetermined moving object of interest may be other than a person’s face, for example a wearable worn by the person.
- the unmanned moving object may be an autonomous vehicle.
- the autonomous vehicle may be operable to follow a predetermined moving object of interest, such as other vehicles, for example in platooning applications.
- the autonomous vehicle may have a degree of automation of any one of SAE Levels 1 through 5.
- FIG. 1 illustrates a method 100 according to an embodiment of the present disclosure.
- the method comprises step (i) of detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object.
- the method comprises step (ii) of obtaining a region of interest comprising the detected representation.
- the method comprises step (iii) of tracking the region of interest.
- the method comprises step (iv) of detecting whether the representation of the predetermined moving object of interest remains in the region of interest. If the representation of the predetermined moving object of interest is no longer in the region of interest, the method comprises step (v) of repeating steps (i) to (iv) with a new image frame obtained by the imaging device.
- the imaging device may be affixed to or otherwise coupled to the unmanned moving object at any suitable location.
- a suitable location may be one that enables the imaging device to capture a scene around, surrounding or substantially surrounding the unmanned moving object.
- the imaging device may be configured to obtain a video sequence of its environment or surroundings.
- the imaging device may be configured to continually obtain a video sequence of its environment or surroundings while the unmanned moving object is in operation.
- the imaging device may provide a wide field of view, ultrawide field of view, or even a 360-degree horizontal and/or vertical field of view.
- the imaging device may be a camera, such as a video camera or a depth camera.
- the imaging device may be any standard imaging system, such as a camera from a smart phone. There may be only one imaging device needed for the disclosed method.
- the imaging device may comprise a lens as disclosed herein.
- the video sequence may be stored in a memory of the computer disclosed herein.
- the video sequence may be processed in a processor of the computer disclosed herein.
- the processor may retrieve the video sequence from the memory of the computer and process it as disclosed herein.
- the video sequence may be processed to obtain one or more image frames.
- An image frame may be input into the method disclosed herein.
- the image frame may comprise a representation of the actual predetermined moving object of interest.
- the representation of the predetermined moving object of interest may be detected in the image frame.
- a series of image frames, or consecutive image frames may be input into the method disclosed herein to enable detection and tracking of the representation as the unmanned moving object moves.
- the object detection stage may include steps (i) and (ii) of the method disclosed herein.
- Fig. 1 shows a method 100 according to an embodiment of the present disclosure.
- the image frame 102 obtained from the video sequence (not shown) may be input into step 104.
- the image frame 102 may be pre-processed.
- the image frame 102 may be scaled or cropped.
- the image frame 102 may be pre-processed to correct any image distortion, e.g., from the lens.
- the pre-processed image frame may be input into step 106.
- the pre-processed image frame may be input into a deep learning object detector in order to detect a representation of the actual predetermined moving object of interest in the image frame.
- the image frame may be fed to a deep learning Al feature extractor model custom trained on a predetermined object of interest.
- Step 106 may comprise detecting a representation of the object of interest in the image obtained by the imaging device that is carried by the unmanned moving object.
- Step 106 may comprise the use of one or more feature extractors.
- the feature extractor may comprise one or more convolution neural networks (CNN), such as a region-CNN.
- Step 106 may comprise the use of one or more feature extractors, classifiers, regressors and/or object localization.
- the representation of the actual predetermined moving object of interest may be detected in the image frame.
- step 108 or step (ii) of the disclosed method a region of interest in the image frame that comprises the representation may be obtained.
- Step 108 may provide the region of interest or bounding box or its coordinates comprising a representation of the object of interest. That is, steps 106 and 108 may comprise the use of one or more feature extractors, classifiers and/or object localization.
- the object detection stage may include steps (i) and (ii) of the disclosed method.
- Steps (i) and/or (ii) may comprise a machine learning model trained to detect the predetermined moving object of interest and output the region of interest. The model may be trained on the predetermined object of interest.
- An output of step 108 may be the region of interest.
- the region of interest may be represented by a bounding box if there is one object of interest, or multiple bounding boxes if there are multiple objects of interest.
- the region of interest may comprise a bounding box if there is one object of interest, or multiple bounding boxes if there are multiple objects of interest.
- Step 108 may comprise obtaining a bounding box comprising the detected representation.
- the bounding box may surround, substantially surround or comprise the representation of the predetermined moving object of interest detected in the image frame.
- An output of step 108 may be proposal(s) for coordinates of the region of interest or bounding box. Coordinates for the most probable region of interest or bounding box proposed by step 108 may be output.
- Step 108 may comprise obtaining coordinates of the region of interest or bounding box comprising the detected representation.
- the coordinates for the region of interest or bounding box may be input into step 110 or step (iii) of the disclosed method.
- the object tracking stage may include step 110 or step (iii) of the disclosed method.
- the region of interest or bounding box or coordinates of the bounding box obtained from step 108 for each image frame may be input into step 110.
- the region of interest or bounding box or coordinates of the bounding box obtained from step 108 for an image frame 102 may be input into step 110.
- Subsequent image frames of the video sequence may directly be input into the object tracking stage. Subsequent image frames may not be required to be input into the object detection stage, such as any one of steps 104, 106 and 108.
- Step 110 may comprise processing or analysing the same or similar region of interest or bounding box or coordinates of the bounding box in subsequent image frames as image frame 102, to track the region of interest throughout the video sequence.
- all image frames of the video sequence in certain predefined situations may be input into method 100 from the object detection stage onwards. Subsequent image frames may be additionally input into step 110 and step 112 to improve confidence levels of the object tracked.
- Step 110 may comprise a tracking algorithm.
- Step 110 may comprise an object tracker.
- the object tracker may comprise tracking the movement of pixels in the region of interest in consecutive image frames of the video sequence.
- the object tracker may comprise tracking the centroid of the region of interest.
- the object tracker may comprise Kalman filters.
- the object tracker may comprise correlation filters, for example channel and spatial reliability tracking (CSRT), kernelized correlation filters, and MOSSE filters.
- the object tracker may be selected to suit the object detection stage.
- Step (iii) of the disclosed method may be conditioned on steps (i) and (ii). Each step of a trained object detection and object tracking method may work together with minimal error.
- Each step of the disclosed method may be trained to minimize error.
- the object tracking stage may include step 112 or step (iv) of the disclosed method.
- the region of interest or bounding box or coordinates of the bounding box of the object tracker from step 110 may be input into step 112.
- the region of interest or bounding box or coordinates of the bounding box may be input into a deep learning object detector in order to detect whether the representation of the actual predetermined moving object of interest remains in the region of interest.
- Step 112 or step (iv) may be as disclosed herein.
- Step 112 may comprise determining whether the object of interest will continue to be in the region of interest.
- Step 112 may comprise a convolution neural network (CNN), such as a region-CNN, comprising classification layer(s) to classify feature(s) in the region of interest or bounding box. The resulting classification may be compared with the classification of the object of interest. Step 112 may comprise determining if the region of interest or bounding box surrounds, substantially surrounds or substantially comprises the representation of the object of interest. The presence of step 112 may advantageously allow a generic object tracker to be used in step 110.
- CNN convolution neural network
- the region of interest or bounding box may surround, substantially surround or substantially comprise the representation of the predetermined moving object of interest detected in the image frame. “Substantially” means that an essential part of the representation is within the region of interest or bounding box. An essential part of the representation may depend on the predetermined moving object of interest. An essential part of the representation may be determined by a core part of the predetermined moving object of interest, without which the object of interest is unrecognisable or undetectable. Alternatively, the representation that is surrounded by or comprised in the region of interest or bounding box in substance, or an essential part of the representation, may be determined by an amount of the representation within the region of interest or bounding box.
- the region of interest or bounding box may substantially surround or substantially comprise the representation when about 50%, 60%, 70%, 80%, 90%, 95%, or more, of the representation is within the region of interest or bounding box.
- the region of interest or bounding box substantially surrounding or substantially comprising the representation may be configurable or may be predefined.
- step 114 or step (v) of the disclosed method if the representation of the predetermined moving object of interest is no longer in the region of interest, steps (i) to (iv) may be repeated with a new image frame 102 obtained by the imaging device. An updated region of interest may therefore be obtained.
- the object detection stage may be repeated.
- the method may return to the object detection stage.
- the representation is no longer surrounded by the region of interest, the method may return to the object detection stage.
- the representation is no longer substantially surrounded by, substantially in or substantially comprised in the region of interest, the method may return to the object detection stage.
- the method as disclosed herein may be performed by a computer. Steps (i) to (v) may be performed by a computer.
- the computer may comprise a processor to execute the method disclosed herein or at least some steps of the method disclosed herein.
- a result of the method disclosed herein is to operate the unmanned moving object to follow the predetermined moving object of interest.
- the method may further comprise causing the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest.
- the computer may instruct a control unit of the unmanned moving object to follow the predetermined moving object of interest.
- the unmanned moving object may be caused, in step 116, to follow the predetermined moving object of interest associated with the tracked region of interest.
- Step 116 may comprise a movement algorithm.
- the movement algorithm may comprise identifying whether the predetermined moving object of interest associated with the tracked region of interest has deviated from the line of sight of the unmanned moving object or the movement path of the unmanned moving object.
- the movement algorithm may comprise determining whether the tracked object of interest is offset with respect to the line of sight or movement path of the unmanned moving object.
- the movement algorithm may comprise determining an offset from the line of sight or the movement path of the unmanned moving object. For example, determining an offset from the line of sight of the unmanned moving object may be determined with respect to the imaging device, i.e. the “eyes” of the unmanned moving object.
- the line of sight or line of movement may be a line intersecting the imaging device and along the heading of the unmanned moving object.
- the line may intersect a point on the imaging device, which may be denoted as the origin or coordinate (0,0). Coordinates of the region of interest or bounding box disclosed herein may be determined from this origin.
- a line connecting the imaging device, or its origin, and the tracked object of interest, or a point on the tracked object of interest, may be obtained.
- the point on the tracked object of interest may be any suitable point, such as a centroid.
- Step 116, or the movement algorithm may comprise determining the angle offset between the line of sight or line of movement and the predetermined moving object of interest.
- Step 116 may comprise determining the angle offset between the line of sight or line of movement and the line connecting the imaging device and the tracked predetermined moving object of interest.
- the offset angles may be predefined in order to determine if the unmanned moving object is to maintain on its movement path or to move along a different movement path.
- a range of offset angles may be predefined to cause the unmanned moving object to maintain on its movement path.
- a range of offset angles may be predefined to cause the unmanned moving object to turn left or right.
- step 116 may comprise causing the unmanned moving object to determine or update its movement path according to the offset angle determined.
- a distance between the unmanned moving object and the predetermined object of interest may be predefined. The predefined distance may be between the origin and a point on the tracked object of interest.
- the movement algorithm may comprise identifying whether the distance between the tracked object of interest exceeds the predefined distance. If the identified distance exceeds the predefined distance, the unmanned moving object may be caused to move along the movement path or an updated movement path, in order to maintain the predefined distance.
- the control unit may move the unmanned moving object according to the movement path determined or according to an updated movement path.
- the control unit may comprise a drivetrain or other system that enables the unmanned moving object to move. In the embodiment where a wide-angle lens is used, the unmanned moving object may advantageously be capable of moving along a wide-angle movement path.
- the disclosed invention may enable the unmanned moving object to follow the predetermined moving object of interest regardless of whether the tracked object of interest is along its line of sight or line of movement.
- the unmanned moving object may be a wheelchair.
- the predetermined moving object of interest may be a smart watch of a mobile person.
- the predefined distance to be maintained between the unmanned moving object and the object of interest may be about 1 to 2 meters.
- the scene of the surroundings may be captured by an imaging device affixed to the wheelchair.
- the region of interest may be a fraction of the image frame obtained from a video sequence captured by the imaging device.
- the region of interest may be about 10-20% of the image frame based on the predefined distance.
- the detecting step (iv) may comprise only a binary classifier.
- the detecting step (i) may comprise a feature extractor, a classifier and a regressor.
- the parameters generated by detecting step (iv) may be about 10-20% of that generated by detecting step (i).
- the detecting step (iv) may enable the disclosed method to be lightweight.
- an unmanned moving object comprising an imaging device as described herein.
- the unmanned moving object comprises a control unit as described herein.
- the unmanned moving object comprises a computer as described herein.
- the imaging device is configured to obtain a video sequence as described herein.
- the computer is configured to perform the method as described herein.
- the control unit is configured to cause the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer as described herein.
- the unmanned moving object may comprise one or more computers, for example, an embedded system or a general computing device.
- the computer may comprise one or more computer-readable storage media or memory modules, which may comprise transitory and non-transitory memory.
- the computer-readable storage media may encompass any electronic component capable of storing electronic information.
- the computer-readable storage media or memory may include transitory processor-readable media such as random access memory (RAM) or cache memory.
- the computer-readable storage media or memory may include non-transitory processor-readable media such as read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc.
- ROM read-only memory
- NVRAM non-volatile random access memory
- PROM programmable read-only memory
- EPROM erasable programmable read only memory
- EEPROM electrically erasable PROM
- flash memory magnetic or optical data storage, registers, etc.
- the memory is in electronic communication with a processor as disclosed herein and/or other processors of the computer.
- Computer-readable instructions such as an operating system, middleware, firmware or other software framework, may reside in the non-transitory computer-readable storage medium.
- Computer-readable instructions may be implemented as a program or a code that can be read by the processor.
- the disclosed method may be implemented as a program or a code that can be read by the processor of the unmanned moving object.
- a computer of an unmanned moving object operable to perform the method as disclosed herein.
- Exemplary processor(s) of the computer include a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, programmable gate arrays, systems-on-chip (SoC), programmable SoCs, or other suitable devices.
- processor may include a combination of processing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration suitable for the disclosed computer.
- the unmanned moving object may comprise computer-readable instructions stored in non-transitory memory.
- the instructions in the unmanned moving object may comprise one or more of the steps disclosed herein.
- the computer-readable instructions When executed by a processor of the unmanned moving object, the computer-readable instructions may cause the processor to execute the method disclosed herein or at least some steps of the method disclosed herein.
- a non-transitory memory storing one or more programs, the one or more programs when executed by the unmanned moving object, causes the unmanned moving object to perform at least one or more steps disclosed herein.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
There is provided a computer-implemented method of operating an unmanned moving object to follow a predetermined moving object of interest, the method comprising: (i) detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object; (ii) obtaining a region of interest comprising the detected representation; (iii) tracking the region of interest; (iv) detecting whether the representation of the predetermined moving object of interest remains in the region of interest; (v) if the representation of the predetermined moving object of interest is no longer in the region of interest, repeating steps (i) to (iv) with a new image frame obtained by the imaging device. There is also provided a computer of an unmanned moving object operable to perform the method disclosed. There is also provided an unmanned moving object comprising an imaging device, a control unit and a computer, wherein: the imaging device is configured to obtain a video sequence; the computer is configured to perform the method disclosed; and the control unit is configured to cause the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer.
Description
Unmanned Moving Object and Method of Operation
Field of Invention
[001] This invention relates to object detection and tracking. In particular, the invention relates to an unmanned moving object and a method of operating an unmanned moving object to follow a predetermined moving object of interest.
Background of Invention
[002] Object recognition is one of the fundamental challenges in computer vision. In automotive applications, detecting and tracking of objects such as pedestrians are often required, for example in various traffic situations. In robotic applications, detecting and tracking of a moving object are often also required, such as in the case of service robots following a person moving about. One of the challenges of object tracking is the ability to follow an object of interest. For example, if the object of interest is a face and there are many faces detected, there may be confusion in the object tracked.
[003] A prior approach uses face detection as well as motion detection to track an object of interest. Flowever, the use of different types of sensors increases the cost of the overall system as well as the cost of computation. In another prior approach, if a tracking failure occurs, subsequent images are analysed to detect and track the object of interest again. Flowever, detecting an object of interest in one image is computationally intensive, let alone detecting and tracking the object of interest in multiple images.
[004] Accordingly, there is a need to provide a method of detection and tracking that overcomes or at least ameliorates one or more of the disadvantages discussed above and other disadvantages.
Summary
[005] It is an object to provide an improved object detection and tracking process. It is an object to provide a lightweight object detection and tracking process. It is an object to provide an improved method to enable an unmanned moving object to accurately follow a predetermined moving object of interest.
[006] To accomplish these and other objects, there is provided, in an aspect, a computer-implemented method of operating an unmanned moving object to follow a predetermined moving object of interest, the method comprising: (i) detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object; (ii) obtaining a region of interest comprising the detected representation; (iii) tracking the region of interest; (iv) detecting whether the representation of the predetermined moving object of interest remains in the region of interest; (v) if the representation of the predetermined moving object of interest is no longer in the region of interest, repeating steps (i) to (iv) with a new image frame obtained by the imaging device.
[007] Advantageously, after detecting the object of interest in an image frame, such as a whole image frame, subsequent detection of the object of interest is undertaken on the region of interest within the image frame. The region of interest may advantageously be a fraction of the image frame. Thus, the present disclosure provides an efficient object detection and tracking process. The disclosed method is also advantageously computationally efficient. Further, the disclosed method advantageously uses memory efficiently since the detecting step (iv) generates parameters only in the region of interest to be stored in memory. Thus, the disclosed method is advantageously lightweight.
[008] Advantageously, the method disclosed herein enables optimal management of resources on the computer that the method is implemented on, such as memory resources and processor load. Hence, regardless of whether the computer has a large amount of memory or high processing speed at its disposal or not, the present
disclosure optimizes the object detection and tracking processes, thereby enabling the method disclosed herein to be implemented even on platforms that have memory constraints. The method disclosed herein therefore solves a technical problem of enabling an unmanned moving object operated by a computer with memory constraints to follow a predetermined moving object of interest.
[009] The method disclosed herein advantageously decouples the two classical stages of object detection and tracking, in the sense that after the detection stage of detecting an object of interest in an image frame, the tracking stage is undertaken using the region of interest, instead of the whole image frame. Input into the detecting step (i), or the classical detection stage, is an image frame. The image frame may be processed during the detection stage. The image frame may be scaled or cropped. The image frame may be processed or analysed with one or more machine learning or artificial intelligence models, such as deep learning, using algorithms, such as convolutional neural networks. The detecting step (i) may comprise applying one or more feature extractors and one or more classifiers. Features of the image in the image frame may be extracted. The features or objects extracted from the image may be classified. A classification of a feature or object may be compared with the classification of the object of interest. If the classifications match, it may be determined that a representation of the object of interest is detected in the image frame. A region of interest comprising the detected representation in the image frame may be obtained. An output of the detection stage may be the region of interest. An output of the detection stage may be input into the tracking stage. Inputs into a typical tracking stage include every image frame of a video sequence and coordinates for a box bounding detected object(s), which is standard in object tracking. However, a technical problem of object tracking is to handle a situation when the tracked object disappears from or moves outside the image frame or when the object tracked is not the correct one. The disclosed solution provides for lightweight tracking, whereby the region of interest is used for tracking. If the tracked object disappears from or moves outside the region of interest or when the object tracked is not the correct one, the disclosed method may restart on a new or current or most recent image frame. Full object detection may advantageously be performed on the new or current or most recent image frame.
The detecting step (i), which may be computationally intensive and/or requires a large amount of memory resources, may advantageously be used only when required. Advantageously, input into the tracking step (iii) of the disclosed method, or a classical tracking stage, may include a region of interest within the image frame, instead of the whole image frame. The region of interest may be input into the tracking step (iii). Consecutive image frames may be input into the tracking step (iii), which processes or analyses the region of interest to track the representation comprised therein. The region of interest may be processed or analysed with machine learning or artificial intelligence models, such as deep learning, using algorithms, such as convolutional neural networks. As the region of interest may already comprise a specific feature or object representing the object of interest, the detecting step (iv) may advantageously not require the use of feature extractors. The region of interest in subsequent frames of the video sequence may then be used for object tracking. Thus, the two classical stages of object detection and object tracking are advantageously decoupled.
[010] During the tracking stage, the regions of interest in subsequent frames of the video sequence may be analysed to determine whether the representation of the predetermined moving object of interest remains in the region of interest. As mentioned above, this is an advantage since the region of interest is analysed instead of the whole image frame. The detecting step (iv) may comprise applying one or more classifiers. The detecting step (iv) may comprise applying one or more binary classifiers. The detecting step (iv) may only comprise applying one or more classifiers. The classifier(s) may comprise predicting whether the representation is present in the region of interest or not. The detecting step (iv) may not comprise the use of feature extractors or regressors. Advantageously, the use of classifier(s) may be more computationally efficient and memory efficient than other object detection processes which may use both classifier(s) and feature extractor(s). If the representation of the predetermined moving object of interest is no longer in the region of interest, the detection stage may be repeated with a new image frame or a current, most recent image frame from the video sequence.
[011] An output of the tracking stage may be instructions to a control unit of the unmanned moving object to move itself. An output of the tracking stage may be input into the control unit. The instructions may cause the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest. The method may further comprise causing the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest. The instructions or the causing step may comprise providing a location of the predetermined moving object of interest associated with the tracked region of interest. The location of the tracked object of interest may be represented in any way, for example by coordinates of the object of interest. The location of the tracked object of interest may be in relation to the imaging device. The location of the imaging device may represent the location of the unmanned moving object. Thus, the unmanned moving object may follow the object of interest as it moves.
[012] The imaging device may comprise a wide-angle lens. The imaging device may comprise a fisheye lens. The lens may be mounted on or attached to any imaging device. Advantageously, a wide-angle or even a 360-degree view or a surround view of the predetermined moving object of interest may be obtained in the image frame. The method may advantageously be undertaken using one imaging device. The image frame comprising the wide-angle, 360-degree or surround view including the predetermined moving object of interest may then be input into the disclosed method. Compared with typical imaging devices having limited field of views, the present disclosure provides flexibility to the unmanned moving object since the predetermined moving object of interest need not be in the line of sight of the unmanned moving object.
[013] In another aspect, there is provided a computer of an unmanned moving object operable to perform the method as disclosed herein.
[014] In yet another aspect, there is provided an unmanned moving object comprising an imaging device, a control unit and a computer, wherein: the imaging device is configured to obtain a video sequence; the computer is configured to perform the method as disclosed herein; and the control unit is configured to cause
the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer.
[015] The unmanned moving object may be an unmanned ground vehicle or unmanned aerial vehicle, such as a robot, service robot, shopping trolley, suitcase, personal mobility devices, drones, etc. The unmanned ground vehicle may be operable to follow a predetermined moving object of interest, such as a person or other moving object. The predetermined moving object of interest may be other than a person’s face, for example a wearable worn by the person. The unmanned moving object may be an autonomous vehicle. The autonomous vehicle may be operable to follow a predetermined moving object of interest, such as other vehicles, for example in platooning applications. The autonomous vehicle may have a degree of automation of any one of SAE Levels 1 through 5.
Brief Description of Drawings
[016] Fig. 1 illustrates a method 100 according to an embodiment of the present disclosure.
Detailed Description
[017] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawing. The detailed description of this invention will be provided for the purpose of explaining the principles of the invention and its practical application, thereby enabling a person skilled in the art to understand the invention for various exemplary embodiments and with various modifications as are suited to the particular use contemplated. The detailed description is not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Modifications and equivalents will be apparent to practitioners skilled in this art and are encompassed within the spirit and scope of the appended claims.
[018] The present disclosure provides a computer-implemented method of operating an unmanned moving object to follow a predetermined moving object of interest. The method comprises step (i) of detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object. The method comprises step (ii) of obtaining a region of interest comprising the detected representation. The method comprises step (iii) of tracking the region of interest. The method comprises step (iv) of detecting whether the representation of the predetermined moving object of interest remains in the region of interest. If the representation of the predetermined moving object of interest is no longer in the region of interest, the method comprises step (v) of repeating steps (i) to (iv) with a new image frame obtained by the imaging device.
[019] The imaging device may be affixed to or otherwise coupled to the unmanned moving object at any suitable location. A suitable location may be one that enables the imaging device to capture a scene around, surrounding or substantially surrounding the unmanned moving object. The imaging device may be configured to obtain a video sequence of its environment or surroundings. The imaging device may be configured to continually obtain a video sequence of its environment or surroundings while the unmanned moving object is in operation. The imaging device may provide a wide field of view, ultrawide field of view, or even a 360-degree horizontal and/or vertical field of view. The imaging device may be a camera, such as a video camera or a depth camera. The imaging device may be any standard imaging system, such as a camera from a smart phone. There may be only one imaging device needed for the disclosed method. The imaging device may comprise a lens as disclosed herein. The video sequence may be stored in a memory of the computer disclosed herein. The video sequence may be processed in a processor of the computer disclosed herein. The processor may retrieve the video sequence from the memory of the computer and process it as disclosed herein. The video sequence may be processed to obtain one or more image frames. An image frame may be input into the method disclosed herein. The image frame may comprise a representation of the actual predetermined moving object of interest. The representation of the predetermined moving object of interest may be detected in
the image frame. A series of image frames, or consecutive image frames, may be input into the method disclosed herein to enable detection and tracking of the representation as the unmanned moving object moves.
[020] The object detection stage may include steps (i) and (ii) of the method disclosed herein. Fig. 1 shows a method 100 according to an embodiment of the present disclosure. The image frame 102 obtained from the video sequence (not shown) may be input into step 104. In step 104, the image frame 102 may be pre-processed. The image frame 102 may be scaled or cropped. The image frame 102 may be pre-processed to correct any image distortion, e.g., from the lens. The pre-processed image frame may be input into step 106. In step 106 or detecting step (i) of the disclosed method, the pre-processed image frame may be input into a deep learning object detector in order to detect a representation of the actual predetermined moving object of interest in the image frame. The image frame may be fed to a deep learning Al feature extractor model custom trained on a predetermined object of interest. Step 106 may comprise detecting a representation of the object of interest in the image obtained by the imaging device that is carried by the unmanned moving object. Step 106 may comprise the use of one or more feature extractors. The feature extractor may comprise one or more convolution neural networks (CNN), such as a region-CNN. Step 106 may comprise the use of one or more feature extractors, classifiers, regressors and/or object localization. The representation of the actual predetermined moving object of interest may be detected in the image frame. In step 108 or step (ii) of the disclosed method, a region of interest in the image frame that comprises the representation may be obtained. Step 108 may provide the region of interest or bounding box or its coordinates comprising a representation of the object of interest. That is, steps 106 and 108 may comprise the use of one or more feature extractors, classifiers and/or object localization. In other words, as mentioned above, the object detection stage may include steps (i) and (ii) of the disclosed method. Steps (i) and/or (ii) may comprise a machine learning model trained to detect the predetermined moving object of interest and output the region of interest. The model may be trained on the predetermined object of interest. An output of step 108 may be the region of interest. The region of interest may be represented by a bounding box if there is one object of
interest, or multiple bounding boxes if there are multiple objects of interest. The region of interest may comprise a bounding box if there is one object of interest, or multiple bounding boxes if there are multiple objects of interest. Step 108 may comprise obtaining a bounding box comprising the detected representation. The bounding box may surround, substantially surround or comprise the representation of the predetermined moving object of interest detected in the image frame. An output of step 108 may be proposal(s) for coordinates of the region of interest or bounding box. Coordinates for the most probable region of interest or bounding box proposed by step 108 may be output. Step 108 may comprise obtaining coordinates of the region of interest or bounding box comprising the detected representation. The coordinates for the region of interest or bounding box may be input into step 110 or step (iii) of the disclosed method.
[021] The object tracking stage may include step 110 or step (iii) of the disclosed method. The region of interest or bounding box or coordinates of the bounding box obtained from step 108 for each image frame may be input into step 110. The region of interest or bounding box or coordinates of the bounding box obtained from step 108 for an image frame 102 may be input into step 110. Subsequent image frames of the video sequence may directly be input into the object tracking stage. Subsequent image frames may not be required to be input into the object detection stage, such as any one of steps 104, 106 and 108. Step 110 may comprise processing or analysing the same or similar region of interest or bounding box or coordinates of the bounding box in subsequent image frames as image frame 102, to track the region of interest throughout the video sequence. Alternatively, all image frames of the video sequence in certain predefined situations, such as scenes identified as safety critical, may be input into method 100 from the object detection stage onwards. Subsequent image frames may be additionally input into step 110 and step 112 to improve confidence levels of the object tracked.
[022] Step 110 may comprise a tracking algorithm. Step 110 may comprise an object tracker. The object tracker may comprise tracking the movement of pixels in the region of interest in consecutive image frames of the video sequence. The object tracker may comprise tracking the centroid of the region of interest. The object
tracker may comprise Kalman filters. The object tracker may comprise correlation filters, for example channel and spatial reliability tracking (CSRT), kernelized correlation filters, and MOSSE filters. The object tracker may be selected to suit the object detection stage. Step (iii) of the disclosed method may be conditioned on steps (i) and (ii). Each step of a trained object detection and object tracking method may work together with minimal error. Each step of the disclosed method may be trained to minimize error.
[023] The object tracking stage may include step 112 or step (iv) of the disclosed method. The region of interest or bounding box or coordinates of the bounding box of the object tracker from step 110 may be input into step 112. In step 112, the region of interest or bounding box or coordinates of the bounding box may be input into a deep learning object detector in order to detect whether the representation of the actual predetermined moving object of interest remains in the region of interest. Step 112 or step (iv) may be as disclosed herein. Step 112 may comprise determining whether the object of interest will continue to be in the region of interest. Step 112 may comprise a convolution neural network (CNN), such as a region-CNN, comprising classification layer(s) to classify feature(s) in the region of interest or bounding box. The resulting classification may be compared with the classification of the object of interest. Step 112 may comprise determining if the region of interest or bounding box surrounds, substantially surrounds or substantially comprises the representation of the object of interest. The presence of step 112 may advantageously allow a generic object tracker to be used in step 110.
[024] The region of interest or bounding box may surround, substantially surround or substantially comprise the representation of the predetermined moving object of interest detected in the image frame. “Substantially” means that an essential part of the representation is within the region of interest or bounding box. An essential part of the representation may depend on the predetermined moving object of interest. An essential part of the representation may be determined by a core part of the predetermined moving object of interest, without which the object of interest is unrecognisable or undetectable. Alternatively, the representation that is surrounded by or comprised in the region of interest or bounding box in substance, or an
essential part of the representation, may be determined by an amount of the representation within the region of interest or bounding box. The region of interest or bounding box may substantially surround or substantially comprise the representation when about 50%, 60%, 70%, 80%, 90%, 95%, or more, of the representation is within the region of interest or bounding box. The region of interest or bounding box substantially surrounding or substantially comprising the representation may be configurable or may be predefined.
[025] In step 114 or step (v) of the disclosed method, if the representation of the predetermined moving object of interest is no longer in the region of interest, steps (i) to (iv) may be repeated with a new image frame 102 obtained by the imaging device. An updated region of interest may therefore be obtained. The object detection stage may be repeated. When the representation of the predetermined moving object of interest is no longer in the region of interest, the method may return to the object detection stage. When the representation is no longer surrounded by the region of interest, the method may return to the object detection stage. When the representation is no longer substantially surrounded by, substantially in or substantially comprised in the region of interest, the method may return to the object detection stage.
[026] The method as disclosed herein may be performed by a computer. Steps (i) to (v) may be performed by a computer. The computer may comprise a processor to execute the method disclosed herein or at least some steps of the method disclosed herein. A result of the method disclosed herein is to operate the unmanned moving object to follow the predetermined moving object of interest. The method may further comprise causing the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest. Once a representation of the predetermined moving object of interest is detected and tracked by the method using the computer, the computer may instruct a control unit of the unmanned moving object to follow the predetermined moving object of interest. In an embodiment, once a region of interest is tracked in step 110 or step (iii), the unmanned moving object may be caused, in step 116, to follow the
predetermined moving object of interest associated with the tracked region of interest.
[027] Step 116 may comprise a movement algorithm. The movement algorithm may comprise identifying whether the predetermined moving object of interest associated with the tracked region of interest has deviated from the line of sight of the unmanned moving object or the movement path of the unmanned moving object. The movement algorithm may comprise determining whether the tracked object of interest is offset with respect to the line of sight or movement path of the unmanned moving object. The movement algorithm may comprise determining an offset from the line of sight or the movement path of the unmanned moving object. For example, determining an offset from the line of sight of the unmanned moving object may be determined with respect to the imaging device, i.e. the “eyes” of the unmanned moving object. The line of sight or line of movement may be a line intersecting the imaging device and along the heading of the unmanned moving object. The line may intersect a point on the imaging device, which may be denoted as the origin or coordinate (0,0). Coordinates of the region of interest or bounding box disclosed herein may be determined from this origin. A line connecting the imaging device, or its origin, and the tracked object of interest, or a point on the tracked object of interest, may be obtained. The point on the tracked object of interest may be any suitable point, such as a centroid. Step 116, or the movement algorithm, may comprise determining the angle offset between the line of sight or line of movement and the predetermined moving object of interest. Step 116, or the movement algorithm, may comprise determining the angle offset between the line of sight or line of movement and the line connecting the imaging device and the tracked predetermined moving object of interest. The offset angles may be predefined in order to determine if the unmanned moving object is to maintain on its movement path or to move along a different movement path. A range of offset angles may be predefined to cause the unmanned moving object to maintain on its movement path. A range of offset angles may be predefined to cause the unmanned moving object to turn left or right. Alternatively, step 116 may comprise causing the unmanned moving object to determine or update its movement path according to the offset angle determined. A distance between the unmanned moving object and
the predetermined object of interest may be predefined. The predefined distance may be between the origin and a point on the tracked object of interest. The movement algorithm may comprise identifying whether the distance between the tracked object of interest exceeds the predefined distance. If the identified distance exceeds the predefined distance, the unmanned moving object may be caused to move along the movement path or an updated movement path, in order to maintain the predefined distance. The control unit may move the unmanned moving object according to the movement path determined or according to an updated movement path. The control unit may comprise a drivetrain or other system that enables the unmanned moving object to move. In the embodiment where a wide-angle lens is used, the unmanned moving object may advantageously be capable of moving along a wide-angle movement path. The disclosed invention may enable the unmanned moving object to follow the predetermined moving object of interest regardless of whether the tracked object of interest is along its line of sight or line of movement.
[028] In an example, the unmanned moving object may be a wheelchair. The predetermined moving object of interest may be a smart watch of a mobile person. The predefined distance to be maintained between the unmanned moving object and the object of interest may be about 1 to 2 meters. The scene of the surroundings may be captured by an imaging device affixed to the wheelchair. The region of interest may be a fraction of the image frame obtained from a video sequence captured by the imaging device. The region of interest may be about 10-20% of the image frame based on the predefined distance. In this example, the detecting step (iv) may comprise only a binary classifier. The detecting step (i) may comprise a feature extractor, a classifier and a regressor. The parameters generated by detecting step (iv) may be about 10-20% of that generated by detecting step (i). Thus, the detecting step (iv) may enable the disclosed method to be lightweight.
[029] In an embodiment, there is provided an unmanned moving object. The unmanned moving object comprises an imaging device as described herein. The unmanned moving object comprises a control unit as described herein. The unmanned moving object comprises a computer as described herein. The imaging
device is configured to obtain a video sequence as described herein. The computer is configured to perform the method as described herein. The control unit is configured to cause the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer as described herein.
[030] The unmanned moving object may comprise one or more computers, for example, an embedded system or a general computing device. The computer may comprise one or more computer-readable storage media or memory modules, which may comprise transitory and non-transitory memory. The computer-readable storage media may encompass any electronic component capable of storing electronic information. The computer-readable storage media or memory may include transitory processor-readable media such as random access memory (RAM) or cache memory. The computer-readable storage media or memory may include non-transitory processor-readable media such as read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. The memory is in electronic communication with a processor as disclosed herein and/or other processors of the computer. Computer-readable instructions, such as an operating system, middleware, firmware or other software framework, may reside in the non-transitory computer-readable storage medium. Computer-readable instructions may be implemented as a program or a code that can be read by the processor. The disclosed method may be implemented as a program or a code that can be read by the processor of the unmanned moving object. Accordingly, in an embodiment, there is provided a computer of an unmanned moving object operable to perform the method as disclosed herein. Exemplary processor(s) of the computer include a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, programmable gate arrays, systems-on-chip (SoC), programmable SoCs, or other suitable devices. The term "processor" may include a combination of processing devices, e.g., a combination of a DSP and a microprocessor, a plurality
of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration suitable for the disclosed computer.
[031] The unmanned moving object may comprise computer-readable instructions stored in non-transitory memory. The instructions in the unmanned moving object may comprise one or more of the steps disclosed herein. When executed by a processor of the unmanned moving object, the computer-readable instructions may cause the processor to execute the method disclosed herein or at least some steps of the method disclosed herein. Accordingly, in an embodiment, there is provided a non-transitory memory storing one or more programs, the one or more programs when executed by the unmanned moving object, causes the unmanned moving object to perform at least one or more steps disclosed herein.
Claims
1. A computer-implemented method of operating an unmanned moving object to follow a predetermined moving object of interest, the method comprising:
(i) detecting a representation of the predetermined moving object of interest in an image frame of a video sequence obtained by an imaging device affixed to the unmanned moving object;
(ii) obtaining a region of interest comprising the detected representation;
(iii) tracking the region of interest;
(iv) detecting whether the representation of the predetermined moving object of interest remains in the region of interest;
(v) if the representation of the predetermined moving object of interest is no longer in the region of interest, repeating steps (i) to (iv) with a new image frame obtained by the imaging device.
2. The method of claim 1 , wherein the region of interest is a fraction of the image frame.
3. The method of claim 1 or 2, wherein the detecting step (iv) comprises applying one or more classifiers.
4. The method of any preceding claim, wherein the detecting step (iv) does not comprise the use of feature extractors.
5. The method of any preceding claim, wherein the detecting step (i) comprises applying one or more feature extractors and one or more classifiers.
6. The method of any preceding claim, wherein each of the detecting steps comprises one or more convolutional neural networks.
7. The method of any preceding claim, further comprising causing the unmanned moving object to follow the predetermined moving object of interest associated with the tracked region of interest.
8. The method of any preceding claim, wherein the imaging device is a camera.
9. The method of any preceding claim, wherein the method is undertaken using one imaging device.
10. The method of any preceding claim, wherein the imaging device comprises a wide-angle lens.
11 .The method of any preceding claim, wherein the imaging device comprises a fisheye lens.
12. The method of any preceding claim, wherein the unmanned moving object is an unmanned ground vehicle or an autonomous vehicle.
13. A computer of an unmanned moving object operable to perform the method according to any preceding claim.
14. An unmanned moving object comprising an imaging device, a control unit and a computer, wherein: the imaging device is configured to obtain a video sequence; the computer is configured to perform the method according to any one of claims 1-12; and the control unit is configured to cause the unmanned moving object to follow a predetermined moving object of interest associated with a region of interest tracked by the computer.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| SG10202107577P | 2021-07-09 | ||
| SG10202107577P | 2021-07-09 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023280653A1 true WO2023280653A1 (en) | 2023-01-12 |
Family
ID=82694056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/EP2022/067871 Ceased WO2023280653A1 (en) | 2021-07-09 | 2022-06-29 | Unmanned moving object and method of operation |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2023280653A1 (en) |
-
2022
- 2022-06-29 WO PCT/EP2022/067871 patent/WO2023280653A1/en not_active Ceased
Non-Patent Citations (3)
| Title |
|---|
| JIANQIANG WANG ET AL: "A Region Tracking-Based Vehicle Detection Algorithm in Nighttime Traffic Scenes", SENSORS, vol. 13, no. 12, 2 December 2013 (2013-12-02), pages 16474 - 16493, XP055497723, DOI: 10.3390/s131216474 * |
| RAMESH BHARATH ET AL: "Long-term object tracking with a moving event camera", BMVC, 2018, pages 1 - 12, XP055961738, Retrieved from the Internet <URL:http://bmvc2018.org/contents/papers/0814.pdf> [retrieved on 2022] * |
| ZHEN XINXIN ET AL: "A Visual Object Tracking Algorithm Based on Improved TLD", ALGORITHMS, vol. 13, no. 1, 1 January 2020 (2020-01-01), pages 15, XP055961740, DOI: 10.3390/a13010015 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230014874A1 (en) | Obstacle detection method and apparatus, computer device, and storage medium | |
| CN113228043B (en) | System and method for obstacle detection and association based on neural network for mobile platform | |
| KR101645722B1 (en) | Unmanned aerial vehicle having Automatic Tracking and Method of the same | |
| US11010622B2 (en) | Infrastructure-free NLoS obstacle detection for autonomous cars | |
| CN110264495B (en) | A target tracking method and device | |
| US12065261B2 (en) | Segmentation-based fuel receptacle localization for air-to-air refueling (A3R) | |
| JP7147420B2 (en) | OBJECT DETECTION DEVICE, OBJECT DETECTION METHOD AND COMPUTER PROGRAM FOR OBJECT DETECTION | |
| CN115004259B (en) | Object recognition method, device, computer equipment and storage medium | |
| CN113239719B (en) | Trajectory prediction method and device based on abnormal information identification and computer equipment | |
| KR101769601B1 (en) | Unmanned aerial vehicle having Automatic Tracking | |
| CN111027381A (en) | Method, device, device and storage medium for identifying obstacles using monocular camera | |
| Talele et al. | Detection of real time objects using TensorFlow and OpenCV | |
| CN113850209A (en) | Dynamic object detection method and device, vehicle and storage medium | |
| CN118189959A (en) | A UAV target positioning method based on YOLO attitude estimation | |
| EP4414746A1 (en) | Multi-object detection and tracking | |
| US20220108104A1 (en) | Method for recognizing recognition target person | |
| CN114926534A (en) | Obstacle sensing method and device, readable storage medium and electronic equipment | |
| CN113661513B (en) | Image processing method, device, image processing system and storage medium | |
| US20220004770A1 (en) | Object detection using a combination of deep learning and non-deep learning techniques | |
| JP2018073308A (en) | Recognition device and program | |
| CN114782484B (en) | Multi-target tracking method and system for detecting loss and association failure | |
| El Gaouti et al. | Using object detection for a robust monocular SLAM in dynamic environments | |
| Nalcakan et al. | Monocular vision-based prediction of cut-in maneuvers with lstm networks | |
| KR101875517B1 (en) | Method and apparatus for processing a image | |
| Liu et al. | Advanced Obstacle Detection and Distance Estimation for Forklift Operations through Integrated Deep Learning Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22746962 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 22746962 Country of ref document: EP Kind code of ref document: A1 |