KR20220155041A - Event camera-based object tracking apparatus and method - Google Patents

Abstract

An embodiment of the present invention discloses a method for tracking an object based on an event camera, which includes the steps of: setting one or more feature regions including feature points in an image of an object; updating the position of the feature region by tracking the movement of the feature point according to an event occurring in the image; and updating the shape of the feature region when a difference between a position after the update of the feature region and a position before the update is equal to or greater than a preset threshold.

Description

Event camera-based object tracking device and method {EVENT CAMERA-BASED OBJECT TRACKING APPARATUS AND METHOD}

The present invention relates to an event camera-based object tracking apparatus and method, and more particularly, to an apparatus and method for tracking an object by measuring an event occurring according to movement of a set feature point in an image.

Conventionally, a shutter-based general camera was used to track an object in an image. In this object tracking method, an object to be tracked is tracked for each frame of an image input from a camera. However, the existing shutter-based vision tracking method has problems such as the amount of calculations caused by the increase in tracking objects and the decrease in resolution. Accordingly, a technique for tracking an object using an event camera having a very short delay time, a high resolution on the time axis, and a very wide response range has been proposed compared to conventional shutter-based general cameras. An event camera is a device that records a brightness change value per pixel based on a brightness change event. Although many studies have been conducted on feature point extraction and tracking techniques using only existing images, feature point tracking is not performed between images, and there is a limit to feature point tracking in high-speed and high dynamic range (HDR) situations. Therefore, in order to overcome this problem, tracking of feature points using events has been studied.

The present invention relates to a method for tracking patch-feature points on a plane using DAVIS (Dynamic and Active-pixel Vision Sensor). DAVIS is a type of event camera and has APS (Active Pixel Sensor) and DVS (Dynamic Vision Sensor) that share an optical sensor array. Thus, a general image and an event measurement (light change) can be obtained simultaneously. Then, patch-feature points are tracked using both images and events.

The present invention is also more stable in high-speed and HDR situations than image-based feature point tracking techniques because only feature point extraction depends on images at first and only events are used thereafter. And feature point tracking is a basic component in fields such as SLAM (Simultaneous Localization and Mapping) or object tracking. In particular, since the present invention has dealt with a method for tracking patch-feature points on a plane, it can be applied to video navigation for a landing while looking at the ground.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and a technical task is to provide an apparatus and method for tracking an object by measuring an event occurring according to the movement of a feature point set in an image.

In addition, the present invention provides an event camera-based object tracking device and method configured to more accurately perform feature point tracking even in high-speed and high dynamic range (HDR) situations by improving a warping model that is not suitable for existing moon landing scenarios, etc. to do as a technical task.

The technical problems to be achieved by the present invention are not limited to the above technical problems, and other technical problems of the present invention can be derived from the following description.

As a technical means for solving the above technical problem, an event camera-based object tracking method according to a first aspect of the present invention includes the steps of setting one or more feature regions including feature points in an image of an object, in the image updating the position of the feature region by tracking the movement of the feature point according to an event that occurs, and when the difference between the updated position of the feature region and the position before the update is equal to or greater than a predetermined threshold value, the feature region Updating the form.

In addition, according to the second aspect of the present invention, an event camera-based object tracking device includes a communication module for receiving an image of an object, a memory for storing an image feature tracking program, and a processor for executing the image feature tracking program. . The processor executes the image feature tracking program to set one or more feature regions including feature points in an image of the object, and tracks movement of the feature points according to an event occurring in the image to locate the feature region. and updating the shape of the feature region when a difference between a position after the update and a position before update of the feature region is greater than or equal to a predetermined threshold value.

According to the above-described problem solving means of the present disclosure, an object may be tracked by measuring an event occurring according to a motion of a feature point set in an image. In particular, tracking of feature points can be performed even when various shape deformations of feature regions set on a flat image occur. In addition, the accuracy of feature point tracking can be increased by separately updating the location and shape of the feature region. Furthermore, even if the positions of some feature regions are incorrectly set during the update process, the positions of the corresponding feature regions can be set to correct positions again in the next update process.

The effects of the present invention are not limited to the effects described above, and include all effects understood from the following description.

1 is a block diagram showing the configuration of an event camera-based object tracking device and DAVIS according to an embodiment of the present invention.
2 to 6 are diagrams for explaining a method of tracking features of an image according to an embodiment of the present invention.
7 is a flowchart illustrating a sequence of an event camera-based object tracking method according to an embodiment of the present invention.
8 and 9 are flowcharts illustrating detailed processes of some steps of the event camera-based object tracking method shown in FIG. 7 .

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. However, the present invention may be implemented in many different forms, and is not limited to the embodiments described herein. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the accompanying drawings. In order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and the size, shape, and shape of each component shown in the drawings may be variously modified. Same/similar reference numerals are assigned to the same/similar parts throughout the specification.

The suffixes "module" and "unit" for the components used in the following description are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description is omitted.

Throughout the specification, when a part is said to be “connected (connected, contacted, or combined)” with another part, this is not only the case where it is “directly connected (connected, contacted, or coupled)”, but also has other members in the middle. It also includes the case of being "indirectly connected (connected, contacted, or coupled)" between them. In addition, when a part "includes (provides or provides)" a certain component, it does not exclude other components, but "includes (provides or provides)" other components unless otherwise specified. means you can

Terms indicating ordinal numbers such as first and second used in this specification are used only for the purpose of distinguishing one element from another, and do not limit the order or relationship of elements. For example, a first element of the present invention may be termed a second element, and similarly, the second element may also be termed a first element.

1 is a block diagram showing the configuration of an event camera-based object tracking device 100 and a Dynamic and Active-pixel Vision Sensor (DAVIS) 200 according to an embodiment of the present invention. Hereinafter, the event camera-based object tracking device 100 will be described in detail with reference to FIG. 1 .

The event camera-based object tracking device 100 is configured to receive an image and an event acquired by the DAVIS 200 and track feature points in the image. The event camera-based object tracking device 100 may be implemented as a computer or portable terminal capable of accessing a server or other terminals through a network. Here, the computer includes, for example, a laptop, desktop, laptop, etc. equipped with a web browser, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), LTE (Long Term Evolution) communication-based terminal, smart All types of handheld-based wireless communication devices such as phones and tablet PCs may be included. In addition, the network may include a wired network such as a Local Area Network (LAN), a Wide Area Network (WAN) or a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and the like. It can be implemented in all kinds of wireless networks such as The event camera-based object tracking device 100 may operate in a cloud computing service model such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). In addition, the event camera-based object tracking device 100 may be built in the form of a private cloud, public cloud, or hybrid cloud system.

DAVIS (Dynamic and Active-pixel Vision Sensor) 200 includes an APS (Active Pixel Sensor) and DVS (Dynamic Vision Sensor) sharing the same optical sensor arrangement. That is, the DAVIS 200 has a combination of an event camera of a dynamic vision sensor and a conventional shutter-based camera. Accordingly, when the DAVIS 200 is used, a general image measurement value can be obtained from the APS, and an event measurement value according to a brightness change can be obtained from the DVS at the same time. The DAVIS 200 may determine that an event has occurred when a change in brightness occurs in the image. Here, the event according to the change in brightness occurs in real time whenever a change in light of a certain size is detected in each optical sensor array, unlike a general frame image measured at a certain time interval. As shown in Equation (1) below, each event (e) has information on occurrence pixel coordinates, occurrence time, and polarity (whether the brightness of light increases or decreases). In the drawings, the event camera-based object tracking device 100 and the DAVIS 200 are separately displayed, but the scope of the present invention is not limited thereto. For example, the event camera-based object tracking device 100 and the DAVIS 200 may be mounted in a form coupled to a moving antibody.

- 식 (1)

- formula (1)

To describe the detailed configuration of the event camera-based object tracking device 100 in more detail, the communication module 110 receives an image of an object and an event from the DAVIS 100. The communication module 110 may include a device including hardware and software necessary for transmitting and receiving a signal such as a control signal or a data signal to and from other network devices through a wired or wireless connection.

The memory 120 stores an image feature tracking program that enables the event camera-based object tracking device 100 to track feature points in an image. The name of the image feature tracking program is named for convenience of description, and may be named with a different name, and the function of the program is not limited by the name itself. The memory 120 may store at least one of data input to the communication module 110, data necessary for functions performed by the processor 130, and data generated according to execution of the processor 130. The memory 120 should be interpreted as collectively referring to a non-volatile storage device that continuously maintains stored information even when power is not supplied and a volatile storage device that requires power to maintain stored information. Also, the memory 120 may temporarily or permanently store data processed by the processor 130 . The memory 120 may include magnetic storage media or flash storage media in addition to volatile storage devices that require power to maintain stored information, but the scope of the present invention is not limited thereto. not.

The processor 130 executes an image feature tracking program stored in the memory 130 . The processor 130 may include various types of devices that control and process data. The processor 130 may refer to a data processing device embedded in hardware having a physically structured circuit to perform functions expressed by codes or instructions included in a program. In one example, the processor 140 includes a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), an FPGA ( field programmable gate array), etc., but the scope of the present invention is not limited thereto.

The processor 130 may be configured to perform the following functions by executing an image feature tracking program.

First, the processor 130 sets one or more feature regions including feature points in an image of an object. More specifically, the processor 130 may obtain an image of an object from the DAVIS 200 and extract a Harris corner corresponding to a feature point from the image obtained from the DAVIS. At this time, the processor 130 may set a rectangular feature region of a certain size in the image so that the Harris corner is the center.

Next, the processor 130 updates the position of the feature region by tracking the motion of the feature point according to the event occurring in the image. More specifically, the processor 130 may receive an event according to a change in brightness in the image from the DAVIS 200, track the motion of the feature point based on the coordinate values of the feature point and the feature region, and update the position of the feature region. . In addition, the processor 130 may calculate the amount of change in brightness in the image during the preset time by accumulating the number of events that occurred during the preset time. The processor 130 may measure the gradient of the image from the image acquired by the DAVIS 200 and calculate a brightness change estimate within the image for the preset time based on the measured gradient. At this time, when a line segment on the image is moved, the principle that a brightness change occurs in a pixel where the line segment passes and an event occurs is used. The processor 130 may calculate a warping parameter such that a difference between the amount of change in brightness in the image and the estimated value of the change in brightness in the image is minimized.

The processor 130 may update the shape of the feature region when a difference between a position after the update and a position before the update of the feature region is equal to or greater than a predetermined threshold value. Here, each of the feature point and one or more feature regions may have a specific coordinate value within the image. In one example, the processor 130 updates the shape of the feature region only when the feature region after being updated moves more than a specific number of pixels, which is a preset threshold, compared to before being updated, and the location of the feature region before being updated and the feature after being updated are updated. Based on the coordinate values of the location of the region, the shape of the feature region may be updated using a homography matrix operation technique.

The tracking method of the event camera-based object tracking device 100 will be described in more detail with reference to FIGS.

Referring to FIG. 2 , the event camera-based object tracking apparatus 100 extracts a feature point patch-feature point from an image. Patches and feature points are elements respectively corresponding to the aforementioned feature areas and feature points. The event camera-based object tracking apparatus 100 measures a change in brightness in a patch area by accumulating a certain number of events generated in the patch area, and updates the shape and position of the patch to correspond thereto. As a result, the event camera-based object tracking apparatus 100 may obtain in real time the shape and position of the patch corresponding to the patch-feature area extracted from the first image. 2, 220 denotes a patch-feature point at the current time t. At this time, when events are accumulated for a certain short period of time before and after time t to obtain a change in brightness at each pixel, a form as shown on the right side of the image of FIG. 2 is obtained. The event camera-based object tracking device 100 has a warping parameter (p) of the patch-feature point so that the brightness change obtained from the accumulated events in this patch and the patch-feature point 210 extracted at the left t = 0 correspond. ) is updated.

The warping parameter (p) is t_x, t_y that determines translation in Equation (2) below. When t_x and t_y are changed, the position of the feature point is changed.

- 식(2)

- Equation (2)

는 이벤트 측정치를 이용해서 계산되기 때문에, p, v와 관련 없는 값이나,

는 p와 v에 대해서 값이 변하게 된다. 최적화 과정에서 패치의 모양을 결정하는 H는 상수이다.Optimization of the warping parameter (p) is a process of obtaining the warping parameter (p) and optical flow (v) that minimize Equation (3) below.

Since is calculated using event measurements, values unrelated to p and v,

changes in value for p and v. H, which determines the shape of the patch in the optimization process, is a constant.

- 식(3)

- Equation (3)

이 정해지는데

와 가장 비슷한 값을 가지는 p,v를 구하는 것이 최적화 과정의 목표이다. 최적화 과정은 일반적으로 Gauss-Newton Method 또는 Levenberg-Marquardt Method를 이용해서 수행된다. 본 발명의 실시예에서 Ceres Solver라는 library를 이용할 수 있다. Gauss-Newton을 예를 들어 설명하면,

일 때,

이면

가 되고 식(3)은

이 된다.

은 아래 식 (4)와 같이 계산된다. After all, depending on the value of p, v

is determined

The goal of the optimization process is to find p,v with the most similar value to . The optimization process is generally performed using the Gauss-Newton Method or the Levenberg-Marquardt Method. In an embodiment of the present invention, a library called Ceres Solver can be used. Taking Gauss-Newton as an example,

when,

the other side

and equation (3) is

becomes

is calculated as in Equation (4) below.

- 식 (4)

- Eq. (4)

가 수렴할 때까지 위 두 식을 반복적으로 계산한다.

는 현재 추정하고 있는 p,v의 값일 수 있다. 최종적으로 수렴한 값 p를 이용해서 특징점의 위치를 계산할 수 있다.

The above two equations are iteratively calculated until convergence.

may be values of p and v currently estimated. Finally, the position of the feature point can be calculated using the converged value p.

)와, 이미지 기울기(

와 optical flow(v)를 내적하여 얻은 밝기 변화의 추정치(

)의 차이가 최소화 하도록 최적화를 수행한다 In order to update the Warping parameter (p), the brightness change measurement obtained by accumulating events (

), and the image gradient (

Estimate of brightness change obtained by dot product of and optical flow (v) (

) is optimized to minimize the difference in

와 같은 warping 모델을 사용하지 않고,

와 같은 모델을 사용하였다.Referring to FIG. 3 , the event camera-based object tracking device 100 uses a homography model. The homography matrix has 8 degrees of freedom, and can express the distortion of the patch due to an arbitrary camera movement when the patch 310-feature points 311 are on a plane. However, when the optimization parameter (p) is increased from the existing 3 degrees of freedom to 8 degrees of freedom, the accuracy of optimization decreases and the accuracy of feature point tracking inevitably decreases. Therefore, in the present invention

Without using a warping model such as

The same model was used.

Referring to FIGS. 4 to 6 , images 400 , 500 , and 600 show changes in the position and shape of patch-feature points according to a brightness change event. In the event camera-based patch-feature point tracking process using the event camera-based object tracking device 100, the position of the patch-feature point is updated whenever an event occurs. However, in the event camera-based patch-feature tracking process according to the present invention, the movement transformation (t) of only two degrees of freedom is set as the optimization parameter (p). Therefore, an event occurs in an arbitrary image 400 in which the patch 410-feature point 411 shown in FIG. 4 is set, and the patch 410-feature point 411 is displayed in When the patch 510-feature point 511 is moved, in the optimization process, the shape of the patch 510 is maintained as the shape of the previous patch 410, and only movement conversion occurs. Also, in the event camera-based object tracking device 100, the homography matrix H is updated whenever patches move by a certain pixel or more. The homography matrix H may be calculated using a relationship between a position where a patch was initially extracted and a position of a currently estimated patch. At this time, since RANSAC (Random Sample Consensus), which can remove the effect of outliers, is used, the effect of incorrectly tracked patches can be excluded. Then, H of all patches is updated using the calculated homography matrix (H), and the movement parameter t is initialized to 0. As a result, the patches 610 to 611 shown in FIG. 6 appear in the image 600 .

Referring to an embodiment of the event camera-based object tracking device 100 based on the contents described so far, first, after extracting the Harris corner based on the image generated by the DAVIS 200, a square patch of a certain size is centered on the Harris corner. - Locate the feature point.

Next, whenever an event occurs, all patch-feature points are checked, and the position of the patch-feature point to which the event belongs is updated. At this time, if a certain number of events that have recently occurred in the patch are accumulated, a brightness change occurring in the patch during the corresponding time can be calculated. The warping parameter that minimizes the difference between the brightness change in the patch and the brightness change estimated based on the image gradient is obtained through optimization.

Next, the movement parameters (tx, ty) of the patch-feature point are updated in real time whenever an event occurs. The homography matrix (H) is updated only when the patch-feature points move more than a certain pixel. A homography matrix (H) is calculated using the initial position of the patch-feature point and the current estimated position. At this time, since RANSAC (Random Sample Consensus) is used, the influence of outliers whose positions are incorrectly estimated can be excluded.

Finally, the location and shape of all patches are updated with the calculated homography matrix (H). At this time, the movement parameters (tx, ty) are reset to 0. Since the homography matrix H is calculated after excluding outliers, outlier patches can also be updated to positions close to the true values.

FIG. 7 is a flowchart illustrating a sequence of an event camera-based object tracking method (hereinafter referred to as “event camera-based object tracking method”) according to an embodiment of the present invention, and FIGS. 8 and 9 are an event camera-based object tracking method. It is a flowchart showing detailed processes for some steps of The event camera-based object tracking method may be performed through the event camera-based object tracking device 100 of FIG. 1 and may be performed as the processor 130 executes an image feature tracking program. Accordingly, the contents described above with reference to FIGS. 1 to 6 may be equally applied to the event camera-based object tracking method. Hereinafter, the event camera-based object tracking method will be described in detail with reference to FIGS. 7 to 9 together with the above description.

Referring to FIG. 7 , the event camera-based object tracking method includes setting a feature region ( S110 ), updating the location of the feature region ( S120 ), and updating the shape of the feature region ( S130 ). The feature region setting step ( S110 ) is a step of setting one or more feature regions including feature points in an image of an object. The step of updating the position of the feature region ( S120 ) is a step of updating the position of the feature region by tracking the motion of the feature point according to the event occurring in the image. The step of updating the shape of the feature region ( S130 ) is a step of updating the shape of the feature region when the difference between the position after the update and the position before the update is equal to or greater than a preset threshold value.

In one example, each feature point and one or more feature regions may be configured to have specific coordinate values in an image. At this time, in the step of updating the shape of the feature region (S130), the shape of the feature region is updated only when the feature region after the update moves more than a specific number of pixels, which is a predetermined threshold value, compared to before the update, and the position and update of the feature region before the update are performed. After the process is performed, a step of updating the shape of the feature region by using a homography matrix operation technique based on the coordinate values of the location of the feature region may be performed.

Referring to FIG. 8 , the step of setting a feature region (S110) is a step of acquiring an image of an object from DAVIS (Dynamic and Active-pixel Vision Sensor) (S111), and a Harris corner corresponding to a feature point from the image obtained from DAVIS. A step of extracting (S112), and a step of setting a feature region in the form of a rectangle of a certain size in the image so that the Harris corner is the center (S113) may be included.

Referring to FIG. 9 , in step S120 of updating the position of a feature region, an event according to a change in brightness in an image is received, the motion of the feature point is tracked based on the feature point and the coordinate values of the feature region, and the position of the feature region is determined. This is the renewal step. The step of updating the location of the feature region (S120) is the step of accumulating the number of events that have occurred during a preset time period and calculating the amount of change in brightness in the image for a preset time period (S121). Calculating an estimate of the change in brightness within the image during operation ( S122 ), and calculating a parameter that minimizes a difference between the amount of change in brightness within the image and the estimate of change in brightness within the image ( S123 ).

The above steps may be performed in a predetermined order, but are not necessarily limited thereto. For example, each step may be performed simultaneously or in a different order from the predetermined order. In addition, an event camera-based object tracking method in which some steps are omitted may be implemented.

The event camera-based object tracking method described above may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

Those skilled in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention based on the above description. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The scope of the present invention is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present invention.

The scope of the present application is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present application.

100: event camera based object tracking device
110: communication module
120: memory
130: processor
200: DAVIS (Dynamic and Active-pixel Vision Sensor)

Claims (19)

In the event camera-based object tracking method in the event camera-based object tracking device,
(a) setting one or more feature regions including feature points in an image of an object;
(b) updating the position of the feature region by tracking the motion of the feature point according to an event occurring in the image; and
(c) updating the shape of the feature region when a difference between a position after the update and a position before the update of the feature region is equal to or greater than a predetermined threshold value. According to claim 1,
The feature point and the one or more feature regions are each configured to have a specific coordinate value in the image, the event camera-based object tracking method. According to claim 1,
In step (a),
Acquiring an image of the object from DAVIS (Dynamic and Active-pixel Vision Sensor); and
And extracting a Harris corner corresponding to the feature point from the image obtained from the DAVIS, event camera-based object tracking method. According to claim 3,
In step (a),
The method of tracking an object based on an event camera further comprising the step of setting the feature region in the form of a rectangle of a certain size in the image so that the Harris corner is a center. According to claim 2,
In step (b),
Receiving an event according to a change in brightness in the image from a Dynamic and Active-pixel Vision Sensor (DAVIS), tracking a motion of the feature point based on coordinate values of the feature point and the feature region, and updating the location of the feature region. An object tracking method based on an event camera. According to claim 5,
In step (b),
Accumulating the number of events occurring during a predetermined time period and calculating a change in brightness in the image during the predetermined period of time, the event camera-based object tracking method. According to claim 6,
In step (b),
The method of tracking an object based on an event camera further comprising measuring a gradient of the image and calculating an estimate of a change in brightness in the image for the preset time period based on the gradient. According to claim 7,
In step (b),
Further comprising calculating a warping parameter such that a difference between the amount of change in brightness in the image and the estimated value of change in brightness in the image is minimized. According to claim 1,
In step (c),
The shape of the feature region is updated only when the feature region after being updated moves more than a specific number of pixels, which is the predetermined threshold value, compared to before the update, and the position of the feature region before the update and the position of the feature region after the update are determined. and updating a shape of the feature region by using a homography matrix calculation technique based on coordinate values. In the event camera-based object tracking device,
A communication module for transmitting and receiving an image of an object;
a memory for storing an image feature tracking program; and
A processor executing the image feature tracking program;
The processor executes the image feature tracking program,
Setting one or more feature regions including feature points in an image of the object, tracking the motion of the feature points according to an event occurring in the image, updating the location of the feature region, and updating the updated feature region An event camera-based object tracking device configured to update a shape of the feature region when a difference between a post position and a position before being updated is equal to or greater than a predetermined threshold value. According to claim 10,
The feature point and the one or more feature regions are each configured to have a specific coordinate value in the image, the event camera-based object tracking device. According to claim 10,
The processor executes the image feature tracking program,
Obtaining an image of the object from a dynamic and active-pixel vision sensor (DAVIS), and further extracting a Harris corner corresponding to the feature point from the image obtained from the DAVIS, event camera-based object tracking Device. According to claim 12,
The processor executes the image feature tracking program,
The event camera-based object tracking device configured to further perform setting the feature region in the form of a rectangle of a certain size in the image so that the Harris corner is a center. According to claim 11,
The processor executes the image feature tracking program,
Receiving an event according to a change in brightness in the image from DAVIS (Dynamic and Active-pixel Vision Sensor), tracking the motion of the feature point based on the coordinate values of the feature point and the feature region, and updating the location of the feature region. An object tracking device based on an event camera, configured to perform further. According to claim 14,
The processor executes the image feature tracking program,
An event camera-based object tracking device configured to further perform calculating a change in brightness in the image during the preset time by accumulating the number of events occurring during the preset time. According to claim 15,
The processor executes the image feature tracking program,
and measuring an inclination of the image and calculating an estimate of a change in brightness in the image for the preset time based on the inclination. According to claim 16,
The processor executes the image feature tracking program,
The event camera-based object tracking device further configured to calculate a warping parameter such that a difference between the amount of change in brightness in the image and the estimate of change in brightness in the image is minimized. According to claim 10,
The processor executes the image feature tracking program,
The shape of the feature region is updated only when the feature region after being updated moves more than a specific number of pixels, which is the predetermined threshold value, compared to before the update, and the position of the feature region before the update and the position of the feature region after the update are determined. An event camera-based object tracking device configured to further perform updating a shape of the feature region by utilizing a homography matrix calculation technique based on coordinate values. A non-transitory computer-readable recording medium on which a computer program for performing the event camera-based object tracking method according to any one of claims 1 to 9 is recorded.

Images