CN104346427A - Apparatus and method for analyzing image including event information - Google Patents

Abstract

An apparatus and method for analyzing an image including event information, the apparatus and method determining a pattern of at least one pixel group corresponding to event information included in an input image, and analyzing an outline of an object and an object based on the at least one pattern at least one of the sports.

Description

Apparatus and method for analyzing images including event information

technical field

Methods and apparatus consistent with the exemplary embodiments relate to an apparatus for analyzing an image, and more particularly, to a method and apparatus for analyzing an object included in an input image.

Background technique

Image processing may refer to all forms of information processing in which images are input and output, for example, image processing may include analysis or processing of photographs, videos, and the like.

A device sensing input data for image processing may be a visual sensor, for example, may include a photoelectric sensor based on a technique used to manufacture a semiconductor device, which has become an integrated circuit, or the like.

Contents of the invention

According to an aspect of an exemplary embodiment, there is provided an apparatus for analyzing an image, the apparatus comprising: a classifier configured to receive an event signal corresponding to at least one pixel of an event-based vision sensor, and configured to determining a pattern of a pixel group of a plurality of pixels of the event-based vision sensor, wherein the pixel group includes the at least one pixel and a plurality of adjacent pixels of the event-based vision sensor adjacent to the at least one pixel; the analyzer, by It is configured to determine at least one of the shape of the object and the movement of the object based on the pattern of groups of pixels.

The classifier may determine whether a group of pixels corresponds to at least one edge pattern among a plurality of predetermined edge patterns.

The analyzer may include: a calculator to calculate a speed corresponding to the pixel group based on the pattern of the pixel group; and a motion analyzer configured to analyze the movement of the object based on the speed corresponding to the pixel group.

The apparatus for analyzing an image may further include a processor configured to calculate a change amount of a relative coordinate of a point with respect to a user input based on the movement of the object, and process the user input based on the change amount of the relative coordinate.

According to an aspect of an exemplary embodiment, there is provided an apparatus for analyzing an image, the apparatus comprising: a classifier configured to receive a first event signal corresponding to a first pixel of an event-based vision sensor and An event based on a second event signal of a second pixel of the vision sensor, determining a first pattern of a first pixel group of a first plurality of pixels of the event based vision sensor, and determining a second pixel group of a second plurality of pixels A second pattern, wherein a first pixel group includes the at least one first pixel and a first plurality of neighboring pixels of an event-based vision sensor adjacent to the at least one first pixel, and a second pixel group includes the at least one first pixel. a second pixel and a second plurality of adjacent pixels adjacent to the at least one second pixel; an analyzer for detecting a first position of the object based on the first pattern, detecting a second position of the object based on the second pattern, and based on the first pattern A position and a second position are used to determine the depth of the object.

According to an aspect of an exemplary embodiment, there is provided a method for analyzing an image, the method comprising: receiving an event signal corresponding to at least one pixel of an event-based vision sensor, determining a plurality of pixels of the event-based vision sensor Analyzing at least one of the shape of the object and the movement of the object based on the pattern of the pixel group, wherein the pixel group includes the at least one pixel and an event-based vision sensor adjacent to the at least one pixel multiple neighboring pixels.

According to an aspect of an exemplary embodiment, there is provided a method for analyzing an image, the method comprising: receiving an input of an event signal corresponding to a pixel of an event-based vision sensor indicating movement of an object, selecting and coordinating the event signal a corresponding pixel adjacent to a plurality of neighboring pixels of the event-based vision sensor, determining a position of an edge of an object based on an edge pattern of the plurality of neighboring pixels, and determining an edge pattern of the plurality of neighboring pixels based on an edge pattern of the object The position of the edge to analyze the shape of the object.

Other features and aspects of the exemplary embodiments will be apparent from the detailed description, drawings, and claims.

Description of drawings

FIG. 1 is a block diagram illustrating an apparatus for analyzing an image according to an exemplary embodiment;

2A to 2C are diagrams illustrating a plurality of predetermined edge patterns according to an exemplary embodiment;

3A and 3B are diagrams illustrating a scheme for classifying patterns of pixel groups according to an exemplary embodiment;

FIG. 4 is a diagram illustrating a scheme for determining a direction of an edge of a pixel group according to an exemplary embodiment;

5A and 5B are diagrams illustrating a scheme for analyzing a shape of an object based on an input image, according to an exemplary embodiment;

FIG. 6 is a diagram illustrating a scheme for calculating a velocity corresponding to a pixel group according to an exemplary embodiment;

7 is a diagram illustrating a scheme for analyzing motion of an object using a rigid body model, according to an exemplary embodiment;

8A to 8D are diagrams illustrating a scheme for improving the accuracy of analyzing a motion of an object according to an exemplary embodiment;

FIG. 9 is a diagram illustrating a scheme for processing a user input based on a moving speed of an object, according to an exemplary embodiment;

FIG. 10 is a diagram illustrating a scheme for processing a user input based on a depth of an object, according to an exemplary embodiment;

11 is a flowchart illustrating a method for analyzing an image according to an exemplary embodiment;

FIG. 12 is a block diagram illustrating an apparatus for analyzing a three-dimensional (3D) image according to an exemplary embodiment.

Throughout the drawings and the detailed description, unless otherwise described, the same drawing reference numerals will be understood to refer to the same elements, features and structures. The relative size and depiction of these elements may be exaggerated for clarity, illustration, and convenience.

Detailed ways

The following detailed description is provided to assist the reader in gaining a comprehensive understanding of the methods, devices and/or systems described herein. Accordingly, various changes, modifications, and equivalents of the methods, devices, and/or systems described herein will suggest themselves to those of ordinary skill in the art. The described progression of processing steps and/or operations is an example; however, the order of processing steps and/or operations is not limited to the order set forth herein, and, except for steps and/or operations that must occur in a specific order, may be The order is modified as would be understood by one of ordinary skill in the art. Also, various descriptions of well-known functions and constructions may be omitted for increased clarity and conciseness.

FIG. 1 is a block diagram illustrating an apparatus 100 for analyzing images according to an exemplary embodiment.

Before describing the apparatus 100 for analyzing images with reference to FIG. 1 , an input image to be used by the apparatus 100 will be briefly discussed. An input image according to an exemplary embodiment may refer to an output image of an event-based vision sensor for capturing an object. An event-based vision sensor may asynchronously output an event signal in response to detecting a predetermined event. The predetermined event may include a change in brightness of light incident on the event-based vision sensor. For example, when an event is detected (eg, an event that brightens light in a predetermined pixel), an event-based vision sensor may output a corresponding ON event for the associated pixel, thereby increasing the brightness. Additionally, when an event is detected (eg, an event that dims light in a predetermined pixel), the event-based vision sensor may output an OFF event corresponding to the associated pixel, thereby reducing brightness.

Unlike a frame-based vision sensor, an event-based vision sensor can output a part of pixel data in which a change in light is detected without scanning photodiodes of a plurality of pixels in frame units. Changes in brightness of light incident on the vision sensor may be caused by movement of the object. For example, assume that the light source is substantially fixed, and assume that the object does not spontaneously emit light over time. In this case, light incident to the vision sensor may refer to light generated from a light source and reflected from an object. When the object is not moving, the brightness of the light incident on the event-based vision sensor does not change because there is essentially no change in the light reflected from the stationary object. Conversely, when an object moves, the brightness of incident light to the vision sensor changes due to reflection of light from the object, and thus light incident to the event-based vision sensor changes based on the movement of the object.

Event-based vision sensors may include dynamic vision sensors. Dynamic vision sensors may include artificial vision sensors that operate on principles of the human retina and optic nerve. In response to the movement of the object, the dynamic vision sensor can output an event signal to the event-based vision sensor. Event signals may include information that is generated asynchronously in response to movement of an object. Event signals may include information such as optic nerve information transmitted from the retina to the human brain. For example, an event signal may be generated when a moving object is detected, but may not be generated for a stationary object. At least one pixel included in the event signal may correspond to an object whose movement is detected.

Referring to FIG. 1 , an apparatus 100 for analyzing an image may include a classifier 110 and an analyzer 120 . The classifier 110 may classify the pattern of at least one pixel group based on the input image including the event signal in which the movement of the object is detected. The at least one pixel group may include a pixel corresponding to the event signal and a plurality of pixels adjacent to the corresponding pixel.

Hereinafter, for convenience of description, a pixel group may include a 3×3 matrix of 9 pixels, assuming that a pixel corresponding to an event signal is arranged at the center of the pixel group, and assuming that 8 adjacent pixels arranged around the corresponding pixel are included in the pixel group. This scheme for configuring pixel groups is only exemplary, and the scheme for configuring pixel groups can be modified in various ways.

The classifier 110 may determine whether a pixel group corresponding to the event signal (ie, a group of pixels where the event occurred) corresponds to a plurality of predetermined edge patterns. For example, referring to FIG. 2A , the plurality of predetermined edge patterns may include 24 edge patterns P ₁ to P ₂₄ . The 24 edge patterns _P1 to _P24 may refer to patterns associated with edges of objects. When the pixel group corresponding to the event signal is determined to correspond to any one of the plurality of predetermined edge images, the classifier 110 may determine the pattern of the pixel group corresponding to the event signal as the corresponding edge pattern. For example, based on a determination result that the pixel group corresponding to the event signal corresponds to the edge pattern P ₁ of FIG. 2A , the classifier 110 may classify the pixel group corresponding to the event signal into the edge pattern P ₁ . The classifier 110 may discard groups of pixels corresponding to event signals that are not associated with any one of the plurality of predetermined edge patterns.

A detailed description related to a scheme in which the classifier 110 classifies patterns of pixel groups corresponding to event signals will be discussed with reference to FIGS. 2A to 3B .

The analyzer 120 may analyze at least one of an object's appearance (such as a shape, contour, or position of the object relative to the position of the event-based sensor) and the motion of the object based on the pattern of the at least one pixel group classified by the classifier 110 . The analyzer 120 may determine a direction of an edge corresponding to the at least one pixel group using the pattern of the at least one pixel group, thereby analyzing the shape of the object. Alternatively, the analyzer 120 may calculate a velocity corresponding to a pixel group associated with an edge of the object, and analyze the motion of the object based on the calculated velocity of the pixel group. The analyzer 120 may determine at least one of a movement speed component of the object, a rotation speed component of the object, and a scaling speed component of the object to analyze the motion of the object.

A detailed description related to the operation of the analyzer 120 will be discussed later with reference to FIGS. 4 to 8D .

2A to 2C are diagrams illustrating a plurality of predetermined edge patterns according to an exemplary embodiment. Referring to FIG. 2A, a plurality of edge patterns may be associated with an edge of an object.

The event signal may include a timestamp of when the predetermined event was detected, an indicator indicating the type of event, and an index of the pixel at which the predetermined event was detected. As discussed below, time stamps corresponding to pixels of resolution may be stored in a table in memory, and a time signal of the pixel's event time may then be utilized.

The apparatus for analyzing an image according to an exemplary embodiment may classify a pattern of a pixel group based on a difference between a time stamp of a pixel at which an event is detected and time stamps of a plurality of neighboring pixels. Devices for analyzing images can determine various types of neighboring pixels to classify patterns of groups of pixels. A device for analyzing an image may calculate a difference between a timestamp of a pixel at which an event was detected and timestamps of a plurality of neighboring pixels, and determine one or more types of neighboring pixels based on a result of the calculation.

A device for analyzing an image may use a data structure for managing time stamps corresponding to all pixels. When an event signal is detected, the device for analyzing the image may update the time stamps of the pixels included in the event signal. At this point, the device for analyzing the image may discard previously stored information and store newly updated information. When a current event is detected, the apparatus for analyzing an image may update a value of a time stamp of a current pixel corresponding to the current event to a current time. The apparatus for analyzing the image may determine the pixel type of the neighboring pixel by calculating the difference between the updated timestamp of the current pixel corresponding to the current event and the timestamp of the neighboring pixel. The time stamps of neighboring pixels may be updated when corresponding previous events with neighboring pixels are detected.

The device for analyzing the image may determine the type of neighboring pixels based on Equation 1.

[equation 1]

${\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; ev</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; nx</span>}}\left\{\begin{array}{c}<span\; class="notranslate">\; \&Greater\; Equal;</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; E.</span>}}<span\; class="notranslate">\; \&Right\; Arrow;</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; type</span>}\\ <span\; class="notranslate">\; \&le;</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; \&Right\; Arrow;</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; type</span>}\end{array}\right.$

Here, t _ev represents the timestamp of the pixel that generated the event, and t _nx represents the timestamp of neighboring pixels, where the difference between t _ev and t _nx indicates the temporal correlation between pixel events, which is used for Indicates the directional movement of the edge; T _E represents the threshold used to determine the E-type (E type), which can correspond to slow events; T _S represents the threshold used to determine the S-type (S type), which can Corresponds to fast events. _TE and _TS can be set based on the sensitivity of the pixel or the application to be used. For example, when an object whose movement is to be detected corresponds to a user's hand, _TE and _TS may be set in a range of milliseconds (ms) to several tens of ms. Alternatively, _TE and _TS may be set to several microseconds (μs) or less when an object whose movement is to be detected corresponds to an object that moves significantly faster than the user's hand. T _E and T _S can be set to different values (where T _S <T _E , as shown in Table 1), and if necessary, can be set to equal values.

When a predetermined period of time elapses from a first time point at which a previous event is detected to a subsequent second time point at which a current event is detected in a neighboring pixel, the apparatus for analyzing an image may determine that the neighboring pixel is an E-type. For example, a neighboring pixel in which a new event has not been detected during a predetermined period of time (eg, _TE ) among pixels neighboring a pixel in which a current event is detected may be classified as an E-type neighboring pixel.

When a current event is detected in a neighboring pixel within a predetermined time period from a time point at which a previous event was detected, the apparatus for analyzing an image may determine the neighboring pixel as an S-type. For example, a neighboring pixel in which a new event is detected within a predetermined time period (eg, T _S ) among pixels neighboring a pixel in which a current event is detected may be classified as an S-type neighboring pixel.

The predetermined edge pattern may include adjacent pixels. For example, as shown in FIG. 2A, when the top, bottom, left, and right pixels closest to the event-generating pixel are used, the predetermined edge pattern may include adjacent pixels n1 to n8. Combinations of types of adjacent pixels for configuring a predetermined edge pattern may be different from each other.

For example, the edge pattern _P1 may include pixels n1, n2, and n4 210 of E-type and pixels n3 and n6 220 of S-type. The device for analyzing the image may classify as an edge pattern P ₁ a set of pixels comprising the directional neighbors of pixels n1, n2, and n4 210 for which no new event has been detected during a predetermined period of time (e.g., T _E ). pixel, and neighboring pixels in the direction of pixel n3 and n6 220 where a new event is detected within a predetermined time period (eg, T _S ). In this case, the apparatus for analyzing an image may analyze a direction of an edge of a corresponding pixel group using the S-type pixels n3 and n6 for which a new event is detected within a predetermined time period (eg, T _S ). This is because, when a new event is generated based on the movement of the object, the event occurring at the position of the pixel included in the edge of the object can be detected at substantially the same point in time. As will be described in detail with reference to FIG. 4 , the edge pattern _P1 may be mapped to an edge in the direction of a line connecting the pixels n3 and n6 220 .

In a similar manner, the edge pattern _P24 may include pixels n5, n7, and n8240 of E-type and pixels n3 and n6250 of S-type. The device for analyzing the image may classify, as an edge pattern P ₂₄ , the group of pixels comprising the directional neighbors of pixels n5, n7, and n8 240 for which no new event has been detected during a predetermined period of time (e.g., _TE ). pixel, and neighboring pixels in the direction of pixel n3 and n6 250 where a new event was detected during a predetermined time period (eg, T _S ). In this case, the apparatus for analyzing an image may analyze a direction of an edge of a corresponding pixel group using the S-type pixels n3 and n6 250 that detect a new event within a predetermined period of time (eg, T _S ). As will be described in detail with reference to FIG. 4 , the edge pattern _P24 may be mapped to an edge in a direction of a line connecting the pixels n3 and n6 250 .

The apparatus for analyzing an image according to another exemplary embodiment may utilize more neighboring pixels than the 8 neighboring pixels shown in FIG. 2A . For example, a device for analyzing an image may use a 5x5 pixel matrix of 24 adjacent pixels (as shown in Figure 2B) or a 7x7 pixel matrix of 48 adjacent pixels (as shown in Figure 2C). The exemplary embodiments in FIGS. 2A , 2B, and 2C are merely exemplary, and may be modified in various ways. The apparatus for analyzing an image may compare types of adjacent pixels included in the pixel group with types of adjacent pixels included in a plurality of predetermined edge patterns, and determine the pattern of the pixel group as the plurality of predetermined edge patterns any of the . Based on the results of the comparison, the device for analyzing the image may determine an edge pattern matching the pixel group from among a plurality of predetermined edge patterns. To determine a match, the device for analyzing the image may compare the type of neighboring pixels included in the group of pixels with the types of neighboring pixels included in the plurality of predetermined edge patterns. In one example, when the first edge pattern and the second edge pattern are included in a plurality of predetermined edge patterns, the device for analyzing the image may compare the types of adjacent pixels included in the pixel group with those included in the first edge pattern The type of neighboring pixels in the comparison. Furthermore, the apparatus for analyzing the image may compare the type of adjacent pixels included in the pixel group with the type of adjacent pixels included in the second edge pattern. The apparatus for analyzing an image may determine the pattern of the pixel group to be that of the first edge pattern when the type of adjacent pixels included in the pixel group corresponds to the type of adjacent pixels included in the first edge pattern. Alternatively, the apparatus for analyzing an image may determine the pattern of the pixel group to be the second edge pattern when a type of adjacent pixels included in the pixel group corresponds to a type of adjacent pixels included in the second edge pattern. The apparatus for analyzing an image may determine an edge pattern having adjacent pixels corresponding to adjacent pixels included in the pixel group as the pattern of the pixel group.

When necessary, a part of adjacent pixels included in a predetermined edge pattern may be agnostically designated as a "non-attention" type. Such pixels are neither classified as E-type nor S-type. For example, edge pattern _P1 may include pixels n5, n7, and n8230 of type "not of interest". The edge pattern _P24 may include pixels n1, n2, and n3 260 of type "not of interest".

The apparatus for analyzing an image may classify the pattern of the pixel group using adjacent pixels that do not correspond to the 'not of interest' type among adjacent pixels included in the edge pattern. In other words, the apparatus for analyzing an image may classify patterns of pixel groups using only those pixels classified into E-type and S-type. For example, the device for analyzing an image may disregard pixels n5, n7, and n8230 when determining whether a group of pixels corresponds to edge pattern _P1 . Similarly, the device for analyzing the image may disregard pixels n1 , n2 and n4 260 when determining whether a group of pixels corresponds to edge pattern P ₂₄ .

The plurality of predetermined edge patterns can be stored in various ways. For example, E-type neighboring pixels and S-type neighboring pixels included in 24 edge patterns _P1 to _P24 may be stored in a bit value format as shown in Table 1.

[Table 1]

Here, PnE denotes an E-type neighboring pixel included in the edge pattern _Pn . When it is assumed that 8 neighboring pixels are used, PnE may be configured as 8 bits which may respectively correspond to pixels n1 to n8. A bit corresponding to a neighboring pixel of E-type among 8 bits may be set to '1', and other remaining bits (S-type or 'not of interest' type) may be set to '0'. For example, the edge pattern _P1 may include pixels n1, n2, and n4 210 that are adjacent pixels of E-type, so the bit value of P1E may be set to "11010000" where the first bit, the second bit, and the fourth bit are "1". ". The bit value "11010000" of P1E can be expressed as a hexadecimal number, and in this case, P1E can be expressed as "D0". When 24 adjacent pixels are used according to another exemplary embodiment, PnE may be configured with 24 bits, and when 48 adjacent pixels are used, PnE may be configured with 48 bits.

Also, PnS denotes an S-type neighboring pixel included in the edge pattern _Pn . When it is assumed that 8 adjacent pixels are used, PnS may be configured as 8 bits respectively corresponding to pixels n1 to n8. A bit corresponding to a neighboring pixel of S-type among 8 bits may be set to '1', and the other remaining bits (E-type or 'non-interest' type) may be set to '0'. For example, the edge pattern _P1 may include pixels n3 and n6220 which are adjacent pixels of S-type, and thus the bit value of P1S may be set to '00100100' with the third and sixth bits being '1'. The bit value "00100100" of P1S can be expressed as a hexadecimal number, and in this case, P1E can be expressed as "24". When 24 adjacent pixels are used according to another exemplary embodiment, PnS may be configured with 24 bits, and when 48 adjacent pixels are used, PnS may be configured with 48 bits.

The apparatus for analyzing the image may check whether the adjacent pixels indicated by PnE correspond to the E-type, and whether the adjacent pixels indicated by PnS correspond to the S-type, and determine whether the group of pixels corresponds to the edge pattern P based on the analyzed pixels. _n corresponding.

A device for analyzing an image may disregard neighboring pixels of the "non-interest" type when a pattern of groups of pixels is classified. Accordingly, an apparatus for analyzing an image may not use information that explicitly indicates neighboring pixels of a "non-interest" type. For example, in both PnE and PnS, bits corresponding to neighboring pixels of the “non-interest” type in the edge pattern _Pn may be set to “0”. However, a "0" bit associated with the result of performing a bitwise OR operation on PnE and PnS may represent a neighboring pixel of the "non-correlated" type. For example, when a logical OR operation in units of bits is performed on P1E="11010000" and P1S="00100100", P1E OR P1S="11110100". Since the "0" bit in P1E OR P1S may correspond to the fifth, seventh, and eighth bits, P1E OR P1S = "11110100" may indicate the "non-concern" type included in the edge pattern _P1 Neighboring pixels may be pixels n5, n7, and n8230.

Table 1 is an exemplary embodiment for representing E-type neighboring pixels and S-type neighboring pixels included in the edge pattern _Pn . Those skilled in the art will understand that various modifications are made to Table 1 to represent the E-type neighboring pixels and the S-type neighboring pixels included in the edge pattern _Pn .

3A and 3B are diagrams illustrating a scheme for classifying patterns of pixel groups according to an exemplary embodiment.

Referring to FIG. 3A , 24 edge patterns P ₁ to P ₂₄ may be grouped into 6 groups 310 , 320 , 330 , 340 , 350 and 360 . For example, 24 edge patterns P ₁ to P ₂₄ may be grouped into 6 groups 310 to 360 based on whether adjacent pixels of E-type co-exist. The group 310 may be configured by edge patterns P ₁ , P ₂ , P ₆ , P ₇ and P ₈ including n1 , n2 and n4 of E-type. The group 320 may be composed of edge patterns P ₄ , P ₅ , P ₉ , P ₁₀ , and P ₁₃ , wherein each of the edge patterns P ₄ , P ₅ , P ₉ , P ₁₀ , and P ₁₃ includes an E-type pixel n2, n3 and n5. As shown in FIG. 3A , common pixels in a group among pixels represented as E-type in FIGS. 2A to 2C may be represented as a cross-hatch pattern.

As shown in Table 2, the 6 groups and the edge patterns included in the 6 respective groups can be divided into masked bit values (E) and extra bit values (G).

[Table 2]

The apparatus for analyzing an image may check whether a neighboring pixel at a position corresponding to the mask bit value (E) among neighboring pixels of a pixel group corresponds to E-type to determine which group a pattern of the pixel group corresponds to. The apparatus for analyzing the image may check whether adjacent pixels at positions corresponding to the corresponding group of extra bit values (G) correspond to E-type to determine the pattern of the pixel group.

For example, because the edge patterns P ₁ , P ₂ , P ₆ , P ₇ , and P ₈ included in the group 310 include pixels n1, n2, and n4 of E-type, the first mask bit E1 representing the group 310 may be bit, the second bit and the fourth bit are set to "1". The apparatus for analyzing an image may verify whether the pixels n1 , n2 , and n4 correspond to neighboring pixels of E-type using the mask bit E1 to determine whether the group of pixels is included in the group 310 .

Also, the apparatus for analyzing an image may determine an edge pattern corresponding to a pixel group classified into the group 310 using the bit values G11 , G12 , and G13 . For example, the bit value G11 may refer to an extra bit value in which the sixth bit is set to "1". The device for analyzing the image may use G11 to verify whether pixel n6 of the pixel group corresponds to the E-type.

The apparatus for analyzing an image may determine that the pixel group corresponds to the edge pattern P ₆ or the edge pattern P ₇ based on a determination result that the pixel n 6 of the pixel group corresponds to the E-type. In addition, the apparatus for analyzing an image may verify whether the pixel n3 of the pixel group corresponds to the E-type using the bit value G12 to determine which pattern of the edge pattern _P6 and the edge pattern _P7 the pixel group corresponds to. For example, when the pixel n3 of the pixel group corresponds to the E-type, the apparatus for analyzing the image may determine that the pixel group corresponds to the edge pattern _P7 . Conversely, when the pixel n3 of the pixel group does not correspond to the E-type, the apparatus for analyzing the image may determine that the pixel group corresponds to the edge pattern _P6 .

Based on the determination that the pixel n6 of the pixel group does not correspond to the E-type, the apparatus for analyzing the image may determine that the pixel group corresponds to the edge pattern P ₁ , P ₂ or P ₈ . In addition, the device for analyzing the image may use the bit value G12 to verify whether the pixel n3 of the pixel group corresponds to the E-type, and when the pixel n3 of the pixel group does not correspond to the E-type, the device for analyzing the image may determine that the pixel Group corresponds to edge pattern _P1 . When n3 of the pixel group corresponds to E-type, the apparatus for analyzing the image may use G13 to verify whether pixel n5 of the pixel group corresponds to E-type. When the pixel n5 of the pixel group corresponds to the E-type, the device for analyzing the image can determine that the pixel group corresponds to the edge pattern _P8 , and when the pixel n5 does not correspond to the E-type, the device for analyzing the image can determine that the pixel Group corresponds to edge pattern _P2 .

The apparatus for analyzing an image may determine that the pattern of the pixel group belongs to the group 350 or the group 360 when adjacent pixels at positions corresponding to the mask bit values of E1 to E4 do not correspond to the E-type. The apparatus for analyzing an image may verify whether adjacent pixels at positions corresponding to extra bit values of the bit values G51 and G52 correspond to S-type, and determine which group among the group 350 and the group 360 the pixel group belongs to.

In one example, an apparatus for analyzing an image may use a bit value G51 to determine whether a group of pixels belongs to group 360, where bit value G51 may refer to an additional bit value with the second bit and the seventh bit set to "1" . The apparatus for analyzing an image may verify whether the pixels n2 and n7 of the pixel group correspond to the S-type using the bit value G51 to determine whether the pixel group belongs to the group 360 . Additionally, the apparatus for analyzing the image may also verify (following the exclusion of pixels n2 and n7) whether the remaining single neighboring pixel corresponds to the E-type to determine which of the edge patterns P ₁₁ and P ₁₄ the pixel group corresponds to corresponding.

In addition, the apparatus for analyzing an image may determine whether the pixel group belongs to the group 350 using a bit value G52, where the bit value G52 may refer to an additional bit value in which the fourth bit and the fifth bit are set to "1". The apparatus for analyzing an image may verify whether the pixels n4 and n5 of the pixel group correspond to the S-type, and determine whether the pixel group belongs to the group 350 . In addition, the device for analyzing the image may also verify (following the exclusion of pixels n4 and n5) whether the remaining single neighboring pixel corresponds to the E-type, and determines among edge pattern _P3 and edge pattern _P22 to be classified as A pattern of patterns of pixel groups.

Referring to FIG. 3B , the apparatus for analyzing an image may classify an edge pattern of a corresponding pixel group using neighboring pixels of an E-type included in a pixel group. As will be discussed later, the apparatus for analyzing an image may classify edge patterns of groups of pixels when the threshold _TE for E-type neighboring pixels is equal to the threshold _TS for S-type neighboring pixels.

More specifically, the apparatus for analyzing an image may check whether neighboring pixels b ₀ to b ₇ correspond to neighboring pixels of E-type. When a plurality of adjacent pixels correspond to E-type, the apparatus for analyzing an image may set a bit corresponding to a relevant pixel to '1', and otherwise, set a bit corresponding to a relevant pixel to '0'. The device for analyzing images may calculate P-val based on Equation 2.

[equation 2]

P-val＝(B<<1) AND B AND(B>>1)

Here, B represents an 8-bit bit value, for example, b ₀ b ₁ b ₂ b ₃ b ₄ b ₅ b ₆ b ₇ ; (B<<1) means that by shifting the 8-bit B bit value to the left in units of bits The value obtained by cyclically shifting the degree of 1 bit, for example, b ₁ b ₂ b ₃ b ₄ b ₅ b ₆ b ₇ b ₀ ; (B>>1) means that by shifting the bit value of 8 bits B to The value obtained by right cyclically shifting the degree of 1 bit, for example, b ₇ b ₀ b ₁ b ₂ b ₃ b ₄ b ₅ b ₆ ; (B>>1) A bit value obtained by performing a logical AND operation.

As shown in Table 3, an apparatus for analyzing an image may determine an edge pattern from the calculated P-val using a look-up table (LUT). For example, when the calculated P-val is "00000001", the apparatus for analyzing the image may determine that the relevant pixel group corresponds to the edge pattern _P11 . When the calculated P-val is a value not present in the LUT of Table 3, the apparatus for analyzing an image may determine that the calculated P-val does not correspond to any of the predetermined edge patterns.

[table 3]

When P-val is equal to 17, 34, 68 and 136 decimal, two edge patterns can be candidates. A P-val corresponding to two edge patterns is considered to be the case where adjacent pixels of the "non-interest" type correspond to adjacent pixels of the E-type, and the device for analyzing the image may select one of the possible edge patterns based on predetermined rules. anyone. For example, the device for analyzing the image may determine the type of the additionally determined neighboring pixels and select any one of the edge patterns from the two edge patterns. Alternatively, the device for analyzing the image may randomly select any one of the edge patterns.

FIG. 4 is a diagram illustrating a scheme for determining a direction of an edge of a pixel group according to an exemplary embodiment. Referring to FIG. 4, a plurality of predetermined edge patterns may be mapped to edges having predetermined directions.

A plurality of predetermined edge patterns may be mapped to an edge having a main direction along which neighboring pixels of the S-type are arranged. For example, edge patterns P ₁ , P ₇ , P ₁₈ , and P ₂₄ 410 may be mapped to an edge 415 having an E2-th direction. In addition, edge patterns P ₁₁ , P ₁₂ , P ₁₃ , and P ₁₄ 420 may be mapped to an edge 425 having an E4th direction. As shown in Table 4, 24 edge patterns can be mapped to edges along 8 directions.

[Table 4]

P1 E2 P13 E4 P2 E1 P14 E4 P3 E0 P15 E5 P4 E7 P16 E6 P5 E6 P17 E0 P6 E3 P18 E2 P7 E2 P19 E3 P8 E0 P20 E6 P9 E6 P21 E7 P10 E5 P22 E0 P11 E4 P23 E1 P12 E4 P24 E2

A device for analyzing an image may classify a pattern of groups of pixels corresponding to pixels in which an event was detected. A number of classified patterns can be mapped to the edges of Table 4, and thus, a device for analyzing the image can identify the direction of the edge in the number of pixels where the event was detected. For example, edge patterns mapped using Table 4 may be stored as edges in multiple events, and the device for analyzing the image may combine the stored edge information for multiple pixels to determine the shape of the object.

More specifically, the movement of the user's hand is an example of a case where an apparatus for analyzing an image receives an event signal generated by the movement. In this case, the event pixels may include both pixels corresponding to the edge of the user's hand and pixels corresponding to the inside of the user's hand. A device for analyzing an image may determine an edge pattern corresponding to a plurality of the event pixels. Here, the predetermined edge pattern may include an edge pattern corresponding to the edge. Accordingly, the apparatus for analyzing the image may determine that pixels corresponding to the inside of the hand do not correspond to any of the predetermined edge patterns, and determine that pixels corresponding to the edge of the user's hand correspond to any of the predetermined edge patterns. When the plurality of pixels corresponding to the edge of the human hand are determined to correspond to any one of the predetermined edge patterns, the apparatus for analyzing the image may, based on Table 4, at the plurality of pixels corresponding to the edge of the human hand Determine the orientation of the edge corresponding to the associated edge pattern in . As a result, the apparatus for analyzing the image may determine the direction of the edge among a plurality of pixels corresponding to the edge of the person's hand, and determine the shape of the user's hand by integrating the directions of the plurality of edges.

5A and 5B are diagrams illustrating a scheme for analyzing a shape of an object based on an input image, according to an exemplary embodiment.

Referring to FIG. 5A , the input image may be the output of an event-based vision sensor that captures 8 connecting rods arranged around the center of the cylinder rotating in a clockwise manner at equal rotational speeds. In this case, an event-based vision sensor can output an event signal by detecting brightening events and dimming events. For example, an event-based vision sensor may output an event signal by detecting the degree to which brightness of a plurality of pixels in an image has increased or decreased by more than a predetermined value via 8 linkages rotating in a clockwise direction. In FIG. 5A, a black dot (■) may refer to the output of the sensor detecting a dimming event in which the brightness is reduced by at least the predetermined value; a white dot (□) may refer to the detection of a brightening event The output of , wherein, in a brighten event, the brightness is increased by at least the predetermined value. Referring to FIG. 5B , the apparatus for analyzing an image may analyze a shape of an object using the input image of FIG. 5A .

The apparatus for analyzing an image may select a pixel group corresponding to a predetermined edge pattern based on the scheme described through FIGS. 1 to 4 and estimate a shape of an object based on a direction of an edge corresponding to the selected pixel group. Indirectly, the apparatus for analyzing an image can effectively remove noise included in an input image due to smearing or the like.

In addition to the shape of the object, the device for analyzing the image may also analyze the motion of the object. The apparatus for analyzing an image may calculate velocities at a plurality of pixels corresponding to an edge of the object, and analyze a motion of the object using the velocities at the plurality of pixels corresponding to the edge after analyzing a shape of the object from an event signal. Hereinafter, the operation of the apparatus for analyzing an image to calculate velocities at a plurality of pixels corresponding to the edge of the object will be described with reference to FIG. operate.

FIG. 6 is a diagram illustrating a scheme for calculating a velocity corresponding to a pixel group according to an exemplary embodiment. Referring to FIG. 6 , a pixel group may include motion direction information, and an apparatus for analyzing an image may calculate a velocity corresponding to the pixel group using neighboring pixel groups.

Here, the apparatus for analyzing an image may calculate a velocity of a pixel group corresponding to an edge of an object. For example, the apparatus for analyzing an image may calculate the velocity of a corresponding pixel group for a group of pixels classified into the predetermined edge patterns _P1 to _P24 of FIGS. to calculate the velocity of the corresponding pixel group. As previously described, since the predetermined edge patterns _P1 to _P24 may include edge patterns corresponding to edges of objects, the apparatus for analyzing an image may calculate velocities of pixel groups corresponding to edges of objects.

The apparatus for analyzing an image may calculate an x-axis direction velocity V _x and a y-axis direction velocity V _y corresponding to a pixel group based on Equation 3.

[equation 3]

$\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\end{array}\right)<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; type</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 8</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; dx</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; dt</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ {\mathrm{<span\; class="notranslate">\; dy</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; dt</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}\right){<span\; class="notranslate">\; ,</span>}_{{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&Integral;</span>}_{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; a</span>}}}^{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}}}\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; d\&theta;</span>}<span\; class="notranslate">\; |</span>}$

θi _,a and θi _,b denote the boundary angle of the i-th neighboring pixel covering the S-type based on the center of the pixel group. For example, when pixel n5 is S-type, _θ5,a 620 and θ5 _,b 610 may be the boundary angles covering pixel n5 based on the center of the pixel group.

The device for analyzing the image may mitigate the sensitivity to noise of the time stamp based on Equation 4.

[equation 4]

$\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\end{array}\right)<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; type</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 8</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; dx</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; dt</span>}<span\; class="notranslate">\; ></span>\\ {\mathrm{<span\; class="notranslate">\; dy</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; dt</span>}<span\; class="notranslate">\; ></span>\end{array}\right)<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; \&Integral;</span>}_{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; a</span>}}}^{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}}}\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; d\&theta;</span>}<span\; class="notranslate">\; |</span>$

Here, <dt> represents the same value as Equation 5.

[equation 5]

The apparatus for analyzing an image may store velocities corresponding to pixel groups, and the velocities may include x-axis direction velocity V _x and y-axis direction velocity V _y calculated through Equations 3 to 5 . The velocity corresponding to the pixel group may refer to the velocity of the event pixel located at the center of the corresponding pixel group. As previously described, the apparatus for analyzing an image may calculate an x-axis direction velocity V _x and a y-axis direction velocity V _y for an event pixel corresponding to an edge of an object using a predetermined edge pattern. Hereinafter, a method for analyzing a motion of an object will be described using a velocity of an event pixel corresponding to an edge of the object with reference to FIG. 7 .

FIG. 7 is a diagram illustrating a scheme of analyzing motion of an object using a rigid body model, according to an exemplary embodiment. Referring to FIG. 7 , an apparatus for analyzing an image may analyze a 4-degree-of-freedom (4-DOF) motion of an object 700 .

For example, according to an exemplary embodiment, an input image in which movement of the object 700 is detected is used. The object 700 can move at a moving velocity V _p 740 on a two-dimensional (2D) surface. Optionally, the object 700 may rotate at an angular velocity ω721 based on the center of rotation _OC 720 . Optionally, the object 700 may be expanded or contracted to a scaling velocity V _z based on the scaling center O _z 730 .

The apparatus for analyzing an image may analyze a movement speed component, a rotation speed component, and a scaling speed component of the object 700 . The apparatus for analyzing an image may calculate a velocity V _i existing at a predetermined point P _i 710 on the edge of the object 700 using the description provided with reference to FIGS. 1 to 6 . For example, the velocity V _i may refer to the x-axis direction velocity V _x and the y-axis direction velocity V _y calculated based on Equation 3 to Equation 5 .

The device used to analyze the image can model the velocity _Vi , as shown in Equation 6.

[equation 6]

V _zi +V _ri +V _p =V _i

Here, V _zi 731 , V _ri 722 , and V _p 740 may refer to scaling speed components, rotation speed components, and movement speed components at the point P _i 710 . As shown in Equation 6, the device for analyzing the image may model the velocity _V to decompose the velocity _V at a point P _i 710 disposed on the edge of the object 700 into scaled velocity components, Rotation velocity component and movement velocity component. Here, Equation 6 may be defined as Equation 7.

[equation 7]

tP _i +ωA(P _i -O _c )+V _p ＝V _i

Here, tP _i represents the scaling velocity component, and the coordinate P _i 710 with O _z 730 as the origin can represent the direction and magnitude of the vector V _zi 731, and the parameter t represents that scaling can be performed on the magnitude of the vector V _zi 731; ωA(P _i − O _c ) represents the rotation velocity component _, and the coordinate difference (P _i -O _c ) can represent the direction and magnitude of the vector from the rotation center O _c 720 toward the coordinate P _i 710; The vector of coordinates _Pi 710 is rotated by a rotation matrix, e.g., the matrix $A=\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right).$ The rotated vector due to matrix A may point to vector V _ri 722 , and the parameter ω may perform scaling on the magnitude of vector V _ri 722 .

The apparatus for analyzing an image may calculate a scaling speed component parameter t, a rotation speed component parameter ω, a rotation center O _c , and a movement speed component V _p based on Equation 7. This is because the device for analyzing the image knows the coordinates P _i 710 of points located on the edge and the velocity V _i at the corresponding points. The apparatus for analyzing an image may analyze at least one of a movement speed component, a rotation speed component, and a scaling speed component (for example, 4-DOF of an object).

The method for calculating the scaling velocity component parameter t, the rotation velocity component parameter ω, the rotation center _Oc , and the movement velocity component _Vp based on Equation 7 can be implemented in various ways. According to an exemplary embodiment, Equation 8 may be derived from Equation 7.

[Equation 8]

$\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&omega;A</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}$

Here, P _i represents the coordinates of the i-th point located on the edge of the object 700, represents an average value of coordinates of points located on the edge of the object 700 . V _i represents the velocity at the i-th point located on the edge of the object 700, represents the average value of the velocities at points located on the edge of the object 700 . A number of variables can be defined in Equation 9 to Equation 12.

[equation 9]

P _i = (x _i , y _i )

[equation 10]

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>$

[equation 11]

V _i = (V _xi , V _yi )

[Equation 12]

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; xi</span>}}<span\; class="notranslate">\; ,</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; yi</span>}}<span\; class="notranslate">\; )</span>$

The apparatus for analyzing an image may calculate the x-axis direction velocity V _x and the y-axis direction velocity V _y based on Equation 3 to Equation 5, and store the calculated V _x and V _y as the velocity V at the pixel P _{i i} . A device for analyzing an image may use the coordinates P _i of a plurality of pixels and the velocities V _i of a plurality of pixels to calculate and A device for analyzing an image may use a plurality of _Pi , a plurality of V _i , and Equation 8 to calculate parameter t and parameter ω. For example, Equation 13 and Equation 14 can be derived from Equation 8 based on the pseudo-inverse scheme.

[Equation 13]

$\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; )</span>}$

[Equation 14]

$\mathrm{<span\; class="notranslate">\; \&omega;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; )</span>}$

Here, σ ² (P)=σ ² (x)+σ ² (y), σ(·) represents the operator used to calculate the standard deviation; σ(x,y)=E[(xE[x])( yE[y])]; E[·] represents the expected value or average value. The apparatus for analyzing an image may calculate a scaling speed component parameter t and a rotation speed component parameter ω based on Equation 13 and Equation 14.

8A to 8D are diagrams illustrating a scheme for improving the accuracy of analyzing a motion of an object 800 according to an exemplary embodiment.

Referring to FIG. 8A , an apparatus for analyzing an image may improve motion analysis accuracy using an observation area 810 having a size larger than that of a pixel group. The observation area 810 may refer to a set of pixel groups 811 including a plurality of pixel groups. For example, the observation area 810 may include an area generated by segmenting the object 800 into equal sizes along an edge of the object 800 .

Upon selecting the viewing area 810, the device for analyzing the image may determine various patterns of pixel groups 811 to be included. For example, a device for analyzing an image may select a viewing area with different patterns of pixel groups 811 and perform motion analysis.

For example, assume the following situation: Object 800 moves to the right 820 without rotation, contraction, and expansion. Here, the object 800 may refer to an object having a rectangular shape, and may be an object moving while being inclined or oriented in an oblique manner. Viewing area 810 may include groups of pixels 811 having the same or similar pattern. When the motion analysis is performed based on the pixel group 811 included in the observation area 810 , although the direction of the actual movement is the right direction 820 , the direction of the actual movement may be analyzed as the movement to the lower right 812 . The apparatus for analyzing the image may select the observation area 830 including the non-linear portion in the edge of the object 800 instead of the observation area 810 including the linear portion. A device for analyzing images may improve the accuracy of analyzing motion by selecting viewing regions that include different patterns of pixel groups.

An apparatus for analyzing an image according to another exemplary embodiment may improve accuracy of analyzing motion using a level of uniformity (LOH). For example, the apparatus for analyzing an image may calculate an LOH of a patch based on Equation 15, and select a patch with a low LOH. Here, the small block may include a pixel group having a size greater than 3×3 pixels.

[Equation 15]

Here, θ _ref and θ _i denote an edge angle (ie, orientation) of a pixel located at the center of a plurality of patches, and an edge angle (ie, orientation) of an i-th neighboring pixel. When LOH is low, patterns of pixel groups included in corresponding small blocks have a certain degree of similarity to each other, and when LOH is high, patterns of pixel groups included in corresponding small blocks may not be similar. The device for analyzing the image may select a patch with a low LOH and select a viewing area that includes a different pattern of pixel groups. Thus, edges can be classified, and the orientation of the edges can be used to determine important features of the image, and can also be applied to determine the shape and/or movement of objects.

FIG. 8B is an example showing that the accuracy of analyzing motion using LOH is improved. For example, assume the following situation: Object 800 moves in direction 820 to the right. When the movement of the object 800 is observed in the observation area 810 , the edge of the object 800 may be observed as an edge 831 at a time point t, and as an edge 832 at a subsequent time point t+Δt. In this case, because the object 800 is moving in a rightward direction 820 , the velocity of the actual movement of the object 800 can be represented as a velocity vector 840 . When motion analysis with LOH is not used, the velocity of movement of object 800 is calculated as velocity vector 850 . Velocity vector 850 may differ in direction and magnitude from velocity vector 840 representing the actual velocity of movement.

When using motion analysis with LOH, the velocity of movement of object 800 may be calculated as velocity vector 860 . The direction of the velocity vector 860 may be the same as the direction of the velocity vector 840 representing the actual movement of the object 800 , however, the magnitude of the velocity vector 860 is different from the magnitude of the velocity vector 840 .

Referring to FIG. 8C, although the object moves at the actual speed 872 having the same speed and direction, the magnitude of the speed 873 calculated based on the shape 871 of the object 800 may be different. For example, the smaller the vector component appearing in the direction (for example, the x-axis direction) along the direction of actual movement, the smaller the magnitude of the calculated velocity.

Referring to FIG. 8D , the apparatus for analyzing an image may correct the magnitude of the moving speed of the object 800 based on Equation 16 and Equation 17. Referring to FIG. More specifically, at operation 881, the apparatus for analyzing an image may receive _Vi as an edge event. The device for analyzing the image can calculate _Vp , _Oc , t, and ω using _Vi . Since similar features described in FIGS. 1 to 7 may be referred to, detailed descriptions for operations 881 and 882 will be omitted. In operation 883, the apparatus for analyzing the image may analyze motion by using the LOH to calculate _Vp . Similarly, the description of FIG. 8A may be applied to operation 883, and thus a detailed description of operation 883 is omitted.

In operation 884, the apparatus for analyzing the image may calculate V _i ^gen based on Equation 16. The definitions of the parameters used in Equation 16 may be the same as those used in Equation 7.

[Equation 16]

V _i ^gen =tP _i +ωA(P _i -O _c )+V _p

In operation 885, the apparatus for analyzing the image may calculate V _i ^cor based on Equation 17. Here, θ represents the angular difference between V _i and V _i ^gen and may correspond to angle 855 shown in FIG. 8B .

[Equation 17]

${{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; cor</span>}}<span\; class="notranslate">\; =</span>\left(\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}\\ \mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right){\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}$

Although V _i has a magnitude similar to the magnitude of the velocity of the actual movement, _Vi may have a direction different from the direction of the actual movement. This is because the speed of movement is calculated for all edges regardless of LOH. Conversely, although V _i ^gen has a smaller magnitude than the magnitude of the velocity of the actual movement, _Vi ^gen may have the same direction as the direction of the actual movement. This is because the speed of movement is calculated for edges within the observation range of the low LOH. Therefore, the device for analyzing the image can obtain the magnitude of the vector from V _i and the direction of the vector from _Vi ^gen to calculate V _i ^cor based on Equation 17.

According to another exemplary embodiment, the apparatus for analyzing an image may iteratively repeat operation 882 to operation 885 . For example, when θ is 90 degrees, since the value of tan90° is not defined, it will be difficult to calculate V _i ^cor . In this case, the device for analyzing the image may iteratively rotate V _i to calculate V _i ^cor . The number of spins can be two or more. Here, need to satisfy Indicates the maximum rotation angle allowed in the kth repetition.

FIG. 9 is a diagram illustrating a scheme for processing a user input based on a moving speed of an object 910 according to an exemplary embodiment.

Referring to FIG. 9 , the apparatus for analyzing an image may process a user input based on a motion of an object 910 . Object 910 may be a user's hand.

The apparatus for analyzing the images may calculate the movement speed 915 of the object 910 using the description provided above with respect to FIGS. 1-8 . The apparatus for analyzing an image may use the calculated movement speed 915 to calculate a change amount with respect to a relative coordinate of a point input by the user.

The device for analyzing the image may process the user input based on the amount of change in the relative coordinates. For example, the device for analyzing the image may move the position of the cursor indicated on the display 920 from the current position 921 to a new position 922 .

Accordingly, relative coordinates may indicate a relative position of an indicator of a user interface (UI), such as a mouse pointer, with respect to a current location of the indicator of the UI. The apparatus for processing the user input may calculate the amount of change in the relative position of the indicator of the UI based on the motion of the object. For example, when an object moves to the right at 1 m/s within 1 second, the change amount of the relative position of the indicator of the UI can be calculated as a vector whose direction is right and whose magnitude is 1 m.

The apparatus for processing the user input may determine the relative position of the pointer of the UI by calculating a new position moved from the current position of the pointer of the UI according to a change amount of the relative position of the pointer of the UI. The means for processing user input may move the indicator of the UI from the current location to the new location.

Although not shown in the diagram, an apparatus for analyzing an image may include a recognizer and a processor. The recognizer may recognize the motion of the object based on an input image including an event signal in which the motion of the object is detected. For example, the recognizer may calculate the movement speed 915 of the object 910 using the description provided above with respect to FIGS. 1-8 . The processor may calculate relative coordinates for user input based on the motion of the object recognized by the recognizer. For example, the processor may use the moving speed 915 calculated by the recognizer to calculate the amount of change in relative coordinates for user input. The processor may update the relative coordinates based on the amount of change of the relative coordinates, and use the updated relative coordinates to process user input.

FIG. 10 is a diagram illustrating a scheme for processing a user input based on a depth of an object, according to an exemplary embodiment.

Referring to FIG. 10 , the apparatus for analyzing an image may also analyze the depth of an object using two different event signals detected at two locations spatially separated from each other for movement of the same object. For example, the sensor 1020 may include a first sensor corresponding to a left eye and a second sensor corresponding to a right eye. The apparatus for analyzing images may measure the depth of an object using disparity between images output from two sensors corresponding to both eyes.

Referring to FIG. 10 , the apparatus for analyzing an image may calculate two images output from two sensors based on a scheme for maximizing a level of similarity (LOS) of a plurality of patches corresponding to left and right eyes. parallax. More specifically, the apparatus for analyzing images may process both the event signal output from the sensor 1020 corresponding to the left eye and the event signal output from the sensor 1020 corresponding to the right eye. For example, the apparatus for analyzing an image may detect pixels corresponding to edges by classifying edge patterns using two respective event signals, and determine the direction of the edges in the corresponding pixels based on the detected edge patterns of the pixels. In this case, the device for analyzing the image may obtain an overlapping image in which the two edges of the object are spaced apart from each other. Two edges of an object that are spaced apart from each other in the two images constitute a disparity between the two images. The apparatus for analyzing an image may apply the algorithm of Table 5 to an overlapping image in which two edges of an object are separated from each other.

[table 5]

$\mathrm{<span\; class="notranslate">\; LOS</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&equiv;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\sqrt{\mathrm{<span\; class="notranslate">\; m</span>}}}\underset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; patch</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; cos</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ,</span>$

θ: azimuth, M: number of effective pixels

$\mathrm{<span\; class="notranslate">\; gLOS</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&equiv;</span>\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; rLOS</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span>$

LOS(x, y, d)≡lLOS(x, y, d)×rLOS(y,d)×gLOS(d)

$\mathrm{<span\; class="notranslate">\; disparity</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; d</span>}}{\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; LOS</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>$

Here, (x, y) represents the coordinates of the small block in the image, and multiple points can be included in the small block; ( _xi , y _i ) represents the points included in the small block of (x, y) coordinates The coordinates of the i-th point, d represents the disparity of the two images. Two images may be received from a sensor corresponding to a left eye and a sensor corresponding to a right eye, and thus, the two images may generally be spaced apart from each other along the direction of the x-axis. Therefore, d may represent the degree to which two images are separated from each other in the direction of the x-axis. θ( _xi ,y _i ) represents an azimuth, and may correspond to the direction of the edge calculated at the point of ( _xi ,y _i ) coordinates.

1LOS(x,y,d) represents the mathematical formula used to determine the LOS in a single patch of (x,y) coordinates; rLOS(y,d) represents the mathematical formula used to determine the y coordinates of a one-dimensional (1D) line Mathematical formula of LOS; gLOS(d) represents a mathematical formula for determining LOS in a 2D area of the entire image.

The device for analyzing the image may calculate d for maximizing LOS(x,y,d). To maximize LOS(x,y,d), the difference between θ( _xi ,y _i ) and θ( _xi +d,y _i ) can be minimized, and therefore, the device used to analyze the image d for minimizing the difference between θ( _xi ,y _i ) and θ( _xi +d,y _i ) can be calculated.

When the disparity between the two images increases, the device for analyzing the images can estimate the depth of the object as shallower, and when the disparity between the two images decreases, the device for analyzing the image can estimate the depth of the object as Estimated to be darkened. The device for analyzing the image may process the user input based on the estimated depth of the object.

The apparatus for analyzing an image may determine an operation mode corresponding to the depth of the object 1010 from the sensor 1020 . For example, a first operating mode area 1031 , a second operating mode area 1032 , a third operating mode area 1033 , and a background area 1034 behind the object 1010 may be predetermined from the sensor 1020 based on depth.

When the depth of the object 1010 is determined to correspond to the second operation mode region 1032 , the apparatus for analyzing an image may process an input using the object 1010 based on a scheme for processing a user input corresponding to the second operation mode region 1032 .

FIG. 11 is a flowchart illustrating a method for analyzing an image according to an exemplary embodiment.

Referring to FIG. 11 , in operation 1110, the apparatus for analyzing an image according to an exemplary embodiment may read event information. In operation 1120, the apparatus for analyzing an image may update an event generation time map (map) based on the location and generation time of the generated event. In operation 1130, the apparatus for analyzing an image may analyze a pattern of event generation times with respect to neighboring events generated by the event generation time graph.

In operation 1140, the apparatus for analyzing an image may classify directionality of an edge based on an event generation pattern. In operation 1150, the apparatus for analyzing an image may extract a velocity component based on an edge direction pattern and an event generation pattern.

In operation 1160, the apparatus for analyzing an image may accumulate event information to analyze movement of an object. In operation 1170, the apparatus for analyzing an image may determine whether the number of accumulated events is sufficient to accurately analyze the movement of the object. As a result of the determination, when it is determined that the number of accumulated events is insufficient to analyze the movement of the object, in operation 1175, the apparatus for analyzing images may determine whether the accumulation time is sufficient to accurately analyze the movement of the object. When it is determined that the accumulation time is not enough to analyze the object movement, the apparatus for analyzing images may return to operation 1110 and further accumulate new event information.

When the number of accumulated events is sufficient to accurately analyze the movement of the object, or the accumulation time is sufficient to accurately analyze the movement of the object although the number of accumulated events is insufficient, in operation 1180, the device for analyzing the image may target a plurality of components Divide the object's position and movement speed. The device for analyzing the image may obtain the velocity of movement, the velocity of expansion or contraction, and the velocity of rotation as components of the velocity of movement.

In operation 1190, the apparatus for analyzing an image may determine a main moving component of an object. For example, the device for analyzing images may determine at least one movement component among movement speed, expansion or contraction speed, and rotation speed that contributes significantly to the movement of the object.

Since similar features described with reference to FIGS. 1 to 10 can be referred to, a detailed description of FIG. 11 will be omitted.

FIG. 12 is a block diagram illustrating an apparatus 1200 for analyzing a three-dimensional (3D) image according to an exemplary embodiment.

12, the apparatus for analyzing a 3D image may include at least two image change detectors 1210 and 1215, edge direction information extractors 1220 and 1225, velocity information extractors 1230 and 1235, and average edge direction information extractors 1240 and 1240. 1245, wherein image change detectors 1210 and 1215 include event-based visual sensors 1211 and 1216, respectively, edge direction information extractors 1220 and 1225 are used to detect edge direction information for one or more events, velocity information extractors 1230 and 1235 for detecting velocity information for one or more events, and average edge direction information extractors 1240 and 1245 for detecting average edge directions of pixels for one or more events at a predetermined time.

In addition, the device 1200 for analyzing 3D images may further include a disparity map extractor 1250, a distance information mapper 1260, and a 3D position/movement analyzer 1270, wherein the disparity map extractor 1250 is used to extract information based on the edge direction information extractor 1220 and The edge direction information of 1225 is used to determine the disparity map, and the distance information mapper 1260 is used to determine the distance information for one or more events.

Since similar features described with reference to FIGS. 1 to 11 can be referred to, detailed descriptions of various modules described in FIG. 12 will be omitted.

The exemplary embodiments shown in the drawings can be implemented by devices including a bus, at least one processor (eg, a central processing unit, a microprocessor, etc.), and a memory, wherein the bus is connected to each of the devices unit, the at least one processor is connected to the bus to control the operation of the device to implement the above functions and execute commands, and the memory is connected to the bus to store commands, received messages and generated messages.

As will be appreciated by those skilled in the art, the exemplary embodiments may be implemented by any combination of software and/or hardware components, such as field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs), that perform particular tasks. A unit or module may conveniently be located on the addressable storage medium and configured to execute on one or more processors or microprocessors. Thus, by way of example, a unit or module may include a component (such as a software component, an object-oriented software component, a class component, and a task component), a process, a function, an attribute, a procedure, a subroutine, a program code segment, a driver, a firmware, a microprocessor Code, Circuits, Data, Databases, Data Structures, Tables, Arrays, and Variables. Functionality provided in components and units may be combined into fewer components and units or modules, or further separated into additional components and units or modules.

The above-described exemplary embodiments can also be implemented in a computer-readable medium including program instructions for implementing various operations performed by a computer. The media may also include data files, data structures, etc., alone or in combination with program instructions. Examples of computer-readable media include magnetic media (such as hard disks, floppy disks, and magnetic tape), optical media (such as CD ROM disks and DVDs), magneto-optical media (such as optical disks), hardware devices specially configured to store and execute program instructions (such as read-only memory (ROM), random-access memory (RAM), flash memory, etc.). Examples of program instructions may include both machine code (such as generated by a compiler) and files containing higher-level code executable by a computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described exemplary embodiments, and vice versa.

Several exemplary embodiments have been described above. However, it should be understood that various modifications may be made. For example, if the described techniques are performed in a different order, and/or if components in the described systems, architectures, devices, and circuits are combined in a different manner and/or replaced or supplemented with other components or their equivalents The described systems, architectures, devices and components within the circuits then achieve suitable results. Accordingly, other implementations are within the scope of the following claims.

Claims (34)

1. An apparatus for analyzing images, said apparatus comprising: A classifier configured to receive an event signal corresponding to at least one pixel of the event-based vision sensor and configured to determine a pattern of a pixel group of a plurality of pixels of the event-based vision sensor, wherein the pixel group includes the at least a pixel and a plurality of neighboring pixels of an event-based vision sensor adjacent to the at least one pixel; An analyzer configured to determine at least one of the shape of the object and the movement of the object based on the pattern of groups of pixels. 2. The device of claim 1, wherein the event signal is indicative of the occurrence of an event at the location of the at least one pixel of the event-based vision sensor. 3. The apparatus of claim 1, wherein the classifier determines whether the group of pixels corresponds to at least one edge pattern among a plurality of predetermined edge patterns. 4. The apparatus of claim 3, wherein the classifier discards the group of pixels in response to determining that the group of pixels does not correspond to the at least one edge pattern. 5. The device of claim 1, wherein the classifier comprises: a type determiner configured to determine a pixel of the plurality of neighboring pixels based on a difference between a time stamp of an event signal corresponding to the at least one pixel and a time stamp of an event signal corresponding to the plurality of neighboring pixels type; A pattern determiner configured to determine a pattern of pixel groups based on pixel types of the plurality of neighboring pixels. 6. The device of claim 5 , wherein the pixel types of the plurality of neighboring pixels include a first pixel type and a second pixel type, wherein the time stamp of the event signal corresponding to the at least one pixel and The difference between the time stamps of the event signals corresponding to neighboring pixels of the first pixel type is less than a first threshold, after the time stamps of the event signals corresponding to the at least one pixel and the event corresponding to neighboring pixels of the second pixel type The difference between the time stamps of the signals is greater than a second threshold. 7. The device of claim 1, wherein the analyzer comprises: A shape analyzer configured to determine the direction of an edge of an object corresponding to a pixel group based on the pattern of the pixel group. 8. The device of claim 1, wherein the analyzer comprises: a calculator configured to calculate a velocity corresponding to the pixel group based on the pattern of the pixel group; A motion analyzer configured to analyze movement of objects based on velocities corresponding to groups of pixels. 9. The apparatus of claim 8, wherein the movement of the object includes at least one of a movement speed component of the object, a rotation speed component of the object, and a scaling speed component of the object. 10. The device of claim 8, wherein the analyzer further comprises: A selector configured to select at least one observation area for analyzing movement of the object from among the plurality of observation areas based on various patterns of pixel groups included in the observation area. 11. The device of claim 1, further comprising: A processor configured to calculate, based on the movement of the object, a change in relative coordinates of a point input by a user, and process the user input based on the change in relative coordinates. 12. An apparatus for analyzing images, said apparatus comprising: A classifier configured to receive a first event signal corresponding to a first pixel of the event-based vision sensor and a second event signal corresponding to a second pixel of the event-based vision sensor, determine a first event-based event signal of the event-based vision sensor a first pattern of a first pixel group of a plurality of pixels, and determine a second pattern of a second pixel group of a second plurality of pixels, wherein the first pixel group includes the at least one first pixel and the at least one a first plurality of adjacent pixels of the event-based vision sensor adjacent to the first pixel, the second pixel group comprising the at least one second pixel and a second plurality of adjacent pixels adjacent to the at least one second pixel; An analyzer that detects a first location of the object based on the first pattern, detects a second location of the object based on the second pattern, and determines a depth of the object based on the first location and the second location. 13. The device of claim 12 , wherein the first event signal is indicative of the occurrence of a first event at a first location of a first pixel of the event-based vision sensor, and the second event signal is indicative of an occurrence of a first event at a first pixel of the event-based vision sensor. Occurrence of a second event at a second location of a second pixel of the sensor. 14. The apparatus of claim 12 , wherein the first event signal corresponds to a signal detecting movement of the object at a first location, and the second event signal corresponds to a second location spatially separated from the first location. The signal corresponding to the movement is detected. 15. The device of claim 12, wherein the analyzer is configured to calculate the distance between the first location and the second location, and when the distance between the first location and the second location increases, the depth of the object Estimated to become shallower, the depth of the object is estimated to become darker as the distance between the first location and the second location decreases. 16. The device of claim 12, further comprising: A processor configured to determine a motion pattern corresponding to the depth of the object, and process user input based on the motion pattern. 17. A method of analyzing an image, the method comprising: receiving an event signal corresponding to at least one pixel of the event-based vision sensor; determining a pattern of pixel groups of a plurality of pixels of the event-based vision sensor, wherein the pixel group includes the at least one pixel and a plurality of adjacent pixels of the event-based vision sensor adjacent to the at least one pixel; At least one of the shape of the object and the movement of the object is determined based on the pattern of groups of pixels. 18. The method of claim 17, wherein the event signal is indicative of the occurrence of an event at the location of the at least one pixel of the event-based vision sensor. 19. The method of claim 17, wherein determining the pattern of groups of pixels comprises: determining a pixel type for the plurality of adjacent pixels based on a difference between a timestamp of an event signal corresponding to the at least one pixel and a timestamp of an event signal corresponding to the plurality of adjacent pixels; determining whether a group of pixels corresponds to at least one edge pattern among a plurality of predetermined edge patterns based on pixel types of the plurality of neighboring pixels; Responsive to determining that the group of pixels does not correspond to the at least one edge pattern, discarding the group of pixels, and in response to determining that the group of pixels corresponds to the at least one edge pattern, determining the pattern of the group of pixels to be the at least one edge pattern. 20. The method of claim 17, wherein determining at least one of the shape of the object and the movement of the object comprises determining a direction of an edge corresponding to the pixel group based on a pattern of the pixel group. 21. The method of claim 17, wherein determining at least one of the shape of the object and the movement of the object comprises: selecting at least one observation area for analyzing movement of the object from among a plurality of observation areas based on various patterns of pixel groups included in the observation area; calculating velocities corresponding to a plurality of pixel groups included in the at least one viewing area; Movement of the object is analyzed based on velocities corresponding to the plurality of pixel groups. 22. The method of claim 17, further comprising: calculating a change amount with respect to the relative coordinates of the point input by the user based on the moving speed of the object included in the movement of the object; User input is processed based on the amount of change in the relative coordinates. 23. A method for analyzing an image, the method comprising: receiving an input of an event signal corresponding to a pixel of the event-based vision sensor indicative of movement of the object; selecting a plurality of neighboring pixels of the event-based vision sensor adjacent to a pixel corresponding to the event signal; determining an edge pattern for the plurality of neighboring pixels; determining a location of an edge of an object based on an edge pattern of the plurality of neighboring pixels; The shape of the object is analyzed based on the positions of the edges of the object. 24. The method of claim 23, wherein the event signal is indicative of the occurrence of an event at the location of a pixel of the event-based vision sensor. 25. The method of claim 23, wherein determining the edge pattern comprises: determining a pixel type for the plurality of neighboring pixels based on a time stamp of an event signal corresponding to the plurality of neighboring pixels and a difference between a time stamp of an event signal corresponding to the pixel; Whether the edge pattern of the plurality of adjacent pixels corresponds to at least one edge pattern of the plurality of edge patterns is determined based on pixel types of the plurality of adjacent pixels. 26. The method of claim 25, wherein the step of determining the edge pattern further comprises: If the edge pattern of the plurality of neighboring pixels does not correspond to the at least one edge pattern, discarding the pixel and the plurality of neighboring pixels. 27. The method of claim 23, further comprising: The movement of the object is analyzed based on the shape of the object. 28. An apparatus for processing user input, the apparatus comprising: a recognizer configured to recognize movement of an object based on an input image, wherein the input image includes an event signal indicative of at least one pixel of an event-based vision sensor that detected movement of the object; A processor configured to update the relative position of the object corresponding to the user input based on the movement of the object. 29. The device of claim 28, wherein the event signal is indicative of the occurrence of an event at the at least one pixel of the event-based vision sensor. 30. The apparatus of claim 28, wherein the movement of the object includes at least one of a movement speed component of the object, a rotation speed component of the object, and a scaling speed component of the object. 31. The device of claim 28, wherein the identifier comprises: a classifier configured to determine a pattern of at least one pixel group in the input image, wherein the at least one pixel group includes the at least one pixel corresponding to the event signal and a plurality of neighboring pixels adjacent to the at least one pixel ; An analyzer configured to analyze movement of the object based on the pattern of the at least one pixel group. 32. The device of claim 31 , wherein the classifier comprises: a type determiner configured to determine a pixel of the plurality of neighboring pixels based on a difference between a time stamp of an event signal corresponding to the at least one pixel and a time stamp of an event signal corresponding to the plurality of neighboring pixels type; A pattern determiner configured to determine a pattern of pixel groups based on pixel types of the plurality of neighboring pixels. 33. The device of claim 31 , wherein the analyzer comprises: a calculator configured to calculate a velocity corresponding to the pixel group based on the pattern of the pixel group; A motion analyzer configured to analyze movement of objects based on velocities corresponding to groups of pixels. 34. The device according to claim 28, wherein the processor is further configured to: calculate an amount of change of the relative position of the object based on the movement of the object, update the relative position of the object based on the amount of change, and based on the updated The relative position of the object to handle user input.