KR102940350B1 - Pedestrian trajectory prediction method using trajectory descriptors representing actual walking patterns - Google Patents

Abstract

The present invention relates to a method for extracting a plurality of path descriptors capable of reflecting the actual walking patterns of pedestrians, and for training and utilizing a pedestrian path prediction model using the same. A pedestrian path prediction method according to one embodiment of the present invention is characterized by comprising the steps of: applying singular value decomposition to the walking paths of a plurality of pedestrians to define the walking paths as a linear combination of path descriptors; and setting the coefficients of the linear combination of the path descriptors as a training dataset to train a pedestrian path prediction model.

Description

Pedestrian Trajourney Prediction Method Using Trajourney Descriptors Representing Actual Walking Patterns

The present invention relates to a method for extracting a plurality of path descriptors capable of reflecting a pedestrian's actual walking pattern, and for learning and utilizing a pedestrian path prediction model using these descriptors.

Pedestrian path prediction technology is a technique that estimates future paths based on a pedestrian's past path and can be applied to various fields such as behavior prediction, crowd movement analysis, abnormal movement detection, and traffic flow analysis.

Various computer vision technologies have been used to predict pedestrian paths, and recently, deep learning technology is being applied to improve prediction accuracy.

Traditional path prediction studies applying deep learning performed path prediction operations based on the pedestrian's 2D coordinates, but there was a problem in that it was difficult for the path prediction model to reflect actual walking patterns because actual pedestrians do not move toward specific coordinates, and there was a problem in that the path prediction model became very heavy due to the large number of coordinates that had to be used for model training, which increased the data dimension.

To address this, recent attempts have been made to reduce data dimensionality through parametric curve fitting, such as Bezier and B-Spline curve functions; however, these methods are based solely on mathematical approaches and have limitations in that they are not suitable for representing human social movements.

The present invention aims to predict a pedestrian path using a descriptor representing the actual walking pattern of a pedestrian, unlike traditional methods that predict paths based on two-dimensional coordinates, when performing pedestrian path prediction using a neural network model.

The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

A pedestrian path prediction method according to an embodiment of the present invention for achieving the aforementioned purpose is characterized by comprising the steps of: applying singular value decomposition to the walking paths of multiple pedestrians to define the walking paths as a linear combination of path descriptors; and setting the coefficients of the linear combination of the path descriptors as a training dataset to train a pedestrian path prediction model.

In one embodiment, the step of applying the singular value decomposition is characterized by including the step of applying the singular value decomposition to a walking path defined by two-dimensional coordinates for each of the plurality of pedestrians.

In one embodiment, the path descriptor is characterized as being a singular vector within a square orthogonal matrix determined by applying singular value decomposition to the walking paths of the plurality of pedestrians.

In one embodiment, the step of defining the walking path as a linear combination of path descriptors is characterized by including the step of calculating the linear combination coefficients by multiplying the walking path by the transpose of the orthogonal matrix determined through the singular value decomposition.

In one embodiment, the plurality of technicians are characterized as singular vectors within a rectangular orthogonal matrix determined by applying truncated singular value decomposition to the walking paths of the plurality of pedestrians.

In one embodiment, the step of training the pedestrian path prediction model comprises: a step of dividing the linear combination coefficients into first and second coefficients according to the flow of time; and

The above pedestrian path prediction model is characterized by including a step of supervising learning to receive the first coefficient as input and output the second coefficient.

In one embodiment, the method further comprises the steps of: calculating observed linear combination coefficients for an observed walking path by multiplying the transpose of an orthogonal matrix determined through the singular value decomposition by the observed walking path of a target pedestrian; inputting the observed linear combination coefficients into a pedestrian path prediction model; and generating a predicted path for each pedestrian by multiplying the predicted linear combination coefficients output from the pedestrian path prediction model by the orthogonal matrix determined through the singular value decomposition.

In one embodiment, the step of training the pedestrian path prediction model comprises: setting a representative value of a plurality of clusters formed by the linear combination coefficients as an anchor; dividing the linear combination coefficients into first and second coefficients over time; and supervising the pedestrian path prediction model to receive the first coefficients as input and output the difference between the second coefficients and the anchor.

In one embodiment, the method further comprises the steps of: calculating observed linear combination coefficients for an observed walking path by multiplying the transpose of an orthogonal matrix determined through the singular value decomposition by the observed walking path of a target pedestrian; inputting the observed linear combination coefficients into a pedestrian path prediction model and applying the difference output from the pedestrian path prediction model to the anchor to calculate the predicted linear combination coefficients; and generating a predicted path for each pedestrian by multiplying the predicted linear combination coefficients by the orthogonal matrix determined through the singular value decomposition.

Based on the observation that socially common walking paths can be represented as a combination of several walking patterns, the present invention defines the walking path as a linear combination of a small number of path descriptors and trains a neural network model using only linear combination coefficients. Consequently, the model can learn the actual walking patterns of pedestrians and reduce the data dimensionality required for training, thereby offering the advantage of improving the model's training speed and prediction accuracy.

In addition to the effects described above, the specific effects of the present invention are described together with the specific details for implementing the invention below.

FIG. 1 is a flowchart illustrating a pedestrian path prediction method according to an embodiment of the present invention.
FIG. 2 is a drawing illustrating a pedestrian's walking path.
FIG. 3 is a drawing illustrating the predicted path of a pedestrian shown in FIG. 2.
Figure 4 is a diagram illustrating a traditional method for pedestrian path prediction.
Figure 5 is a diagram illustrating the process of applying singular value decomposition to a walking path.
FIG. 6 is a diagram illustrating the process of applying frustum singular value decomposition to a walking path.
Figure 7 is a diagram illustrating the process of extracting a path descriptor from a walking path.
FIG. 8 is a diagram visualizing a path descriptor according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating the calculation of linear combination coefficients of path descriptors using an orthogonal matrix determined through singular value decomposition.
Figure 10 is a diagram illustrating the process of training a pedestrian path prediction model by setting linear combination coefficients as the training dataset.
FIG. 11 is a diagram illustrating a pedestrian path prediction process according to an embodiment of the present invention.
Figure 12 is a diagram illustrating the use of anchors for linear coupling coefficients in the training of a pedestrian path prediction model.
FIG. 13 is a diagram illustrating a pedestrian path prediction process according to another embodiment of the present invention.

The aforementioned objectives, features, and advantages are described in detail below with reference to the attached drawings, thereby enabling those skilled in the art to easily implement the technical concept of the present invention. In describing the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions would unnecessarily obscure the essence of the invention. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

In this specification, terms such as "first," "second," etc. are used to describe various components, but these components are not limited by these terms. These terms are used merely to distinguish one component from another, and unless specifically stated otherwise, the first component may be the second component.

Additionally, in this specification, the statement that any configuration is disposed on the "upper (or lower)" or "upper (or lower)" of a component may mean not only that any configuration is disposed in contact with the upper (or lower) surface of said component, but also that another configuration may be interposed between said component and any configuration disposed on (or below) said component.

Furthermore, where it is stated in this specification that one component is "connected," "coupled," or "connected" to another component, it should be understood that while the components may be directly connected or connected to each other, another component may be "interposed" between each component, or each component may be "connected," "coupled," or "connected" through another component.

Additionally, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be included.

Additionally, in this specification, "A and/or B" means A, B, or A and B unless specifically stated otherwise, and "C to D" means C or more and D or less, unless specifically stated otherwise.

The present invention relates to a method for extracting a plurality of path descriptors capable of reflecting a pedestrian's actual walking pattern, and for learning and utilizing a pedestrian path prediction model using the same. Hereinafter, a pedestrian path prediction method using path descriptors representing actual walking patterns (hereinafter, pedestrian path prediction method) will be described in detail with reference to FIGS. 1 to 13.

FIG. 1 is a flowchart illustrating a pedestrian path prediction method according to one embodiment of the present invention.

FIG. 2 is a drawing illustrating a pedestrian's walking path, and FIG. 3 is a drawing illustrating a predicted path of the pedestrian shown in FIG. 2.

Figure 4 is a diagram illustrating a traditional method for predicting pedestrian paths.

Figure 5 is a diagram illustrating the process of applying singular value decomposition to a walking path, and Figure 6 is a diagram illustrating the process of applying frustum singular value decomposition to a walking path.

FIG. 7 is a diagram illustrating the process of extracting a path descriptor from a walking path, and FIG. 8 is a diagram visualizing a path descriptor according to an embodiment of the present invention.

Figure 9 illustrates the calculation of linear combination coefficients of path descriptors using an orthogonal matrix determined through singular value decomposition.

Figure 10 is a diagram illustrating the process of training a pedestrian path prediction model by setting linear combination coefficients as the training dataset.

FIG. 11 is a diagram illustrating a pedestrian path prediction process according to one embodiment of the present invention.

Figure 12 illustrates the use of anchors for linear coupling coefficients in the training of a pedestrian path prediction model.

FIG. 13 is a diagram illustrating a pedestrian path prediction process according to another embodiment of the present invention.

Referring to FIG. 1, a pedestrian path prediction method according to one embodiment of the present invention can be broadly divided into a learning phase and an application phase.

First, the learning step may include a step of defining a walking path as a linear combination of path descriptors through singular value decomposition (S10) and a step of training a pedestrian path prediction model using the linear combination coefficients of the path descriptors (S20).

Additionally, the application step may include a step of calculating observed linear combination coefficients for the observed walking path of a target pedestrian (S30) and a step of generating a predicted path by inputting the observed linear combination coefficients into a pedestrian path prediction model (S40).

However, the pedestrian path prediction method illustrated in FIG. 1 is according to one embodiment, and the steps constituting the invention are not limited to the embodiment illustrated in FIG. 1, and some steps may be added, changed, or deleted as needed.

Each step illustrated in FIG. 1 can be performed by a processor, and the processor may include at least one physical element among ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, and microcontrollers to perform the operation of the invention described below.

The processor may collect multiple pedestrian videos (100) containing at least one pedestrian for training a pedestrian path prediction model. These pedestrian videos (100) may be videos from various viewpoints, for example, first-person view (FPV) videos or bird's-eye view videos.

The processor may collect pedestrian images (100) from other devices or any storage medium. For example, the processor may collect pedestrian images (100) in front of a vehicle from a vehicle, may collect pedestrian images (100) within a field of view (FOV) from a CCTV, or may collect previously stored pedestrian images (100) from any database.

The processor can identify the pedestrian's walking path (L) in the walking video (100).

Referring to FIG. 2, the walking path (L) refers to the path taken by the pedestrian (110) in the walking video (100) and can be identified from the images of consecutive frames. That is, the walking path (L) can be defined by the position of the pedestrian (110) over time.

The processor can detect the position of the pedestrian (110) frame by frame of the walking video (100) and identify the walking path (L) based on the position changing over time. To this end, the processor may use any object detection and tracking algorithm known in the art.

Referring to FIG. 3, the processor can generate a predicted path (L _p ) of a pedestrian (110) based on a previously identified walking path (L). Specifically, the processor can determine a predicted path (L p) of a second time interval (t2) that follows the first time interval (t1) based on the walking path (L) for the first time interval ( _t1 ).

Referring to FIG. 4, for the aforementioned pedestrian path prediction, the traditional pedestrian path prediction model (200) performed the path prediction operation based on the pedestrian's 2D coordinates. In this case, as shown in the drawing, there were many input and output parameters, that is, the dimension of the data became very large, and there was a problem that the pedestrian path prediction model (200) became very heavy.

In addition, since actual pedestrians do not move toward specific coordinates, there was a fundamental limitation in that coordinateizing the walking path (L) did not reflect the actual walking pattern, and accordingly, there was a limitation in that the pedestrian path prediction model (200) could not reflect human social movement.

To solve the problem of this traditional method, the present invention is characterized by using a path descriptor capable of reflecting actual social walking patterns, and the learning method of the pedestrian path prediction model (200) will be explained first below.

The processor can represent the walking path (L) of multiple pedestrians using several path descriptors that represent actual walking patterns. To do this, the processor can define the walking path (L) as a linear combination of path descriptors by applying Singular Value Decomposition (SVD) to the walking path (L) of multiple pedestrians (S10).

Singular Value Decomposition is a generalized method of Eigenvalue Decomposition that can be used to extract important information, namely principal components, contained in data represented as matrices. To apply Singular Value Decomposition, the processor can define the walking path (L) for each of multiple pedestrians as a matrix.

Specifically, the processor can define a walking path (L) as two-dimensional coordinates for each of multiple pedestrians and apply singular value decomposition thereto. As described with reference to FIG. 4, the walking path (L) may be a set of two-dimensional coordinates over time, and the processor can define the set of coordinates of each pedestrian over time, (x _n ^t , y _n ^t ), as the walking path (L).

Taking Fig. 5 as an example, the processor [represents] the walking path (L) for N pedestrians Defined as the matrix (hereinafter walking path matrix, T), and through the singular value decomposition defined by [Equation 1] below, two orthogonal matrices (U, V ^T ) and a diagonal matrix ( Can generate ).

[Mathematical Formula 1]

Specifically, the processor applies singular value decomposition to the walking path matrix (T) The orthogonal matrix of (hereinafter the left orthogonal matrix, U) and, The diagonal matrix of ( ), and An orthogonal matrix (hereinafter referred to as the right orthogonal matrix V ^T ) can be generated. Here, the column vectors belonging to the left orthogonal matrix (U) and the row vectors belonging to the right orthogonal matrix (V ^T ) may be singular vectors.

The processor may set a square orthogonal matrix determined through singular value decomposition, that is, a singular vector within the left orthogonal matrix (U) (hereinafter left singular vector, u ₁ ~ u _L ) or a singular vector within the right orthogonal matrix (V ^T ) (hereinafter right singular vector, v ₁ ^T ~ v _N ^T ) as a path descriptor. However, since the rows constituting the walking path matrix (T) correspond to 2D coordinates, in order to extract the main features contained in the coordinates as a path descriptor, the processor may set a left singular vector (u ₁ ~ u _L ) whose number of components corresponds to the number of coordinates as a path descriptor.

After path descriptors are set, the processor can define a walking path (L) as a linear combination of path descriptors. Specifically, the processor can define the coordinates of the pedestrian as shown in [Equation 2] below.

[Mathematical Formula 2]

(Here, n is the pedestrian index, and _{c_n} is the linear combination coefficient)

Next, the processor can calculate linear combination coefficients for each pedestrian by multiplying the transpose matrix (U ^T ) of the orthogonal matrix determined through singular value decomposition by the walking path (L). Specifically, [Equation 2] can be transformed into [Equation 3], and the processor can calculate linear combination coefficients (c _n ) based on [Equation 3].

[Mathematical Formula 3]

Since U _L is defined by the aforementioned L path descriptors and L _n is defined by the time-sequential coordinates of each pedestrian, the processor can quantitatively calculate the linear combination coefficient for each pedestrian.

When calculating linear combination coefficients as described in [Equation 3] above, the number of dimensions of the linear combination coefficients corresponds to the number of pedestrian coordinates, and the processor can extract fewer path descriptors defining the walking path (L) to reduce the learning data dimensions of the pedestrian path prediction model (200) to be described later.

To this end, the processor can apply truncated singular value decomposition to the walking paths (L) of multiple pedestrians. Here, the truncated singular value decomposition is a diagonal matrix ( generated through singular value decomposition) A method of defining a left orthogonal matrix (U) and a right orthogonal matrix (V ^T ) by keeping only the top k diagonal elements of ), where k can be set by the user.

To explain in detail with reference to Fig. 6, the processor applies the frustum singular value decomposition defined by [Equation 4] to the walking path matrix (T) to obtain the left and right orthogonal matrices (U, V ^T ) and the diagonal matrix ( ) can be generated. In this case, the diagonal matrix( Since the number of elements (k) of ) is arbitrarily changed by the user, the result of the matrix multiplication operation may be an approximate matrix (T') rather than the actual walking path matrix (T).

[Mathematical Formula 4]

Consequently, through frustum singular value decomposition, the processor The left orthogonal matrix (U) of, and, The diagonal matrix of ( ), and A right orthogonal matrix (V ^T ) can be generated. Similar to singular value decomposition, column vectors in the left orthogonal matrix (U) and row vectors in the right orthogonal matrix (V ^T ) can be singular vectors.

The processor may set a left singular vector (u ₁ ~ u _k ) or a right singular vector (v ₁ ^T ~ v _k ^T ) determined through frustum singular value decomposition as a path descriptor. However, as previously explained, since the rows constituting the walking path matrix (T) correspond to 2D coordinates, in order to extract the main features contained in the coordinates as a path descriptor, the processor may set a left singular vector (u ₁ ~ u _k ) whose number of components corresponds to the number of coordinates as a path descriptor.

After path descriptors are set, the processor can define each pedestrian's walking path (L) as a linear combination of path descriptors. Specifically, the processor can define the walking path (L) as shown in [Equation 5] below.

[Mathematical Formula 5]

Next, the processor can calculate linear combination coefficients for each pedestrian by multiplying the transpose of the orthogonal matrix determined through frustum singular value decomposition by the walking path (L). Specifically, [Equation 5] can be approximated by [Equation 6], and the processor can calculate linear combination coefficients (c _n ) based on [Equation 6].

[Mathematical Formula 6]

Since U _k is defined by the aforementioned k path descriptors and L _n is defined by the time-sequential coordinates of each pedestrian, the processor can quantitatively calculate the linear combination coefficient for each pedestrian.

When frustum singular value decomposition is applied as described above, the L-dimensional linear combination coefficients calculated for each pedestrian can be reduced to k dimensions desired by the user, and thereby the dimension of the training data of the pedestrian path prediction model (200) described later can be reduced.

Referring to Fig. 7, the processor can generate an orthogonal matrix (U) by applying frustum singular value decomposition to each pedestrian's walking path (L). Subsequently, the processor can set six singular vectors (u ₁ to u ₆ ) within the orthogonal matrix as path descriptors. At this time, the six singular vectors (u ₁ to u ₆ ) can be visualized as actual walking patterns that commonly occur in society.

Taking Fig. 8 as an example, the six singular vectors (u ₁ to u ₆ ) can be visualized along the horizontal axis (X), vertical axis (Y), and time axis (T). For convenience of explanation, the +x axis direction will be described as a straight direction below.

The first singular vector (u ₁ ) can correspond to a walking pattern in which a pedestrian moves straight for a certain distance, and the second singular vector (u ₂ ) can correspond to a walking pattern in which a pedestrian moves to the left for a certain distance. Additionally, the third singular vector (u ₃ ) can correspond to a walking pattern in which a pedestrian moves straight for a certain distance, then turns around and returns in the opposite direction, and the fourth singular vector (u ₄ ) can correspond to a walking pattern in which a pedestrian moves to the right for a certain distance, then turns around and returns in the opposite direction. Additionally, the fifth singular vector (u ₅ ) can correspond to a walking pattern in which a pedestrian lingers while repeatedly moving straight and backward, and the sixth singular vector (u ₆ ) can correspond to a walking pattern in which a pedestrian lingers while repeatedly moving left and right.

Referring to FIG. 9, the processor can calculate linear combination coefficients by multiplying each pedestrian's walking path (L) by the transpose of the previously determined orthogonal matrix (U ^T ), and can define each walking path (L) as a linear combination of path descriptors.

Once the linear combination coefficients are determined through the process described above, the processor can set the linear combination coefficients as a training dataset to train the pedestrian path prediction model (200) (S20).

In the present invention, the learning and application method of the pedestrian path prediction model (200) can be divided into two types, and the first method will be explained first below.

The pedestrian path prediction model (200) can be supervised learning by a training dataset. As described with reference to FIG. 4, the pedestrian path prediction model (200) performs a task of predicting a future pedestrian path (L _p ) based on a past pedestrian path (L), so the processor can set an argument corresponding to the past pedestrian path (L) as input data and an argument corresponding to the future pedestrian path (L _p ) as output data.

To this end, the processor may divide the linear combination coefficient determined in step (S10) into first and second coefficients according to the flow of time. The linear combination coefficient may be determined by all walking paths (L) of each pedestrian observed over time, and the processor may divide the total observation time into a first time interval and a second time interval that is continuous with the first time interval, and then divide the linear combination coefficient calculated by the walking path (L) of the first time interval into the first coefficient and the linear combination coefficient calculated by the walking path (L) of the second time interval into the second coefficient.

Next, referring to FIG. 10, the processor can supervise the pedestrian path prediction model (200) so that it receives a first coefficient as input and outputs a second coefficient. In other words, the processor can supervise the pedestrian path prediction model (200) by setting the input data and output data of the pedestrian path prediction model (200) as the first and second coefficients, respectively.

Here, the pedestrian path prediction model (200) is a variety of baseline networks used for pedestrian path prediction in the relevant technical field, such as STGCNN (A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction), SGCN (Sparse Graph Convolution Network), STGAT (Spatial-Temporal Graph Attention network), PECNet (Predicted Endpoint Conditioned Network), Trajectron++, LB-EBM (Latent Belief Energy-Based Model), MID (Stochastic trajectory prediction via Motion Indeterminacy Diffusion), etc.

According to supervised learning, the pedestrian path prediction model (200) learns the relationship between the linear combination coefficient of the first time section and the linear combination coefficient of the second time section, and after the learning is completed, receives the linear combination coefficient of the time section that was not used in the learning and the linear combination coefficient of another time section that is continuous with the corresponding time section ( Can output ).

Below, the application step of the present invention will be specifically described, which involves generating a predicted path (L _p ) of a target pedestrian using a pedestrian path prediction model (200) that has been trained according to the first method described above.

First, the processor can calculate the observed linear combination coefficients for the observed walking path of the target pedestrian (S30).

Referring to FIG. 11, when a walking path (L) of a target pedestrian that is the subject of path prediction is observed, the processor can calculate the observed linear combination coefficients for the observed walking path (L) by multiplying the transpose matrix (U ^T ) of the orthogonal matrix determined earlier through singular value decomposition by the observed walking path (L) of the target pedestrian.

Next, the processor can generate a predicted path by inputting observed linear combination coefficients into the trained pedestrian path prediction model (200) (S40).

Referring again to FIG. 11, the processor can input the observed linear combination coefficients ( _c1 to _c6 ) into the pedestrian path prediction model (200) that has completed learning. As described with reference to FIG. 10, since the pedestrian path prediction model (200) has learned the correlation between the linear combination coefficients for consecutive time intervals, the pedestrian path prediction model (200) can output the predicted linear combination coefficients for the time after the target pedestrian is observed.

The processor can generate a predicted path (L p ) for each pedestrian by multiplying the predicted linear combination coefficients output from the pedestrian path prediction model ( ₂₀₀ ) by the orthogonal matrix (U) determined earlier through singular value decomposition in order to represent them in two dimensions.

Next, the second method of training and applying the pedestrian path prediction model (200) will be described, and further explanation regarding the matters described in the first method will be omitted. Accordingly, any content not described below should be understood as identical to the first method.

Referring to FIG. 12, the processor may cluster all linear combination coefficients determined in step (S10) (only two linear combination coefficients ( _c1 , _c2 ) are shown in the drawing for convenience of explanation), set representative values of multiple clusters as anchors, and store them in advance in an anchor bank (140). At this time, various methods used in the relevant technical field may be applied to clustering and setting representative values.

Next, the processor can classify the linear combination coefficients determined in step (S10) into first and second coefficients over time, just like the first method, and supervise the pedestrian path prediction model (200) so that it receives the first coefficient as input and outputs the difference between the second coefficient and the anchor (hereinafter, difference vector). In other words, the processor can supervise the pedestrian path prediction model (200) by setting the input data of the pedestrian path prediction model (200) as the first coefficient and the output data as the difference vector.

According to supervised learning, the pedestrian path prediction model (200) learns the relationship between the linear combination coefficient of the first time interval and the difference vector of the second time interval, and after the learning is completed, it can receive the linear combination coefficient of the time interval that was not used in the learning and output the difference vector for another time interval that is consecutive to the corresponding time interval.

In this way, when using anchors, the pedestrian path prediction model (200) only needs to predict the refinement value, i.e., the difference vector, instead of directly predicting the linear combination coefficients, so the prediction accuracy and learning speed can be improved.

Below, the application step of the present invention will be specifically described, which involves generating a predicted path (L _p ) of a target pedestrian using a pedestrian path prediction model (200) that has been trained according to the second method described above.

First, the processor can calculate the observed linear combination coefficients for the observed walking path of the target pedestrian (S30).

Referring to FIG. 13, when a walking path (L) of a target pedestrian that is the subject of path prediction is observed, the processor can calculate the observed linear combination coefficients for the observed walking path (L) by multiplying the observed walking path (L) by the transpose matrix (U ^T ) of the orthogonal matrix determined earlier through singular value decomposition.

Next, the processor can generate a predicted path by inputting observed linear combination coefficients into the trained pedestrian path prediction model (200) (S40).

Referring again to FIG. 13, the processor can input the observed linear combination coefficients into the pedestrian path prediction model (200). As described with reference to FIG. 12, since the pedestrian path prediction model (200) has learned the relationship between the linear combination coefficients and the difference vector for consecutive time intervals, the pedestrian path prediction model (200) can output the difference vector for the time after the target pedestrian is observed.

The processor can calculate the predicted linear combination coefficients by applying the difference vector to the anchors stored in the anchor bank (140), and to represent the predicted linear combination coefficients in two dimensions, it can generate the predicted path (L _p ) of each pedestrian by multiplying the predicted linear combination coefficients by the orthogonal matrix (U) determined earlier through singular value decomposition.

As described above, the present invention has the advantage that, for the training of a pedestrian path prediction model (200), important information, namely key features, contained in the walking paths of multiple pedestrians are extracted as path descriptors, and the walking path is defined as a linear combination of path descriptors, and the model is trained using only the linear combination coefficients, thereby enabling the model to learn the actual walking patterns of pedestrians that occur in society.

In addition, as explained with reference to Fig. 9, socially common walking paths can be defined as a linear combination of a small number (e.g., 6) of path descriptors, and the present invention has the effect of being able to generate a model with high prediction performance using only low-dimensional data by training the model using only a small number of linear combination coefficients.

In summary, the present invention takes into account the fact that socially common walking paths can be represented as a combination of several walking patterns. By defining the walking path as a linear combination of a small number of path descriptors and training a neural network model using only linear combination coefficients, the model can learn the actual walking patterns of pedestrians and reduce the data dimensionality for training, thereby having the advantage of improving the model's training speed and prediction accuracy.

Although the present invention has been described above with reference to the illustrated drawings, the present invention is not limited by the embodiments and drawings disclosed in this specification, and it is obvious that various modifications can be made by a person skilled in the art within the scope of the technical concept of the present invention. Furthermore, even if the effects of the configuration according to the present invention were not explicitly described while explaining the embodiments of the present invention above, it is natural to acknowledge that the effects predictable by said configuration should also be recognized.

Claims (9)

A step of defining the walking path as a linear combination of path descriptors by applying singular value decomposition to the walking paths of multiple pedestrians;
A step of setting a representative value of a multiple cluster formed by the linear combination coefficients of the above path descriptors as an anchor;
A step of dividing the above linear combination coefficients into first and second coefficients according to the flow of time;
A step of supervising the pedestrian path prediction model so that the pedestrian path prediction model receives the first coefficient as input and outputs the difference between the second coefficient and the anchor;
A step of calculating observed linear combination coefficients for the observed walking path of a target pedestrian by applying the above singular value decomposition;
A step of inputting the observed linear combination coefficients into the learned pedestrian path prediction model and calculating the predicted linear combination coefficients based on the difference vector output from the pedestrian path prediction model; and
A step comprising generating a predicted path for each pedestrian based on the above-mentioned predictive linear combination coefficients
Pedestrian path prediction method.
In paragraph 1,
The step of applying the above singular value decomposition is
The step of applying singular value decomposition to a walking path defined by two-dimensional coordinates for each of the plurality of pedestrians.
Pedestrian path prediction method.
In paragraph 1,
The above path descriptor is a singular vector within a square orthogonal matrix determined by applying singular value decomposition to the walking paths of the above multiple pedestrians.
Pedestrian path prediction method.
In paragraph 1,
The step of defining the above walking path as a linear combination of path descriptors
A step comprising calculating the linear combination coefficients by multiplying the transpose of the orthogonal matrix determined through the singular value decomposition by the walking path.
Pedestrian path prediction method.
In paragraph 1,
The above multiple technical experts are singular vectors within a rectangular orthogonal matrix determined by applying truncated singular value decomposition to the walking paths of the above multiple pedestrians.
Pedestrian path prediction method.
delete delete delete delete