KR20240044888A - Method and apparatus for determining driving speed using pedestrian intention estimation - Google Patents

Abstract

This disclosure relates to a method and device for determining driving speed using pedestrian intent estimation. One embodiment of the present disclosure includes the steps of recognizing a pedestrian and obtaining recognition data and state data; updating a pedestrian intention vector by estimating a probability distribution for a plurality of intentions that the pedestrian may have through at least one of the recognition data and the state data; estimating pedestrian intent at a next time point based on the updated pedestrian intent vector; and determining the driving speed of the vehicle in consideration of the pedestrian's intention at the next time point.

Description

Method and apparatus for determining driving speed using pedestrian intention estimation {METHOD AND APPARATUS FOR DETERMINING DRIVING SPEED USING PEDESTRIAN INTENTION ESTIMATION}

The present disclosure provides a method and apparatus for determining driving speed using pedestrian intent estimation.

Smartization of vehicles is rapidly progressing due to the convergence of information and communication technology and the vehicle industry. Due to smartization, vehicles are evolving from simple mechanical devices to smart cars, and in particular, autonomous driving is attracting attention as a core technology for smart cars. Autonomous driving is a technology that allows a vehicle to reach its destination on its own without the driver having to operate the steering wheel, accelerator pedal, or brakes.

Various additional functions related to autonomous driving are continuously being developed, and research continues on how to provide a safe autonomous driving experience to both passengers and pedestrians by controlling the car by recognizing and judging the driving environment using various data. is being requested.

The above-mentioned background technology is technical information that the inventor possessed for deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be known art disclosed to the general public before filing the application for the present invention.

The purpose of the present disclosure is to provide a method and device for determining driving speed using pedestrian intent estimation. The problem that the present disclosure aims to solve is not limited to the problems mentioned above, and other problems and advantages of the present disclosure that are not mentioned can be understood through the following description and can be understood more clearly by the examples of the present disclosure. It will be. In addition, it will be appreciated that the problems and advantages to be solved by the present disclosure can be realized by the means and combinations thereof indicated in the patent claims.

A first aspect of the present disclosure includes recognizing a pedestrian and obtaining recognition data and state data; updating a pedestrian intention vector by estimating a probability distribution for a plurality of intentions that the pedestrian may have through at least one of the recognition data and the state data; estimating pedestrian intent at a next time point based on the updated pedestrian intent vector; and determining the driving speed of the vehicle in consideration of the pedestrian's intention at the next time point.

A second aspect of the present disclosure includes a memory storing at least one program; and a processor operating by executing the at least one program, wherein the processor recognizes a pedestrian and obtains recognition data and state data, and determines that the pedestrian may have the information through at least one of the recognition data and the state data. Update the pedestrian intention vector by estimating the probability distribution for a plurality of possible intentions, estimate the pedestrian intention at the next time point based on the updated pedestrian intention vector, and drive the vehicle considering the pedestrian intention at the next time point. A device for determining speed may be provided.

A third aspect of the present disclosure may provide a computer-readable recording medium recording a program for executing the method of the first aspect on a computer.

In addition to this, another method for implementing the present invention, another device, and a computer-readable recording medium recording a program for executing the method may be further provided.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to the problem solving means of the present disclosure described above, a safe autonomous driving experience can be provided to both passengers and pedestrians even in a traffic environment with high traffic volume.

In addition, according to the problem-solving means of the present disclosure, it is possible to provide a method for estimating pedestrian intent that takes into account the uncertainty of the pedestrian's movement and the driving state of the vehicle.

1 to 3 are diagrams for explaining an autonomous driving method according to an embodiment.
Figure 4 is an example diagram schematically showing the environment around a vehicle driving path according to an embodiment.
Figure 5 is a conceptual diagram of a method for determining driving speed using pedestrian intention estimation according to an embodiment.
Figure 6 is a diagram for explaining the first to third intentions of a pedestrian according to an embodiment.
Figure 7 is a flowchart of a method for determining driving speed using pedestrian intention estimation according to an embodiment.
Figure 8 is a block diagram of an apparatus for determining driving speed using pedestrian intention estimation according to an embodiment.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms, and should be understood to include all transformations, equivalents, and substitutes included in the spirit and technical scope of the present invention. do. The examples presented below are provided to make the disclosure of the present invention complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or may be implemented by circuit configurations for certain functions. Additionally, for example, functional blocks of the present disclosure may be implemented in various programming or scripting languages. Functional blocks may be implemented as algorithms running on one or more processors. Additionally, the present disclosure may employ conventional technologies for electronic environment setup, signal processing, and/or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

Additionally, connection lines or connection members between components shown in the drawings merely exemplify functional connections and/or physical or circuit connections. In an actual device, connections between components may be represented by various replaceable or additional functional connections, physical connections, or circuit connections.

Hereinafter, 'vehicle' may refer to any type of transportation used to move people or objects with engine, such as a car, bus, motorcycle, kickboard, or truck.

Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

Referring to FIG. 1, the autonomous driving device according to an embodiment of the present invention can be mounted on a vehicle to implement an autonomous vehicle 10. The autonomous driving device mounted on the autonomous vehicle 10 may include various sensors (including cameras) to collect surrounding situation information. For example, the autonomous driving device may detect the movement of the preceding vehicle 20 running in front through an image sensor and/or an event sensor mounted on the front of the autonomous vehicle 10. The self-driving device may further include sensors for detecting not only the front of the self-driving vehicle 10, but also other driving vehicles 30 running in the lane next to the self-driving vehicle 10 and pedestrians around the self-driving vehicle 10.

At least one of the sensors for collecting situational information around the autonomous vehicle may have a predetermined field of view (FoV), as shown in FIG. 1 . For example, when a sensor mounted on the front of the autonomous vehicle 10 has a field of view (FoV) as shown in FIG. 1, information detected at the center of the sensor may have relatively high importance. This may be because the information detected at the center of the sensor includes most of the information corresponding to the movement of the preceding vehicle 20.

The self-driving device processes information collected by the sensors of the self-driving vehicle 10 in real time to control the movement of the self-driving vehicle 10, while storing at least some of the information collected by the sensors in a memory device. .

Referring to FIG. 2, the autonomous driving device 40 may include a sensor unit 41, a processor 46, a memory system 47, and a vehicle body control module 48. The sensor unit 41 includes a plurality of sensors (including cameras) 42-45, and the plurality of sensors 42-45 include an image sensor, an event sensor, an illumination sensor, a GPS device, an acceleration sensor, etc. It can be included.

Data collected by sensors 42-45 may be transmitted to processor 46. The processor 46 stores the data collected by the sensors 42-45 in the memory system 47 and controls the vehicle body control module 48 based on the data collected by the sensors 42-45 to control the vehicle body control module 48. movement can be determined. The memory system 47 may include two or more memory devices and a system controller for controlling the memory devices. Each of the memory devices may be provided as one semiconductor chip.

In addition to the system controller of the memory system 47, each of the memory devices included in the memory system 47 may include a memory controller, and the memory controller may include an artificial intelligence (AI) operation circuit such as a neural network. The memory controller may generate calculation data by assigning a predetermined weight to data received from the sensors 42 - 45 or the processor 46 and store the calculation data in a memory chip.

Figure 3 is a diagram showing an example of image data acquired by sensors (including cameras) of an autonomous vehicle equipped with an autonomous driving device. Referring to FIG. 3, image data 50 may be data acquired by a sensor mounted on the front of an autonomous vehicle. Therefore, the image data 50 may include the front part 51 of the autonomous vehicle, the preceding vehicle 52 in the same lane as the autonomous vehicle, the vehicles 53 and the background 54 around the autonomous vehicle. .

In the image data 50 according to the embodiment shown in FIG. 3, the data in the area where the front part 51 and the background 54 of the autonomous vehicle appear are data that are unlikely to affect the operation of the autonomous vehicle. It can be. In other words, the front 51 and background 54 of the autonomous vehicle can be considered data with relatively low importance.

On the other hand, the distance to the preceding vehicle 52 and the lane change movement of the driving vehicle 53 may be very important factors in the safe operation of an autonomous vehicle. Accordingly, in the image data 50, data in an area including the preceding vehicle 52 and the driving vehicle 53 may have relatively high importance in the operation of the autonomous vehicle.

The memory device of the autonomous driving device may store the image data 50 received from the sensor by assigning different weights to each region. For example, a high weight is given to data in an area that includes the preceding vehicle 52 and the driving vehicle 53, and a low weight is given to data in an area where the front 51 and background 54 of an autonomous vehicle appear. can be given.

Hereinafter, operations according to various embodiments may be understood as being performed by the autonomous driving device or a processor included in the autonomous driving device.

Figure 4 is an example diagram schematically showing the environment around a vehicle driving path according to an embodiment.

Referring to FIG. 4, various objects 410, 420, 430, and 440, including vehicles and pedestrians, may exist around the driving path of the vehicle 400. Although only pedestrians around the driving path are shown in Figure 4, the method according to the present invention can be applied to vehicles as well as pedestrians. In other words, the explanation regarding pedestrians described later can also be applied to vehicles.

In order for the vehicle 400 to drive safely, the behavior of pedestrians 410, 420, 430, and 440 must be taken into consideration. Accordingly, in one embodiment, a device (hereinafter referred to as 'device') that determines driving speed using pedestrian intent estimation according to the present invention sets the range to be considered when driving the vehicle 400 as an area of interest (not shown). Thus, the driving speed of the vehicle 400 can be determined based on the area of interest (not shown).

In one embodiment, the device may recognize pedestrians 410, 420, 430, and 440 present around the vehicle 400. As described above, the device may include various sensors (including cameras) for collecting surrounding situational information, and the sensors may collect information about the surroundings of the vehicle 400 to detect pedestrians 410, 420, 430, 440. ) can be recognized. Information about the surroundings of vehicle 400 may include recognition data and status data.

Pedestrians 410, 420, 430, and 440 recognized by the device may be in a stationary or moving state depending on whether they are moving, and when in a moving state, they may have lateral speed and longitudinal speed depending on the direction of movement.

Additionally, pedestrians 410, 420, 430, and 440 may have coordinates relative to the vehicle. The coordinates may be Cartesian frame-based coordinates or Fresne frame-based coordinates.

A Cartesian frame refers to a coordinate system that represents a coordinate plane or coordinate space using axes that are orthogonal to each other. When the driving environment of a road is configured as a Cartesian frame, the x-axis and y-axis can be formed regardless of the shape of the road.

A Fresne frame refers to a coordinate system defined by a unit tangent vector, a principal unit normal vector, and a binormal vector, and can describe the movement of a curve with curvature. When the driving environment of a road is configured with a Fresne frame, an s-axis may be formed in the direction of the road shape, and a d-axis may be formed in a direction perpendicular to the road. That is, the s coordinate represents the travel length and the d coordinate can represent how far away from the travel path in the lateral direction.

In one embodiment, pedestrians 410, 420, 430, and 440 may each have a corresponding pedestrian intent vector and pedestrian intent.

The pedestrian intention vector may be a normalized vector that estimates the probability distribution for multiple intentions that a pedestrian may have, and is a vector with dimensions corresponding to the multiple intentions. In addition, the pedestrian intention is an estimated value of the intention of the pedestrians 410, 420, 430, and 440, which is unobservable and can be defined in a discrete space where the number of actions is finite.

In one embodiment, the plurality of intentions may include a first intention to traverse the travel path of the vehicle 400. For example, the device determines that there is a high probability that the pedestrian 430 intends to cross the driving path of the vehicle 400 in light of the lateral position, vertical position, lateral speed, and degree of dependence according to the coordinates of a random pedestrian 430. Then, the device can calculate a high value for the first intention of the pedestrian intention vector of the corresponding pedestrian 430.

In one embodiment, the plurality of intentions may include a second intention to wait without crossing the travel path of the vehicle 400. For example, the device determines the probability that the pedestrian 410 intends to wait without crossing the driving path of the vehicle 400 in light of the lateral position, vertical position, lateral speed, and dependency according to the coordinates of a random pedestrian 410. If determined to be high, the device may calculate a high value for the second intention of the pedestrian intention vector of the corresponding pedestrian 410.

In one embodiment, the plurality of intentions may include a third intention excluding the first intention and the second intention. For example, the device determines the pedestrian's (420, 440) intention to cross the driving path of the vehicle (400) in light of the lateral and vertical positions, lateral speed, and dependence according to the coordinates of any pedestrian (420, 440). If it is determined that there is a high probability that the intention is not a factor in the driving of the vehicle 400, other than the intention to wait without crossing, the device assigns a high value to the third intention of the pedestrian intention vector of the corresponding pedestrian 420, 440. It can be calculated.

In one embodiment, the device may estimate or determine the intent corresponding to the maximum value of the pedestrian intent vector as the pedestrian intent. For example, when the first intention is named 'CROSSING', the second intention is 'WAIT', and the third intention is 'NONE', the pedestrian intention vector can be defined as {CROSSING, WAIT, NONE}. Assuming that the device estimates the pedestrian intention of a pedestrian 430 around the vehicle 400 and determines that there is a high probability that the device intends to cross the driving path of the vehicle 400, the pedestrian intention vector of the pedestrian 430 can be calculated as {0.8, 0.15, 0.05}, and the first intention corresponding to the maximum value of 0.8 among the plurality of intentions (CROSS, WAIT, NONE) of the pedestrian intention vector is estimated as the pedestrian intention of the pedestrian 430. Or you can decide.

Figure 5 is a conceptual diagram of a method for determining driving speed using pedestrian intention estimation according to an embodiment.

Referring to FIG. 5, the device may include a recognition module (Observer) 510, a pedestrian intention estimation module (Pedestrian Intention Estimator) 520, and a driving speed determination module (Motion Planner) 530. For convenience of explanation, each module is described as performing a separate function, but each module may perform the same function or may not be a separate module.

In one embodiment, the recognition module 510 may recognize pedestrians around the vehicle, obtain recognition data 540 and state data 540, and transmit them to the pedestrian intent estimation module 520.

In one embodiment, the recognition module 510 may obtain state data 540 in state space. State data 540 refers to all information about the current state of the surrounding environment. That is, at one point in time, the surrounding environment and the state data 540 may have the same meaning. For example, the state space may include coordinate information including the lateral and vertical positions of the pedestrian, and speed information including the lateral speed and dependence of the pedestrian.

Additionally, the state space may include the pedestrian's current intention (not shown). Specifically, the pedestrian intention (not shown) at the current point in time may mean the pedestrian intention estimated by the device at a previous point in time. Descriptions of the previous and current times will be provided later.

In one embodiment, the recognition module 510 may acquire recognition data 540 in recognition (or observation) space. The recognition data 540 is a subset of the state data 540 and may not have some information compared to the state data 540. For example, the recognition space may include coordinate information including the lateral and vertical positions of the pedestrian, and speed information including the lateral speed and dependence of the pedestrian. On the other hand, unlike the state space, pedestrian intent, which is unobservable and defined in a discontinuous space, is not included in the recognition space.

In one embodiment, the pedestrian intent estimation module 520 may update the pedestrian intent vector through at least one of the recognition data 540 and the state data 540 received from the recognition module 510.

In one embodiment, the device may determine an update cycle. The update cycle may refer to the cycle of determining the vehicle's driving speed by updating the pedestrian's intention. As the update cycle is determined, the timing at which the recognition module 510 acquires the recognition data 540 and the state data 540 is determined, and the pedestrian intent estimation module 520 updates the pedestrian intent vector or pedestrian intent ( The point in time to estimate 550) can be determined. In other words, the recognition module 510 acquiring the recognition data 540 and state data 540 may mean acquiring the current state data 540 and the current recognition data 540.

Meanwhile, the point in time after one update cycle based on the current point of time can be defined as the next point in time, and the point in time before one update cycle based on the current point in time can be defined as the previous point in time. Therefore, the next time point can be obtained as the device determines the update cycle.

In one embodiment, the pedestrian intent estimation module 520 may update the pedestrian intent vector at the current time to the pedestrian intent vector at the next time based on the update cycle.

For example, the pedestrian intention estimation module 520 may apply Bayes' Rule to update the pedestrian intention vector at the current time to the pedestrian intention vector at the next time. Bayes' rule is a rule that expresses the relationship between the prior and posterior probabilities of two random variables. Using Bayes' rule, it is possible to calculate posterior probabilities containing uncertainty from prior probabilities whose values can be found through observation. Therefore, the pedestrian intention estimation module 520 calculates the pedestrian intention vector at the next time from the recognition data 540 and/or the pedestrian intention vector at the current time using Bayes' rule, and calculates the pedestrian intention vector at the current time. can be updated with the pedestrian intent vector at the next point in time.

For example, the pedestrian intention estimation module 520 has an observation probability function that uses recognition data 540 at the current time as input and a transition probability function that uses state data 540 at the current time as input. Based on the function), the probability of obtaining state data at the next time point can be calculated.

The state transition function refers to a function that calculates the probability of obtaining state data at the next time point with respect to the state data 540 at the current time point. Specifically, the state transition function may be a function that calculates the conditional probability of obtaining state data at the next time using the state data 540 at the current time as a sample space. Therefore, the state transition function may be a function related to state data at the current time and the pedestrian intention vector at the current time.

The observation function refers to a function that calculates the probability of obtaining recognition data 540 at the current time with respect to state data at the next time. Specifically, the observation function may be a function that calculates the conditional probability of obtaining recognition data 540 at the current time using the state data at the next time as a sample space.

In one embodiment, the state transition function and the observation function may be functions designed empirically using Gaussian Distribution. Alternatively, the observation function may be a function designed based on performance indicators of the recognition module 510, such as recognition performance for pedestrians, standard deviation, and degree of noise.

In summary, the pedestrian intention estimation module 520 can calculate the probability of obtaining state data at the next time with respect to the recognition data 540 at the current time and the pedestrian intention vector at the current time.

In one embodiment, the pedestrian intention estimation module 520 may update the pedestrian intention vector of the current time point to the pedestrian intention vector of the next time point based on the calculated probability of obtaining state data of the next time point. The pedestrian intention vector at the next time point is a vector that estimates the probability distribution of multiple intentions that the pedestrian may have at the next time point.

In one embodiment, the device may update the pedestrian intent vector at the next time point based on the pedestrian model (Pedestrian model). The pedestrian behavior model is a model that analyzes the characteristics and behavior patterns of pedestrians and predicts their behavior at the next time based on their behavior at the current time. Specifically, by using a pedestrian behavior model that exists in response to each pedestrian intention, the behavior at the next time can be predicted for the pedestrian intention vector, and based on the predicted behavior, a plurality of intentions can be determined from the state transition function and observation function. Probabilities can be calculated.

The device may determine the probability of each of the pedestrian's plural intentions at the next time point based on the pedestrian behavior model, and update the pedestrian intention vector through this. In other words, the device can design a state transition function using the pedestrian behavior model and update the pedestrian intention vector.

In one embodiment, the pedestrian intention estimation module 520 estimates the pedestrian intention 550 at the next time point based on the updated pedestrian intention vector, and uses the estimated pedestrian intention 550 at the next time point in the driving speed determination module 530. ) can be passed on.

For example, the pedestrian intention estimation module 520 may estimate the intention corresponding to the maximum value of the updated pedestrian intention vector as the pedestrian intention 550 at the next time point. Specifically, if the pedestrian intention vector is defined as {CROSSING, WAIT, NONE} and the pedestrian intention vector at the next time of a certain pedestrian is {0.8, 0.15, 0.05}, the pedestrian intention 550 of the next time of the pedestrian is " It could be “CROSSING.”

Figure 6 is a diagram for explaining the first to third intentions of a pedestrian according to an embodiment.

The device may recognize a pedestrian 630 estimated to have a first intent, a pedestrian 610 estimated to have a second intent, and pedestrians 620 and 640 estimated to have a third intent around the vehicle 600. Among the descriptions of the vehicle 600 and pedestrians 610, 620, 630, and 640 in FIG. 6, descriptions that overlap with the descriptions of the vehicle 400 and pedestrians 410, 420, 430, and 440 in FIG. 4 are omitted. Do this.

When the device determines the driving speed of the vehicle 600, the intention of the pedestrians 610, 620, 630, and 640 must be taken into consideration in order to provide a safe driving plan to both passengers and pedestrians 610, 620, 630, and 640. there is.

The pedestrian 630, which is presumed to have the first intention, determines the driving path of the vehicle 600 at the next time in light of the lateral and vertical positions of the pedestrian 630 at the current time and the lateral speed and dependency of the pedestrian 630 at the current time. Since there is a high probability that it will be located on the pedestrian movement model, the first intention among the pedestrian intention vectors calculated based on the pedestrian behavior model may have the maximum value.

As described above, the pedestrian 610 presumed to have the second intention has a low probability of being located on the driving path of the vehicle 600 at the next time in light of the information at the current time of the pedestrian 610, but the distance from the driving path is low. Since there is a certain probability that will be located on the driving path at or near the time, the second intention among the pedestrian intention vectors calculated based on the pedestrian behavior model may have the maximum value.

Pedestrians 620, 640 estimated to have a third intention have no probability of being located on the driving path of the vehicle 600 at the next time, based on the information of the pedestrians 620, 640 at the current time, or have a speed in the opposite direction to the driving path. Since there is no need to consider it in the driving speed plan of the vehicle 600, the third intention among the pedestrian intention vectors calculated based on the pedestrian behavior model may have the maximum value.

In one embodiment, the device determines the intent corresponding to the maximum value of the pedestrian intent vector, i.e., the pedestrian intent of the next view of the pedestrian 630 is the first intent, and the pedestrian intent of the next view of the pedestrian 610 is the second intent. The pedestrian's intention at the next point in time (620, 640) can be estimated as the third intention.

In one embodiment, the driving speed determination module may determine the driving speed of the vehicle by considering the pedestrian intention at the next time point received from the pedestrian intention estimation module.

For example, the driving speed determination module determines the weight of the pedestrians (610, 620, 630, 640) from the estimated pedestrian intention, and determines the driving speed of the vehicle based on the weight of the pedestrians (610, 620, 630, 640). You can decide.

Referring to FIG. 6 , the device may determine to assign greater weight to the pedestrian 630 estimated to have the first intent than to the pedestrian 610 estimated to have the second intent. Additionally, the device may determine to assign greater weight to the pedestrian 610 estimated to have the second intent than to the pedestrians 620 and 640 estimated to have the third intent.

The weight given to the pedestrians 610, 620, 630, and 640 may determine how much the behavior of the pedestrian is reflected in determining the driving speed of the vehicle 600. In one embodiment, acceleration and deceleration can be determined conservatively for the pedestrian 630 estimated to have the first intention, and the driving speed can be determined with uncertainty for the pedestrian 610 estimated to have the second intention. The driving speed may be determined with relatively little reflection or little consideration of the behavior of the pedestrians 620 and 640, which are assumed to have a third intention.

In one embodiment, the device may repeat the above-described steps or the above-described embodiment at every time point determined based on the update cycle. For example, the device recognizes pedestrians 610, 620, 630, and 640 at each time point to obtain recognition data and status data at the current time point, and through this, pedestrians 610, 620, 630, and 640 The pedestrian intention vector is updated by estimating the probability distribution for the plurality of intentions at the next time point, the pedestrian intention at the next time point is estimated based on the updated pedestrian intention vector, and the driving speed of the vehicle 600 is calculated by taking this into account. can be decided. The device can drive to the next point in time according to the determined driving speed and then repeat the above steps at the next point in time to determine the driving speed of the vehicle 600 at the time when two update cycles have elapsed based on the current point in time. Accordingly, the driving speed can be determined to reflect the intention of the pedestrians 610, 620, 630, and 640, which changes at each time point.

In one embodiment, assuming that “Cross” means the first intention, “Wait” means the second intention, and “None” means the third intention, the device assigns the greatest weight to the pedestrian estimated to have the first intention. This allows you to determine the driving speed. In addition, the device can determine the driving speed by assigning a greater weight to the pedestrian estimated to have the second intention than the weight estimated to be the third intent, and the device can determine the driving speed by assigning the smallest weight to the pedestrian estimated to have the third intention. You can decide the speed. In other words, the device assigns conservative weights to pedestrians estimated to have the first intention, and weights that rapidly decrease as the distance from the vehicle 600 increases to pedestrians estimated to have the third intention, thereby adjusting the driving speed. You can decide. However, the method by which the device determines the weight of a pedestrian is not limited to this.

Figure 7 is a flowchart of a method for determining driving speed using pedestrian intention estimation according to an embodiment.

The method of determining the driving speed using pedestrian intention estimation shown in FIG. 7 is related to the previously described embodiments, so even if the content is omitted below, the content described above can also be applied to the method of FIG. 7. there is.

The operations shown in FIG. 7 can be executed by the above-described device. Specifically, the operations shown in FIG. 7 may be executed by a processor included in the above-described device.

In step 710, the device may recognize a pedestrian and obtain recognition data and status data.

In one embodiment, the recognition data is obtained in a recognition space consisting of at least one of the lateral position of the pedestrian, the vertical position of the pedestrian, the lateral speed of the pedestrian, and the degree of dependence of the pedestrian, and the state data includes the lateral position of the pedestrian, the vertical position of the pedestrian, It can be obtained from a state space consisting of at least one of the pedestrian's lateral speed, the pedestrian's dependence, and the pedestrian's intention.

In one embodiment, the device may acquire recognition data at the current time and state data at the current time.

In step 720, the device may update the pedestrian intention vector by estimating a probability distribution for a plurality of intentions that the pedestrian may have through at least one of recognition data and state data.

In one embodiment, the device may determine an update cycle and obtain the next time point based on the update cycle.

In one embodiment, the device may update the pedestrian intent vector at the current time to the pedestrian intent vector at the next time based on the update cycle.

In one embodiment, the device may calculate the probability of obtaining state data at the next time with respect to the recognition data at the current time and the pedestrian intention vector at the current time. For example, the probability of obtaining state data at the next time point can be calculated based on an observation function that takes recognition data of the current time point as input and a state transition function that takes the state data of the current time point as an input.

In one embodiment, the state transition function may be a function that calculates the probability that state data at the next time point will be obtained with respect to the state data at the current time point, and the observation function may be a function that calculates the probability that the state data at the next time point is obtained with respect to the state data at the next time point. It may be a function that calculates the probability of occurrence.

In one embodiment, the device may update the pedestrian intent vector for the next time point based on the probability that state data for the next time point will be obtained.

In one embodiment, the device may determine the probability of each of a plurality of intents based on a pedestrian behavior model.

In one embodiment, the plurality of intentions may include at least one of a first intention, a second intention, and a third intention.

In step 730, the device may estimate the pedestrian intent at the next time point based on the updated pedestrian intent vector.

In one embodiment, the device may estimate the intent corresponding to the maximum value of the updated pedestrian intent vector as the pedestrian intent at the next time point.

In step 740, the device may determine the driving speed of the vehicle by considering the pedestrian's intention at the next time.

In one embodiment, the device may determine the weight of the pedestrian from the estimated pedestrian intent and determine the driving speed of the vehicle based on the weight.

In one embodiment, the device may repeat steps 710 to 740 at every time determined based on the update cycle.

Figure 8 is a block diagram of an apparatus for determining driving speed using pedestrian intention estimation according to an embodiment.

Referring to FIG. 8, the device 800 may include a communication unit 810, a processor 820, and a DB 830. In the device 800 of FIG. 8, only components related to the embodiment are shown. Accordingly, those skilled in the art can understand that other general-purpose components may be included in addition to the components shown in FIG. 8.

The communication unit 810 may include one or more components that enable wired/wireless communication with an external server or external device. For example, the communication unit 810 may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiver (not shown). In one embodiment, the communication unit 810 can receive traffic information and use it to recognize pedestrians around the driving path.

The DB 830 is hardware that stores various data processed within the device 800, and can store programs for processing and control of the processor 820.

The DB 830 is a random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- It may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory.

The processor 820 controls the overall operation of the device 800. For example, the processor 820 can generally control the input unit (not shown), display (not shown), communication unit 810, DB 830, etc. by executing programs stored in the DB 830. The processor 820 may control the operation of the device 800 by executing programs stored in the DB 830.

The processor 820 may control at least part of the operation of the device for determining the driving speed described above in FIGS. 1 to 7.

The processor 820 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be implemented using at least one of micro-controllers, microprocessors, and other electrical units for performing functions.

In one embodiment, device 800 may be a mobile electronic device. For example, the device 800 may be implemented as a smartphone, tablet PC, PC, smart TV, personal digital assistant (PDA), laptop, media player, navigation, device equipped with a camera, and other mobile electronic devices. Additionally, the device 800 may be implemented as a wearable device such as a watch, glasses, hair band, or ring equipped with communication functions and data processing functions.

In another embodiment, device 800 may be an electronic device embedded within a vehicle. For example, the device 800 may be an electronic device inserted into a vehicle through tuning after the production process. In this case, the locations of the device 800 and the above-described vehicle (or autonomous vehicle) may be the same.

In another embodiment, device 800 may be a server located outside the vehicle. A server may be implemented as a computer device or a plurality of computer devices that communicate over a network to provide commands, codes, files, content, services, etc. The server can receive data necessary to determine the driving speed from devices mounted on the vehicle, and determine the driving speed based on the received data.

In another embodiment, the process performed in the device 800 may be performed by at least some of a mobile electronic device, an electronic device embedded in the vehicle, and a server located outside the vehicle.

Embodiments according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. At this time, the media includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROM. , RAM, flash memory, etc., may include hardware devices specifically configured to store and execute program instructions.

Meanwhile, the computer program may be designed and configured specifically for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer programs may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

According to one embodiment, methods according to various embodiments of the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or between two user devices. It may be distributed in person or online (e.g., downloaded or uploaded). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Unless there is an explicit order or statement to the contrary regarding the steps constituting the method according to the invention, the steps may be performed in any suitable order. The present invention is not necessarily limited by the order of description of the above steps. The use of all examples or illustrative terms in the present invention is simply for explaining the present invention in detail, and the scope of the present invention is not limited by the examples or illustrative terms unless limited by the claims. Additionally, those skilled in the art will recognize that various modifications, combinations and changes may be made depending on design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the scope of the claims are within the scope of the spirit of the present invention. It will be said to belong to

Claims (11)

In the method of determining the driving speed,
(a) recognizing a pedestrian and obtaining recognition data and status data;
(b) updating a pedestrian intention vector by estimating a probability distribution for a plurality of intentions that the pedestrian may have through at least one of the recognition data and the state data;
(c) estimating pedestrian intent at a next time point based on the updated pedestrian intent vector; and
(d) determining the driving speed of the vehicle in consideration of the pedestrian's intention at the next point in time;
A method for determining driving speed, including. According to claim 1,
The recognition data is,
Obtained in a recognition space consisting of at least one of the lateral position of the pedestrian, the vertical position of the pedestrian, the lateral speed of the pedestrian, and the degree of dependence of the pedestrian,
The status data is,
Obtained in a state space consisting of at least one of the lateral position of the pedestrian, the longitudinal position of the pedestrian, the lateral speed of the pedestrian, the degree of dependence of the pedestrian, and the pedestrian intention,
How to determine driving speed. According to claim 1,
In step (b),
determining an update cycle; and
updating the pedestrian intent vector at the current time to the pedestrian intent vector at the next time based on the update cycle;
A method for determining driving speed, including. According to claim 3,
In step (a),
Obtaining recognition data at the current time and state data at the current time;
It further includes,
In step (b),
Obtaining a next time point determined based on the update cycle;
calculating a probability of obtaining state data at a next time with respect to the recognition data at the current time and the pedestrian intention vector at the current time; and
updating the pedestrian intention vector of the next time point based on the probability of obtaining state data of the next time point;
A method for determining driving speed, including. According to claim 4,
The step of calculating the probability of obtaining state data at the next time point is,
Calculating the probability of obtaining the state data at the next time point based on an observation function that inputs the recognition data at the current time point and a state transition function that inputs the state data at the current time point,
The state transition function is a function that calculates the probability of obtaining state data at the next time point with respect to the state data at the current time point,
The observation function is a function that calculates the probability of obtaining recognition data at the current time with respect to the state data at the next time point,
How to determine driving speed. According to claim 1,
In step (b),
Including; determining a probability of each of the plurality of intentions based on a pedestrian behavior model,
The plurality of intentions includes at least one of a first intention to cross the travel path of the vehicle, a second intention to wait without crossing the travel path of the vehicle, and a third intention excluding the first intention and the second intention. doing,
How to determine driving speed. According to claim 1,
The step (c) is,
estimating the intent corresponding to the maximum value of the updated pedestrian intent vector as the pedestrian intent at the next time point;
A method for determining driving speed, including. According to claim 1,
In step (d),
determining a weight of the pedestrian from the estimated pedestrian intent; and
determining a driving speed of the vehicle based on the weight;
A method for determining driving speed, including. According to claim 3,
The above method is,
(e) repeating steps (a) to (d) at each time point determined based on the update cycle;
A method for determining driving speed, further comprising: In the device for determining driving speed,
a memory in which at least one program is stored; and
A processor that operates by executing the at least one program;
The processor,
Recognize pedestrians to obtain recognition data and status data,
Update a pedestrian intention vector by estimating a probability distribution for a plurality of intentions that the pedestrian may have through at least one of the recognition data and the state data,
Estimate the pedestrian intention at the next time based on the updated pedestrian intention vector,
Determining the driving speed of the vehicle considering the pedestrian's intention at the next point in time,
A device that determines driving speed. A computer-readable recording medium recording a program for executing the method of claim 1 on a computer.

Images