KR101982298B1 - Method and apparatus for detecting sudden moving objecj appearance at vehicle - Google Patents

Abstract

In accordance with an embodiment of the present invention, there is provided a method for detecting appearance of a moving object in a vehicle, the method comprising: a first learning classification model for classifying a travelable area and a remaining area on a road; Extracting a travelable area and an obstacle area on the road from a traveling image photographed at the time of traveling; Extracting a boundary for dividing a road and an obstacle according to a predetermined constraint condition as a boundary for dividing the energy function of the pixels constituting the boundary into a minimum; Estimating heights of the obstacles located at the partition boundary, and specifying a region where the height change of the obstacle is equal to or greater than a threshold value as a potential danger area; And detecting a moving object by applying a moving object detection algorithm based on image processing to the moving image, wherein the step of detecting the moving object includes applying a moving object detecting algorithm to the potential dangerous area among the running images with the highest priority , And detecting the moving object by setting the arrangement density of the sliding window to the highest.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for detecting an abruptly emerging moving object in a vehicle,

The present invention relates to a sudden pedestrian emergence warning method and system based on understanding the driving environment.

Conventional advanced driver assistance system (ADAS) technologies for preventing pedestrian collision in the finished car market have been developed based on distance sensors. Typically, ADAS technology has been used with laser sensors and Lidar.

ADAS technology based on distance sensors can accurately measure distances between surrounding obstacles and objects. Therefore, the ADAS technology based on the distance sensor has an advantage that it is easy to detect the obstacle and control the vehicle accordingly. However, ADAS technology based on the distance sensor has very limited information coming into the sensor, so it is difficult to understand the surrounding situation accurately. Also, the ADAS technology based on the distance sensor can be easily influenced by the shaking of the car body or the change of the weather condition depending on the sensor type. The ADAS technology based on the distance sensor must be installed in a form exposed to the outside of the vehicle to ensure precision, so the outer shape of the vehicle body must be deformed and the installation cost is high

Therefore, recently, ADAS technology such as a pedestrian collision prevention technique based on a computer vision technique using an image sensor such as a black box is attracting attention. Computer vision based pedestrian collision avoidance techniques typically perform a search for pedestrians based on a sliding window and notify a dangerous situation when pedestrians are detected on a driving route. However, the conventional techniques require a large amount of calculation in that the entire image must be searched, and misunderstandings may occur frequently depending on the driving environment.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems of the conventional art, and provides a sudden pedestrian appearance warning method and system based on understanding of the driving environment.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided a method for detecting appearance of a moving object in a vehicle, the method comprising the steps of: Extracting a travelable area and an obstacle area on the road from a traveling image photographed during driving based on a second learning classification model for extracting a model and an obstacle area; Extracting a boundary for dividing a road and an obstacle according to a predetermined constraint condition as a boundary for dividing the energy function of the pixels constituting the boundary into a minimum; Estimating heights of the obstacles located at the partition boundary, and specifying a region where the height change of the obstacle is equal to or greater than a threshold value as a potential danger area; And detecting a moving object by applying a moving object detection algorithm based on image processing to the moving image, wherein the step of detecting the moving object includes applying a moving object detecting algorithm to the potential dangerous area among the running images with the highest priority , And detecting the moving object by setting the arrangement density of the sliding window to the highest.

According to another aspect of the present invention, there is provided an apparatus for detecting appearance of a moving object in a vehicle, the apparatus comprising: a memory storing a running image received from a camera disposed in an autonomous vehicle, And a processor for executing a risk detection operation from the running image by executing a program stored in a memory, wherein the processor includes: a first learning classification model for classifying the travelable area and the remaining area on the road; and a second learning Based on the classification model, a travelable area and an obstacle area are extracted from a road from a photographed image, and a division boundary for dividing a road and an obstacle is extracted according to a predetermined constraint condition, and an energy function of the pixels forming the boundary is extracted Extracts a boundary to be the minimum as the division boundary, Estimating a height of an obstacle located at a boundary, specifying a region where a height change of the obstacle is equal to or greater than a threshold value as a potential dangerous area, applying a moving object detection algorithm based on image processing to the road image, The processor may be a moving object detection device that applies the moving object detection algorithm to the potential dangerous area with the highest priority among the images, or detects the moving object by setting the arrangement density of the sliding window to the highest.

The present invention recognizes the surrounding environment information and extracts the boundary between the obstacle area and the travelable area. It is possible to improve the warning accuracy of the appearance of the pedestrian in an unexpected situation.

The present invention estimates a potential incipient occurrence area along the extracted boundary line. If the pedestrian is searched using the estimated area, the search can be performed more effectively than searching the whole area of the image in the conventional method. Therefore, the present invention can reduce the amount of calculation. In addition, the present invention proposes a potential risk area estimation method based on image segmentation and obstacle height estimation rather than distance variation information, and can more accurately perform danger area estimation than existing methods even in various driving environments. The present invention also has the effect of further reducing the amount of processing computation by expressing an image as a low-dimensional graph based on temporal / spatial constraints in the image segmentation process.

As an incidental effect, the present invention has an advantage in that it can reduce the pedestrian and the misidentification of the dangerous situation in the process of extracting the surrounding situation information. In addition, since the information such as the road surface and the horizon line are handled together in the obstacle area estimation process, the present invention can reduce the problem of erroneously recognizing and warning a dangerous situation or misidentifying an object such as a tree or a pole as a pedestrian.

1 is an exemplary diagram illustrating a dangerous situation caused by a pedestrian according to an embodiment of the present invention.
2 is a block diagram of a system for detecting the appearance of moving objects in a vehicle according to an embodiment of the present invention.
3 is an exemplary diagram of color-based type information, which is one of the characteristics of learning information according to an embodiment of the present invention.
4 is an exemplary diagram of contour information, which is one of the characteristics of learning information according to an embodiment of the present invention.
5 is an exemplary diagram of optical flow information, which is one of the characteristics of learning information according to an embodiment of the present invention.
6A and 6B are views illustrating the results of estimating a boundary between an obstacle and a road according to an embodiment of the present invention.
7 is a diagram illustrating an example of height estimation of an obstacle according to an embodiment of the present invention.
Figure 8 is an illustration of a potential hazard area according to one embodiment of the present invention.
9A and 9B are diagrams illustrating an information providing method according to an embodiment of the present invention.
10 is a flowchart illustrating a process of detecting a moving object according to an embodiment of the present invention.
11 is a diagram illustrating a configuration of an apparatus for detecting the appearance of a moving object in a vehicle according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when a part is referred to as " including " an element, it does not exclude other elements unless specifically stated otherwise.

1 is an exemplary diagram illustrating a dangerous situation caused by a pedestrian according to an embodiment of the present invention.

First, referring to FIG. 1, a collision situation between a pedestrian and a vehicle can be defined in two forms.

The first situation is that the most part of the body of the pedestrian is exposed to the driver's gaze in a dangerous situation, so pedestrian detection is useful, and it is possible to judge the condition of the pedestrian through various analysis such as behavior recognition and trajectory analysis .

The following situation is that pedestrian's body is covered by an unexpected situation and it is difficult to judge the risk situation because there is not enough information to extract because it is not recognized as a normal pedestrian recognizer. Examples of unexpected situations include the appearance of sudden pedestrians or getting off the vehicle in an alleyway or around the vehicle.

Driving on a life zone road has a characteristic that the width of the road is narrow and the distance between the surrounding obstacle (parked vehicle, wall, etc.) and the driving vehicle is short. Therefore, when the warning is issued while the pedestrian completely enters the road, it may be difficult to braking the vehicle already. Therefore, it is important for pedestrians to quickly determine the situation in such a situation.

However, this unexpected situation occurs for a short period of time in a very short interval. And this unexpected situation is also the case that the driving vehicle is moving. Therefore, the system which warns of sudden situation must be processed within a very short time from the preprocessing to the final warning, which makes it difficult to handle with the existing pedestrian detection and collision prediction system.

Conventional techniques require a large amount of calculation because they sequentially search all the positions of the entire image. Here, the main factor of calculation time for risk situation judgment can be predominantly the time due to the detection time of the pedestrian and the movement of the search window. In the case of pedestrian detection time, GPU-based calculations and algorithms have been significantly improved, making further improvements difficult. Therefore, in order to reduce the computation time in the existing pedestrian detection framework, reducing the range of the candidate region to be used for the detection of the pedestrian can be an effective method for reducing the time.

The present invention proposes a system suitable for sudden situation warning by recognizing surrounding obstacle areas and reducing the number of pedestrian search candidate areas based thereon.

Conventionally, there is a method of defining a dangerous area and reducing the detection time of a pedestrian. However, the conventional method utilizes distance information or depth information based on a laser sensor or a binocular camera. The conventional method can not be performed in a monocular camera environment like a black box camera image. In addition, the conventional invention can define only a dangerous area in which a change in distance with time is large in the surrounding environment, and assign a temporal priority in the search for a pedestrian. Therefore, the conventional invention is difficult to cope with various unexpected situations that may occur in an actual driving environment.

The selection of the potential danger zone based on the local minimum point and the local maximum point of the distance variation may often result in the estimation of the wrong position as a dangerous area by various variables such as body movement and surrounding environment configuration in the driving situation . In addition, the conventional method is vulnerable to a situation caused by a changing obstacle (a car door upon getting off) rather than a fixed obstacle.

The present invention improves the problem related to the selection of the potential danger area by recognizing the position and the size of the obstacle by utilizing the context information along the boundary line of the obstacle-drivable area.

2 is a block diagram of a system for detecting the appearance of moving objects in a vehicle according to an embodiment of the present invention.

Referring to FIG. 2, a system for detecting the appearance of moving objects in the vehicle includes an image input module (a), a surrounding environment understanding module (b), a potential danger area estimation and pedestrian detection module (c) ). ≪ / RTI >

First, the video input module (a) plays a role of receiving the surrounding situation information of the moving vehicle. Therefore, the apparatus for photographing may require a camera capable of capturing an image of HD class or higher and an image of 30 fps or higher in consideration of the characteristics of an unexpected situation, but the present invention is not limited thereto. That is, the input image module (a) can warn a dangerous situation even at a lower resolution than the HD grade. In addition, the present invention is not limited to the single-camera camera for photographing the input image, and other constraints such as the multi-camera and the stereo camera.

Next, the image received through the image input module (a) is extracted from the feature information of each pixel through the surrounding environment understanding module (b). At this time, the extracted feature information is used to distinguish the region where the vehicle can travel from the image and the optimal region by using the previously learned prior information model in the present invention.

Here, in order to understand the surrounding environment, information for distinguishing the travelable area and the other area should be learned in advance. For example, the area, obstacle, and other areas in which the vehicle can travel can be expressed in the form of a divided image through the shape of the color of the surrounding environment or the outline light flow information, and a detailed description thereof will be described later .

After that, the pedestrian detection area is reduced based on the boundary between the travelable area and the obstacle area, and the pedestrian detection module and the pedestrian detection module perform the role of detecting the pedestrian.

Finally, the risk warning module (d) provides information on the surrounding environment so that the driver can take precautions while driving in the previously estimated potential danger area. At this time, the information can be transmitted to the driver through an acoustic signal such as a display or a warning sound, but the warning method does not limit the present invention.

FIGS. 3 to 5 are exemplary views of feature information necessary for an apparatus of the present invention to learn the surrounding environment, according to an embodiment of the present invention. Therefore, before proceeding to the detailed description of each drawing, the structure of the learning data and the learning method will be described first.

The learning data may be composed of a frame image divided into the travelable area 301, the obstacle area 302 and the remaining area 303, the actual traveling image, and class information on the pixel position in the image. At this time, it is preferable that the learning data is learned by 30 minutes or more of data based on 30 fps.

According to an embodiment of the present invention, at least two following models can be used as a model for progressing learning through dictionary information.

A classification model that divides the travelable area 301 and the other areas (the obstacle area 302 and the remaining area 303) may be used first. The model can internally be trained using three features (color-based shape, contour, light flow).

The parameters of the model can be expressed by the mean and variance calculated using each feature. Model parameters can be learned through machine learning techniques such as binary classification or clustering methods. Examples of machine learning techniques include Gaussian Mixture Model (GMM), K-means Clustering, Fuzzy C Clustering, and Kernel Density Estimation ), RBF based Neural Network, and SVM.

Hereinafter, a method of acquiring parameters based on GMM among machine learning techniques will be described as an example.

Accordingly, the processor according to an embodiment of the present invention can set two Gaussian Mixtures and assign the entire learning data to each Mixture. At this time, the allocation criterion can be determined by using the arbitrary average and dispersion values set at the initial stage, calculating the probability distribution for each mixture, and allocating the data to the mixture having a high probability value. This process can be repeatedly performed through the EM algorithm to estimate the optimal probability distribution. Thus, parameters for two probability distributions that can best represent the entire data can be obtained. In addition, when the above-described process is repeated for each feature data, the average and variance for two classes can be obtained for one feature, so 12 parameters can be obtained for three features in total.

Next, a recognition model for extracting the obstacle area can be used. Recognition model models can be trained internally through three features (color-based shape, contour, light flow) as well as classification models.

Also, clustering or classification schemes mentioned above may be used for extraction of the obstacle region 302, and non-parametric estimation techniques such as KDE or Deep Neural Network may be used.

Hereinafter, the features used for learning about the model will be described in more detail.

3 is an exemplary diagram of color-based type information, which is one of the characteristics of learning information according to an exemplary embodiment of the present invention.

The color-based morphology information is a kind of segmentation information, and the pre-information is conceived in the form as shown in FIG. The color-based shape information may be color-based by labeling with three classes such as the travelable area 301, the obstacle area 302, and the remaining area 303 based on the average and variance values of the pixel values.

Secondly, FIG. 4 is an example of contour information, which is one of the features of learning information according to an embodiment of the present invention.

The outline information can represent the texture as information indicating the boundaries of the objects in the image or image. The contour information includes the shape and direction of the obstacle. Generally, the outline information is created in the form of a map as shown in FIG. 4 and then used as feature information.

5 is an exemplary diagram of optical flow information, which is one of the characteristics of learning information according to an embodiment of the present invention.

The optical flow information is feature point information obtained by extracting a position change of a specific frame in an image in a form of a vector (305). Feature points include force and direction, and are mainly based on moving objects. Therefore, the energy function based on the probability distribution can be calculated using the feature point distribution of the obstacle (object).

By using the three feature information described above, it is possible to distinguish between the travelable area 301 and the obstacle area 302. [ At this time, a boundary 304 can be set where the travelable area 301 and the obstacle area 302 are in contact with each other, which is shown in FIGS. 6A and 6B according to an embodiment of the present invention.

Referring to FIGS. 6A and 6B, the boundary 304 follows the following three conditions.

The first relates to the location of the boundary 304, where the obstacle-road boundary 304 should be below the horizon. This is because the vanishing point is below the horizon on a common life road, except in special cases such as when the vehicle enters a highway. Therefore, the condition must be defined such that the boundary 304 also lies below the horizon.

The second is that the points constituting the boundary 304 are continuous and should be located on the left-right side of the image. The road extends to the front of the running vehicle, where obstacles along the road exist along the road. Therefore, the boundary 304 in the present invention sets the left-right pixel information to be more important than the upper-lower pixel information based on the image. Also, in the present invention, the boundary 304 divides the image into columns as many as the pixel width, and each of the segments can be utilized as one graph node.

Third, it is assumed that the pixels constituting the boundary 305 are affected only by surrounding pixels. The reason for this is that assuming that certain pixels in the image are only affected by the surrounding pixels, it is possible to reduce the amount of computation required in assuming the probability model.

If the boundary of pixels of the whole image is estimated without a predetermined condition as in the conventional invention, it is difficult to real-time process because of a large amount of calculation, and it is difficult to guarantee an optimum value.

Accordingly, in the present invention, since the three constraints described above are set when setting the boundary 304, the calculation necessary for estimating the boundary 304 can be implied as a low dimensional graph segmentation problem. At this time, the low dimensional graph optimization problem can be solved based on existing sequence modeling or Bayesian based probability models.

For example, typical sequence modeling that solves low-dimensional graph optimization problem is structured continuous type such as Markov Random Field (MRF), Conditional Random Field (CRF), Hidden Markov Model (HMM) and Deep Recurrent Neural Network A data prediction technique may be used, and a Bayesian based probability model may be an MRF framework.

 The following explains an example of how to solve low-dimensional graph optimization problem with Bayesian-based probabilistic model MRF framework.

The MRF is used to calculate the energy function for the pixels constituting the divided boundary 304 on the image and to find the point where the value is minimum. In this case, the energy function expresses whether the feature value of the current pixels is the partition boundary based on the previously learned parameter in the form of a log function.

At this time, the energy function for segmentation of the image can be composed of two factors, "class belonging score" and "surrounding weight".

The class membership score refers to a probability distribution value that a particular pixel belongs to a specific class, which indicates whether the pixel at a specific position is suitable for a particular class. In addition, the surrounding weights are obtained by comparing the pixels of a specific position and the surrounding pixels based on the probability distribution, and then quantifying how the comparison results of the two pixels match. In this case, the surrounding weights are subjected to a "+ penalty" as the surrounding pixels and the inferred values do not match in the viewpoint of dividing the image, and the "- penalty" The value of the energy function becomes large, and the possibility that the pixel at the position is not selected as the division boundary becomes large.

Here, a Gaussian distribution can be used as a method for estimating the probability distribution, and a formula for obtaining a generalized energy function is shown in Equation 1 below.

[Equation 1]

From the viewpoint of image segmentation, the first term of Equation (1) means the class belonging score and the second term means the weighting score for the surrounding pixels.

In other words, energy function in the process of image segmentation is calculated for the entire image when the probability model parameters for data and features are given, and the degree of correspondence between the pixel belonging to each class and the surrounding pixels. Here, the lower the energy, the better the result of image segmentation.

A graph is constructed based on the energy function in the image, and the division boundary can be calculated. The value of the node of the graph is calculated by the class membership score of each pixel, and the relationship with neighboring pixels of each graph can be represented by the weight value of the trunk in the graph.

In this case, the nodes constituting the graph must exist at a position lower than the horizontal line in the image due to the first constraint described above, and the nodes have the form of a line crossing the left and right by the second constraint, And the pixel relation in the up-down direction may not be considered.

Through the above process, the present invention can perform a process of finding an image segmentation boundary 304 having a minimum energy with respect to a graph. At this time, the present invention can use a graph cut algorithm, a cutting plane algorithm, or the like as a method for finding an optimized solution.

Using the boundary 304, the obstacle region 302 and the height 306 of the obstacle region derived above according to an embodiment of the present invention, the search region of the moving object can be implied in the travelable region 301 . In addition, a potential danger area 307 is set as an area in which a moving object is highly likely to appear among the search areas of the moving object, thereby giving priority to the search.

Further, the potential risk area 307 can be searched using a generalized single framework processing technique.

More specifically, in order to infer the potential dangerous area 307, it is necessary to estimate the height 306 of the obstacle area around the boundary 304 by using the light flow, contour, and color information, . At this time, the reason for estimating the height of the obstacle is that it is difficult to accurately grasp the situation of the road only by the boundary 304 derived from the above. Especially, since the area around the obstacle and the obstacle is more likely to appear than the other areas, the area needs to receive a high weight in the search of the moving object.

Accordingly, FIG. 7 is an exemplary view of height estimation of an obstacle according to an embodiment of the present invention.

Referring to FIG. 7, the height 306 of the obstacle can be calculated in the longitudinal direction with respect to the boundary 304, according to an embodiment of the present invention.

At this time, in order to increase the accuracy with respect to the height, the degree of matching with the pixels of the obstacle region 302 is calculated using the color and contour information of the pixels existing in the region around the boundary 304, do.

Figure 8 is an illustration of a potential hazard area according to one embodiment of the present invention.

By using the height 306 of the obstacle area calculated as described above, the potential danger zone 307 can be estimated.

To be more specific, the potential dangerous area 307 can be set by a difference in height between the obstacle area 302 and the surrounding environment, which occurs at the time when the vehicle under running is viewed. In particular, the difference in height at which the potential hazardous area 307 begins to be set is possible through learning or setting an empirically derived threshold value.

A sliding window is arranged spatially around the potential dangerous area 307 and the sliding windows are disposed at a lower density so that the driver can navigate around the potential dangerous area 307 more than other areas And then make it possible for the surroundings of the obstacle region 302 and the boundary 304 to be explored.

In the sliding window, a framework can be used to search for a moving object. A typical explorer can be a deformable model-based pedestrian recognizer capable of robust recognition of an obscured object.

9A and 9B are diagrams illustrating an information providing method according to an embodiment of the present invention.

According to an embodiment of the present invention, when a moving object is detected on the road under travel, the system provides dangerous situation information through the dangerous situation warning module (d) so that the driver can recognize it. At this time, information can be provided through a head-up display (HUD), a navigation screen, etc., and a display method or a warning indicating a dangerous situation to the driver is not limited to the present invention.

For example, in the following, an augmented reality based on a head-up display will be described as an example.

 The risk warning consists of two steps: providing the driving environment information in a safe situation and warning when an unexpected situation occurs. First, in the driving environment information providing step in a safe situation, the driver is informed about a safe driving area and a place to watch out for.

FIG. 9A is an exemplary view illustrating a driving environment information providing method in a safe situation according to an embodiment of the present invention. FIG.

Referring to FIG. 9A, the position of a potential dangerous area 307 in which a moving object around a traveling route can appear is displayed in a yellow box form 308, and a pedestrian warning sign can be displayed on the top have. At this time, since the signboard can obscure the driver's view, the present invention is displayed at the top of the cover, and warning is not displayed when the potential dangerous area is judged to be far away from the vehicle.

Further, it is possible to display a light green gradient in the travelable area 301, or a green state at the lower left of the HUD. This allows the driver to recognize that the surrounding environment is now safe.

9B is an exemplary diagram illustrating an information providing method in a situation where an unexpected situation occurs according to an embodiment of the present invention.

In contrast to the above-described case, when a moving object invades the travelable area 301, the driver can quickly guide the moving object to the moving object. At this time, the moving object may be displayed as a red box (309), and a stop sign may be displayed on the upper side thereof.

In addition, by displaying the travelable region 301 in a light red color gradient or displaying a red state in the lower left of the HUD, it is possible to prompt the driver to take a quick action.

10 is a flowchart illustrating a process of detecting a moving object according to an embodiment of the present invention.

Referring to FIG. 10, visual information on the road under travel is received through the camera module attached to the vehicle or the driver assistance system terminal 100 (S210).

The time information received in step S210 is divided into the travelable area 301 and the obstacle area 302 through an algorithm built in the processor (S220).

A boundary 304 between the two separated areas is set in step S230 and a threshold value according to the height of the obstacle area 302 is calculated based on the boundary 304 in step S240.

A step S250 is reached in which a potential danger area 307 is set as a part of the area covered by the obstacle according to the threshold value at step S250 and the detection of the moving object that can appear around the dangerous area is performed at step S260. .

11 is a diagram illustrating a configuration of a driver assistance system terminal that detects occurrence of a moving object in a vehicle according to an embodiment of the present invention.

Referring to FIG. 11, a terminal 100 for detecting the appearance of moving objects in a vehicle according to an embodiment of the present invention includes a processor 110, a memory 120, a camera module 130, and a display module 140 .

In detail, the memory 120 records a program for performing a method of detecting the appearance of moving objects in the vehicle. In addition, the processor 110 performs a function of temporarily or permanently storing data to be processed.

The processor 110 controls the entire process as a kind of central processing unit to provide a method for detecting the appearance of moving objects in a vehicle. The processor 110 controls all operations except those performed in the display module, the camera module, and the memory.

The display module 140 functions to display an interface for performing a method of detecting the appearance of moving objects in the vehicle. The output format of the display module 140 may have various forms as described above, and the present invention is not limited thereto.

The camera module 130 is an apparatus for photographing the surrounding environment to perform a method of detecting the appearance of moving objects in the vehicle, and performs the function of the image input module (a) described above.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. The computer-readable medium may also include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: Driver assistance system terminal
110: Processor 120: Memory
130: camera module 140: display module
301: travelable area 302: obstacle area
303: remaining area 304: border
305: Vector 306: height of obstacle
307: Potential Hazard Areas

Claims (12)

A method for detecting the appearance of a moving object in a vehicle,
Based on the first learning classification model for classifying the travelable area and the remaining area on the road, and the second learning classification model for extracting the obstacle area, the travelable area and the obstacle area on the road are extracted from the traveling image photographed during traveling step;
Extracting a boundary for dividing a road and an obstacle in accordance with a predetermined constraint condition as the dividing boundary so that an energy function of the pixels forming the boundary is minimized;
Estimating heights of the obstacles located at the divisional boundaries and specifying a region where the height variation of the obstacle is equal to or greater than a threshold value as a potential dangerous area; And
Detecting a moving object by applying a moving object detection algorithm based on image processing to the traveling image,
Detecting the moving object comprises detecting a moving object by setting a placement density of a sliding window to a highest level with respect to the potential dangerous area of the traveling image. The method according to claim 1,
The first learning classification model learns color flow, shape of object, and optical flow information in the image as feature information. Based on these feature information, the travelable area and the remaining area of each image are classified However,
Wherein the second learning classification model learns the optical flow information, texture information, and color information as feature information, and classifies the obstacle region of each image based on the feature information. The method according to claim 1,
And overlaying a highlight indication on the periphery of the detected moving object and outputting through the display. The method according to claim 1,
The step of extracting the division boundary
A first constraint that the boundary between the road and the obstacle is present at a position lower than or equal to the horizontal line, a point constituting the boundary between the road and the obstacle is continuous, And the third constraint condition that the points constituting the boundary line are affected only by surrounding pixels. The method according to claim 1,
The step of identifying as the potential danger zone
Estimating the height of each obstacle based on the degree of coincidence of the feature information while searching vertically upward from the boundary line based on the feature information of the preset image,
Wherein the feature information of the image includes light flow information, texture information, and color information. An apparatus for detecting the appearance of moving objects in a vehicle,
A memory for storing a traveling image received from a camera disposed in an autonomous vehicle and storing a program for implementing a risk detection method,
And a processor for executing a program stored in the memory to execute a danger detection operation from the running image,
The processor extracts a travelable area and an obstacle area from the road based on a first learning classification model for classifying the travelable area and the remaining area on the road and a second learning classification model for extracting the obstacle area on the road, Extracting a boundary for dividing roads and obstacles according to a preset constraint condition as a boundary for minimizing an energy function of pixels constituting the boundary, And detecting a moving object by applying a moving object detection algorithm based on image processing to the road image,
Wherein the processor detects the moving object by setting the placement density of the sliding window to the highest level with respect to the potentially dangerous area of the image. The method according to claim 6,
The first learning classification model learns color flow, shape of object, and optical flow information in the image as feature information. Based on these feature information, the travelable area and the remaining area of each image are classified However,
Wherein the second learning classification model learns the optical flow information, texture information, and color information as feature information, and classifies an obstacle region of each image based on the feature information. The method according to claim 6,
Wherein the processor overlays a highlight indication in the vicinity of the detected moving object and outputs it via a display disposed in the autonomous vehicle. The method according to claim 6,
Wherein the processor is a predetermined constraint condition that a boundary between the road and an obstacle exists at a position equal to or lower than a horizontal line, the points constituting the boundary between the road and the obstacle are continuous, Wherein the second constraint condition that the first boundary condition and the second boundary condition intersect each other in the horizontal direction and the points constituting the boundary line are affected only by surrounding pixels. The method according to claim 6,
The processor estimates the height of each obstacle on the basis of the degree of coincidence of the feature information while searching vertically upward from the boundary line based on the feature information of the preset image, and the feature information of the image includes optical flow information, texture information, And color information. The method according to claim 1,
Wherein the step of detecting the moving object comprises applying the moving object detection algorithm to the potential dangerous area among the traveling images with the highest priority. The method according to claim 6,
Wherein the processor detects the moving object by first applying the moving object detection algorithm to the potential dangerous area of the traveling image.

Images