KR20210006086A - Method and system for providing notification of lane change availability - Google Patents

Abstract

Disclosed are a method for providing a notification of whether a lane change is possible, and a system thereof. According to the present invention, the method for providing a notification of whether a lane change is possible comprises the steps of: determining a state related to a lane change by analyzing an image input through a camera in relation to a side-rear region of a vehicle; and providing a notification using different media according to a depth of information on whether a lane change is possible as a state related to the lane change.

Description

METHOD AND SYSTEM FOR PROVIDING NOTIFICATION OF LANE CHANGE AVAILABILITY}

The description below relates to a technology that provides notification of whether a lane change is possible.

Technologies for lane change assistance include blind spot detection (BSD) and blind spot information system (BLIS). However, in the case of a device (or system) that uses a radar sensor, these conventional technologies are expensive, and the device is buried in the exterior of the vehicle (for example, a bumper), so that it is damaged even in a minor accident or the calibration information is damaged. There is a problem that the replacement or reinstallation is required. Therefore, it has a disadvantage that the system installation and maintenance cost is expensive. On the other hand, the camera sensor has a competitive advantage in terms of price, and the proposed system has robust characteristics for the installed location. More specifically, in the case of a radar sensor, there is a disadvantage in that the recognition error of a stationary object and a very close object increases due to the physical characteristics of the sensor. In addition, the existing BLIS has a problem of notifying the driver only when it is impossible to change the lane according to the state of the blind spot of the lane to be changed and the side mirror.

A method and system capable of providing a lane change notification for a certain period of time from a lane change request point to a lane change completion point are provided.

A method and system capable of providing differentiated notifications according to a depth of state information on whether a lane change is possible is provided.

A method for providing a lane change notification executed in a computer device, wherein the computer device includes at least one processor configured to execute computer readable instructions included in a memory, and the lane change notification providing method comprises: the at least one processor By analyzing the image input through the camera in relation to the side-rear region of the vehicle, determining a state related to a lane change; And providing, by the at least one processor, a notification using different media according to a depth of information on whether a lane change is possible as a state related to the lane change. to provide.

According to one aspect, the state related to the lane change is divided into a state in which lane change is impossible (Block) and a state in which lane change is possible (Free), and a state in which the lane change is impossible is subdivided into n (where n is A natural number) may include a lower level divided into a state in which lane change is possible after a unit time.

According to another aspect, the providing may include outputting information on an upper level through a first display device installed in the vehicle, and outputting information on a lower level through a second display device installed in the vehicle. .

According to another aspect, in the providing step, the first display device installed in the vehicle is used as a means for emphasizing a state in which lane change is impossible, and the remaining time until the lane change is possible is emphasized. A second display device installed in the vehicle may be used.

According to another aspect, in the providing step, the notification may be provided for a predetermined time from a time when a lane change request is recognized to a time when a lane change completion is recognized.

According to another aspect, at least one of a case where a turn indicator of the vehicle is turned on and a case where a lane change guidance is output during navigation route guidance may be recognized as the lane change request.

According to another aspect, among the cases where the turn indicator of the vehicle is turned off, when it is determined that the lane has changed as a result of positioning for location coordinates, and when it is determined that the lane has changed through ADAS (Advanced Driving Assistance System) sensing. At least one may be recognized as the lane change completion.

According to another aspect, the providing may include outputting information on whether the lane change is possible through sound.

According to another aspect, the outputting may output sounds of different pitches according to a state in which lane change is impossible (Block) and lane change is possible (Free).

There is provided a non-transitory computer-readable recording medium in which a program for executing the lane change notification providing method on a computer is recorded.

A computer apparatus comprising: at least one processor configured to execute computer readable instructions contained in a memory, wherein the at least one processor analyzes an image input through a camera in relation to a side-rear region of the vehicle It provides a computer apparatus, comprising: determining a state related to a lane change, and providing a notification using different media according to a depth of information on whether a lane change is possible as a state related to the lane change.

According to embodiments of the present invention, a lane change notification may be provided for a predetermined period of time from a time when a lane change request is made to a time when a lane change is completed.

According to embodiments of the present invention, a differentiated notification may be provided according to a depth of state information on whether a lane change is possible.

1 is a view showing an example of annotated images according to an embodiment of the present invention.
2 shows an example of comparing the attention result of CAM with the attention result of the method according to an embodiment of the present invention.
3 is a diagram showing an example of an integration method according to an embodiment of the present invention.
4 is a block diagram showing an example of a computer device according to an embodiment of the present invention.
5 is a flowchart illustrating an example of a method of learning when to change lanes according to an embodiment of the present invention.
6 is a diagram illustrating an example of assigning a label according to an embodiment of the present invention.
7 is a flowchart illustrating an example of a method for providing a lane change notification according to an embodiment of the present invention.
8 is a diagram illustrating an example of a display device that outputs a lane change notification according to an embodiment of the present invention.
9 to 13 are diagrams showing an example of visualization of a lane change notification according to an embodiment of the present invention.
14 is a diagram illustrating another example of a display device that outputs a lane change notification according to an embodiment of the present invention.
15 is a diagram illustrating an example of a lane change notification using sound in an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, a method and system for learning a camera-based lane changeable time point will be described.

The method for learning a lane change point according to embodiments of the present invention may be performed by at least one computer device to be described later. At this time, a computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the lane change point learning method according to the embodiments of the present invention under control of the driven computer program. I can. The above-described computer program may be combined with a computer device and stored in a computer-readable recording medium to execute the method according to the embodiments of the present invention on the computer device.

In embodiments of the present invention, a weak supervision is used to overcome the extreme data imbalance issue by extracting vehicle-aware image features and a few-shot classification. We propose a system for lane-change assistance. For self-driving agents and human drivers, the proposed system can monitor side-to-rear spaces and predict when lane changes are possible. By using weakly supervised learning and attention maps, the system can extract new image features of objects moving behind an ego-vehicle. In addition to converting the binary labels into quantized time slots, a side-rear view image dataset to which binary labels (FREE or BLOCKED) are manually assigned may be reused. By using a few-shot classification method, it is possible to indirectly predict the state of the target lane and quickly adapt to various road scenarios, and the proposed system is spatial and spatial around the own-vehicle to change lanes. Can recognize temporal properties.

1. Introduction

For safety and convenience, lane-change decision aid is one of the key functions in ADAS (Advanced Driving Assistance System). Thus, automakers are trying to equip drivers with blind spot detection systems that alert and warn drivers of potential collisions in the side-to-rear space. It is also required that autonomous vehicles perform cooperative lane-change on roads where human drivers coexist. Because embedded computer vision systems are cost-effective, they are becoming more popular in implementing vehicle safety functions.

The end-to-end learning framework can classify spatial attributes of images to help determine lane-change. In this end-to-end learning framework, side-to-rear view images can be collected and annotated with binary classes (or labels) called FREE if the self-vehicle can move to the target lane, otherwise BLOCKED. While the side-to-rear view image dataset is a valuable asset for image-based lane-change studies, this binary classification is insufficient for interpreting various road scenarios, especially driving environments with interactions with other vehicles. Prior to a lane-changing action, a safe distance must be established between vehicles behind or in front of the target lane, and the definition of the safe distance is the relative speed and type of road between the two vehicles (e.g. urban road vs. highway). It can change according to. Therefore, a new system is required that can inform the remaining time to change lanes regardless of road conditions.

The success of computer vision algorithms based on supervised learning in recent years requires very large datasets. However, it is inefficient to collect all datasets and annotate each of them in all possible scenarios, especially in the area of intelligent vehicles. To reduce the burden on this location-level annotation task, weakly supervised learning methods for object localization have been studied. Interestingly, by designing image features through weak maps, the internal mechanisms of deep convolution neural networks can be understood. Based on such weak supervised learning, a vehicle-aware image feature extraction method that focuses on a primary target in a side-rear space may be proposed. Pixelwise regularization and multiscale aggregation will be described in order to focus on the overall shape of the vehicle rather than on some discriminating parts of the vehicle such as wheels and grills.

To make the system able to interpret temporal attributes as well as spatial attributes in the scene, we change the binary-class labels of the dataset to n (e.g., 5) class labels. , These represent the time remaining to perform a safe lane-change. This change task is done systematically and can alleviate the need for tedious manual annotation work. However, it is inappropriate to apply simple image classification to the data set because the modified data set will have a severe data imbalance with huge intra-class variation. Instead, the whole dataset is separated according to various road types and driving scenarios, so that the subset includes representative examples of various road environments and driving styles (few-shot classification scheme). ) Can be used. As a result, the classification method by fractional learning can enable the proposed system to quickly adapt to various road scenarios that it sees for the first time.

In the following, a new vehicle computer vision application that predicts when to change lanes, task-specific image features that focus on main goals based on weak supervised learning, and a lane-change system proposed in various road environments are applied. In order to explain the formalization of the decimal learning problem.

2. Related research

Deep learning for auto-driving. Large road datasets have led to a combination of intelligent vehicle and computer vision research. This has contributed to advances in machine perception algorithms such as detection, tracking, stereo matching, optical flow and semantic segmentation. Recently, some studies have used deep learning techniques to help computer vision applications jump from perception to control. For example, an end-to-end driving model that directly creates vehicle control factors such as steering angle from image input has been proposed, and generates structured calculation results for the controller of the auto-driving agent or reflects various driving modes. There are studies that use large-scale video datasets collected in the wild to implement a comprehensive model or simplify the lane-change decision problem to binary classification that measures the occupancy state of the target lane. However, a technique for estimating the time to change lanes has not been developed so far.

Weakly Supervised Object Localization. Existing weakly guided object location recognition methods can be classified into sequential approaches and integrated approaches. Sequential methods can first suggest a region where an object can appear, and then perform classification. The Multiple Instance Learning (MIL) method creates positive and negative labeled bags of the instance area to select some instances within the non-convex optimized positive label bags. do. Weakly Supervised Deep Detection Networks (WSDDN) use spatial pyramid pooling for domain proposals and combine recognition and detection values through elementwise multiplication.

On the other hand, the integrated methods simultaneously classify objects and recognize location through class-level saliency maps, which are secondary outcomes in the learning process. Class Activation Mapping (CAM) algorithms are fully connected layers for classification tasks to learn the adaptive weights of feature maps for class-specific activations. ) With a global average pooling layer. In order to get a fine-grained attention region, the derived backpropagation scores can be multiplied by the weighted feature map. However, since these algorithms create an importance map based on cross entropy for image classification, the activation result deals with some discriminant parts rather than the whole object. To solve this problem, the Hide-and-Seek algorithm was implemented to deliberately cover a part of the image in order to learn an error resilience model. The adversarial erasing technique has been proposed to allow the network to understand the complete body of the object by repeatedly erasing the most discriminating parts. In embodiments of the present invention, per-pixel attention regularization and multiscale inference are proposed in order to induce an attention map covering the entire object area without fragmenting parts. .

Meta-learning for classification by fractional learning. Meta-learning aims to resemble human intelligence that can quickly adapt to new tasks with a small number of examples. In computer vision, few-shot classification is a difficult task because the scarce data is insufficient to contain the pristine diversified visual features. To deal with unbalanced training data, a model regression network has been proposed to learn the classifier transformation from a small sample model to a large sample model. In Model-agnostic meta-learning, an objective function is defined that accepts parameters across all ancillary tasks. Through gradient-based optimization, parameters are sensitive to the loss of a new task, causing rapid adaptation of the update model. There is also a mechanism to train a model with unlabeled data. On the other hand, a non-parametric approach is to train an embedding function that generates a representative value for each class in units of distance. Embodiments of the invention follow a non-parametric approach as most drivers naturally react to new road conditions based on their previous driving experience.

3. Dataset

Previously, it has been described that a binary label indicating whether the target lane is free (FREE) or blocked (BLOCKED) can be assigned to each side-rear view image according to the spatial property of the scene. The system according to an embodiment may additionally use the temporal information of the scene to change the data set in order to predict the point at which the target lane becomes free for lane-change in a very short time. For example, images previously assigned the label BLOCKED may be remapped to one of 1s, 2s, 3s, and BLOCKED. Here, ns can indicate that the target lane is currently blocked but will be free within about n seconds. Meanwhile, all images to which the previous label FREE has been assigned may remain as it is. In order to avoid expensive and time-consuming manual annotation tasks, a systematic method of utilizing a set of original datasets can be utilized. In more detail, given a bundle of side-rear view scenes, the minimum number of frames that have become FREE in the BLOCKED state can be counted, and the frame count can be converted into seconds according to the number of frames per second of the bundle. When allocating a new label, ±1 frame can be tolerated to mitigate quantization errors.

1 is a view showing an example of annotated images according to an embodiment of the present invention. The images in the first row in FIG. 1 show the order in which the own-vehicle overtakes another vehicle in the target lane. The images in the second row show the reverse order. Also, the images in the dataset of FIG. 1 are obtained from various places including residential areas (images in the first row) and highways (images in the second row).

Table 1 below shows an example of overall statistics of a data set converted according to an embodiment.

Existing label Converted label Proportions FREE FREE 55.41% BLOCKED FREE in 1s 1.17% FREE in 2s 1.04% FREE in 3s 0.47% BLOCKED 41.91%

Table 1 shows an example of data distribution when a FREE label and a BLOCKED label are allocated to side-rear view scenes, and a FREE label, a FREE in 1s label, and a FREE label to side-rear view scenes according to an embodiment of the present invention. An example of data distribution in the case of allocating in 2s label, FREE in 3s label, and BLOCKED label is shown. For example, the images in FIG. 1 show examples in which these transformed labels are assigned. In relation to lane change, there is a previously constructed data set including a total of 109,416 images. However, this data set has two problems in order to become suitable training data for deep learning. First, images within the same class show a large variation. For example, the fourth column of FIG. 1 is two images within the same 1s class (label), where the vehicle positions on the target lane are very different. This is because the images in this dataset are obtained from different types of roads (city roads vs. highways), and the self-vehicle deals with various road structures and lane-change scenarios such as passing other vehicles and vice versa. do. Second, the relabeling results show severe imbalance across the five classes as shown in Table 1. In Table 1, the 1s, 2s, and 3s classes occupy only 3% of the dataset, while the FREE and BLOCKED classes are overwhelmingly numerous in the overall dataset.

4. Methodology

Hereinafter, a new image feature extraction method that focuses on a primary target in the side-rear space will be described. Then, how the classification by fractional learning enables sub-best-change decisions within the system according to embodiments of the present invention.

4.1 Learning Features via Weak Supervision

Pre-learned features have been widely used in various image recognition tasks. In embodiments of the present invention, for more reliable results for a task, it is possible to learn new image features dealing with objects that can move on the road. Using an attention mechanism can improve perception of image features associated with fast-moving objects while suppressing the effects of features not related to the task. In order to obtain these task-specific features, weak supervised learning can be used with an attention mechanism using a saliency map of a specific class. In addition, there is a need to understand the inside of a convolutional neural network (CNN) so that the proposed system can achieve safety and reliability as a vehicle application.

4.1.1 Class Activation Mapping

Most of the CNN architecture for image classification consists of convolutional layers for feature extraction and subsequent linear layers for classification tasks. In the process of reshaping the output of convolutional layers, it may be required to read spatial information of image features. The main idea of the Class Activation Mapping (CAM) algorithm is that each activation unit encodes a separate image feature, and thus activation units bring different importance to predicting a class. In particular, the CAM performs global average pooling on the last convolutional layer and can calculate adaptive weights of feature maps across activation units.

를 내재적으로(implicitly) 학습하여 클래스가 지정된 중요도 맵

을 얻을 수 있다.Equationally, while learning, the CAM algorithm is the weight for each class as shown in Equation 1 below.

Class-assigned importance map by implicitly learning

Can be obtained.

는 예를 들어, 소프트맥스(softmax)와 같은 활성화 함수(activation function)를 나타낼 수 있고,

는 마지막 합성곱 계층에서 k 번째 활성화 유닛을 나타낼 수 있다. 달리 말하면, 클래스가 지정된 가중치

와 특징 맵

의 선형 조합은 분류를 위해 지역적으로 의미 있는 영역을 강조하는 픽셀 단위의 중요도 맵

가 될 수 있다. 그러므로,

는 클래스

가 될 가능성이므로 CAM은 클래스의 예상을 위해 이 수치를 이용할 수 있다.here

May represent, for example, an activation function such as softmax,

May represent the k- th activation unit in the last convolutional layer. In other words, the weight to which the class is assigned

And feature map

The linear combination of is a pixel-level importance map that highlights regions of regional significance for classification

Can be. therefore,

The class

Is likely to be, so CAM can use this number for class predictions.

4.1.2 Pixelwise Regularization

In an embodiment of the present invention, an importance map for vehicle-related classes may be used as an attention mask that focuses on a movable object on the road. Because CAM uses cross entropy loss to make predictions that maximize the overall attention associated with the class, it tends to highlight only the most discriminating parts of the object. In order to address this problem, in the present embodiment, it is possible to apply pixel-by-pixel normalization to the CAM and minimize the effect of local characteristics common to all classes. As a result, for each class, the normalization method according to the present embodiment can suppress unrelated areas and highlight areas corresponding to the class.

의 집합으로 중요도 맵

의 집합에 대한 평균적인 이진 크로스 엔트로피 손실을 계산할 수 있다. 여기서

는 클래스의 총 개수를 나타낸다. 다른 정보(예를 들어, 이진 분할 맵(binary segmentation map))가 이용 가능하다면, 클래스들과 관련 있는 모양 정보(shape information)를 만들기 위하여 픽셀 단위의 실측 자료 맵(정답 맵)이 정의될 수 있다. 그러나, 본 실시예에서는 확장된 실측 자료를 이진 행렬로 간단히 설계할 수 있다. 예를 들어, 모든

에 대하여

이고, 여기서

와

는 각각 영상의 폭(width)과 높이(height)를 나타낼 수 있다. 이때, 픽셀 단위의 회귀 손실(pixelwise regression loss)은 다음 수학식 2와 같이 정의될 수 있다.In this embodiment, the expanded actual data (correct answer)

Importance map as a set of

We can calculate the average binary cross entropy loss for a set of. here

Represents the total number of classes. If other information (e.g., binary segmentation map) is available, a pixel-by-pixel actual data map (correct answer map) can be defined to create shape information related to classes. . However, in this embodiment, it is possible to simply design the extended measured data as a binary matrix. For example, all

about

Is, where

Wow

May represent the width and height of the image, respectively. In this case, the pixelwise regression loss may be defined as in Equation 2 below.

Finally, the attention model can learn a linear combination of losses as shown in Equation 3 below:

는 분류를 위한 크로스 엔트로피 손실일 수 있으며,

는 초매개변수(hyperparameter)일 수 있다. 이진 분류기로서 주의 엔진(attention engine)을 학습시키면서, 차량-유사 객체(양성 샘플) 및 그 외의 것(음성 샘플)과 관련된 서브세트가 활용될 수 있다. 데이터 샘플링에 대해서는 이후 더욱 자세히 설명한다.here,

May be the cross entropy loss for classification,

May be a hyperparameter. While training the attention engine as a binary classifier, a subset related to vehicle-like objects (positive samples) and others (negative samples) can be utilized. Data sampling will be described in more detail later.

4.1.3 Multiscale Aggregation

Since most of the datasets to be used are composed of object-oriented images, it is difficult to learn size-invariant features from them. However, moving objects often appear in various sizes, especially in side-to-rear view road scenes. Therefore, a multi-scale integration method capable of obtaining an importance map regardless of the object size can be used. The proposed integration method can use the model learned by Equation 3 again, and can operate in the inference stage. Therefore, no additional learning is required.

을 생성할 수 있다. 이 이후부터, 명료함을 위하여

의 표기법을

으로 대체한다.

을 영상의 사이즈를 앵커 영상의 사이즈로 바꾸게 하는 변환(예를 들어, 이중 선형 보간법(bilinear interpoliation))이라고 하자. 이때 다중 스케일의 주의 맵

을 얻기 위하여 다음 수학식 4와 같이 모든 스케일에 대해 중요도 맵들이 적응적으로 통합될 수 있다.The system may generate image pyramids having various sizes for one input image. In this case, the initial input image may be referred to as an "anchor image" afterwards to distinguish it from a resized image. The system can obtain multi-scale importance maps by repeatedly inputting the resized set of images into the model. In order to accept various sizes for the input, a minimal transformation to the model can be applied by adjusting the size of the global average pooling operation at the top of the last convolutional layer. m For the first pyramid input image, the attention engine represents the importance map for the car class.

Can be created. From this onwards, for clarity

The notation of

Replaced with

Let be a transformation (for example, bilinear interpoliation) that changes the size of the image to the size of the anchor image. At this point, the multi-scale attention map

The importance maps may be adaptively integrated for all scales as shown in Equation 4 below.

은 m 번째 크기 변경된(scaled) 입력에 대한 예상 수치를 나타내고,

은 다양한 크기 변경된 입력들에 대한 예상 수치들 중에서 소프트맥스(softmax)가 계산한 각 크기의 상대적인 중요도를 결정할 수 있다.here,

Represents the expected number for the m th scaled input,

May determine the relative importance of each size calculated by softmax among predicted values for various size-changed inputs.

2 shows an example of comparing the attention result of CAM with the attention result of the method according to an embodiment of the present invention. In FIG. 2, pixelwise regularization (PR) and multiscale aggregation (MA) have been described above, and indicate that the proposed method can improve the recognition of the entire vehicle shape in an image. As described above, the pixel-by-pixel normalization and multi-scale integration approach according to the present embodiment can extend the limited cognitive domain of the CNN without an additional learning procedure.

4.2 Few-Shot Classification for Lane-Change

To overcome the serious data imbalance problem, prime number learning can be applied to the dataset. In the sense that humans can adapt behavior patterns to various driving scenarios based on their experience, the system can adopt a prototype network, which is a non-parametric algorithm.

4.2.1 Prototype Network

에 대해

개의 레이블링된(labeled) 예시들을 포함하는 지원 세트(support set)인

를 갖는다고 가정하자. 여기서,

은 데이터 지점(data point)이고,

는 대응하는 정답 레이블(ground truth label)을 나타낼 수 있다. 프로토타입 네트워크는 각 클래스에 대해 프로토타입(prototype)인

를 표현하기 위하여 임베딩 함수(embedding function)

를 학습할 수 있다. 여기서

는 학습 변수(learnable parameters)를 의미할 수 있다. 임베딩 공간(embedding space) 내 거리 단위

로, 프로토타입 네트워크는 소프트맥스(softmax)를 사용하여 클래스들 상의 분포를 도출할 수 있으며, 쿼리

에 대한 레이블을 다음 수학식 5와 같이 예상할 수 있다.Here, we briefly review the prototype network for completeness. Set class

About

A support set containing four labeled examples.

Suppose you have here,

Is the data point,

May represent a corresponding ground truth label. The prototype network is a prototype for each class.

To express the embedding function (embedding function)

You can learn. here

Can mean learnable parameters. Unit of distance in embedding space

As a result, the prototype network can derive the distribution over the classes using softmax, and query

The label for can be expected as shown in Equation 5 below.

4.2.2 Feature Embedding for Lane-Change

In this embodiment, it is assumed that a place where pre-computed image features are used is a fixed feature space. In order to consider the temporal property on the scene, image features calculated at 1-second intervals may be linked, and the annotation of the linked feature may be copied from the annotation of the last time slot. At this time, image features may be integrated with attention. This may be fuse after concatenation (FAC) or concatenate after fusion (CAF). Using FAC, attention results are applied globally to the extended feature map, while CAF can link individual attention regions of each feature map. 3 is a diagram showing an example of an integration method according to an embodiment of the present invention. In detail, FIG. 3 shows a comparison of different spatiotemporal integration methods of image features and attention maps. The left side of FIG. 3 shows an example using the FAC method, and the right side of FIG. 3 shows an example using the CAF method. Image features and attention masks can be combined by multiplication or addition. Table 2 shows an example of comparing performance for various integration scenarios.

Fusion Op. 1-shot(5-way) 5-shot(5-way) FAC Mul 56.00±0.78% 75.95±0.70% CAF 56.70±0.81% 81.04±0.67% FAC Add 68.76±0.80% 84.88±0.64% CAF 74.09±0.83% 86.48±0.65%

Table 2 shows the performance of the VGG-16 embedded in terms of the integration method.

Euclidean distance metric may be applied to the embedding space. Feature embedding networks consist of 3 convolutional building blocks, each block being a convolutional layer (with a 3Х3 kernel size), batch normalization, ReLU (Rectified Linear Unit), and max pooling (with a 2Х2 kernel size). ) Can be formed as a stack. If the dimensions of the input features depend on the backbone model, the dimensions of the hidden units in the embedding network can be fixed at 512 and 256, respectively.

4 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Systems according to the embodiments of the present invention described above may be implemented by the computer device 400 shown through FIG. 4, and the method according to the embodiments of the present invention may be performed by the computer device 400. I can.

In this case, as shown in FIG. 4, the computer device 400 may include a memory 410, a processor 420, a communication interface 430, and an input/output interface 440. The memory 410 is a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), read only memory (ROM), and a disk drive. Here, a non-destructive large-capacity recording device such as a ROM and a disk drive may be included in the computer device 400 as a separate permanent storage device separate from the memory 410. In addition, an operating system and at least one program code may be stored in the memory 410. These software components may be loaded into the memory 410 from a computer-readable recording medium separate from the memory 410. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, and memory card. In another embodiment, software components may be loaded into the memory 410 through a communication interface 430 rather than a computer-readable recording medium. For example, software components may be loaded into the memory 410 of the computer device 400 based on a computer program installed by files received through the network 460.

The processor 420 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 420 by the memory 410 or the communication interface 430. For example, the processor 420 may be configured to execute a command received according to a program code stored in a recording device such as the memory 410.

The communication interface 430 may provide a function for the computer device 400 to communicate with other devices through the network 460. For example, a request, command, data, file, etc., generated by the processor 420 of the computer device 400 according to a program code stored in a recording device such as the memory 410, is transmitted to the network according to the control of the communication interface 430. 460) can be transferred to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 400 through the communication interface 430 of the computer device 400 via the network 460. Signals, commands, data, etc. received through the communication interface 430 may be transmitted to the processor 420 or the memory 410, and the file may be a storage medium (described above) that the computer device 400 may further include. Permanent storage).

The input/output interface 440 may be a means for an interface with the input/output device 450. For example, the input device may include a device such as a microphone, keyboard, camera, or mouse, and the output device may include a device such as a display and a speaker. As another example, the input/output interface 440 may be a means for interfacing with a device in which input and output functions are integrated into one, such as a touch screen. The input/output device 450 may be configured with the computer device 400 and one device.

Further, in other embodiments, the computer device 400 may include fewer or more components than the components of FIG. 4. However, there is no need to clearly show most of the prior art components. For example, the computer device 400 may be implemented to include at least a portion of the input/output device 450 described above, or may further include other components such as a transceiver and a database.

The communication method is not limited, and not only a communication method utilizing a communication network (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network) that the network 460 may include, but also short-range wired/wireless communication between devices may be included. have. For example, the network 460 is a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , Internet, and the like. In addition, the network 460 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

5 is a flowchart illustrating an example of a method of learning when to change lanes according to an embodiment of the present invention. The method for learning a lane change time point according to the present embodiment may be performed by the computer device 400 for implementing a lane change time point learning system. For example, the processor 420 of the computer device 400 may be implemented to execute a code of an operating system included in the memory 410 or a control instruction according to the code of at least one program. Here, the processor 420 may cause the computer device 400 to perform the steps 510 to 550 included in the method of FIG. 5 according to a control command provided by a code stored in the computer device 400. Can be controlled.

In operation 510, the computer device 400 may receive an image in relation to the side-rear region of the vehicle through the camera and to which the first label or the second label is assigned to each frame. Here, the image to which the first label is assigned is an image representing a state in which lane change is blocked (lane change is not possible), and the image to which the second label is assigned represents a state in which lane change is free (lane changeable state). Each may include an image. As an example, the first label may refer to the aforementioned label “BLOCKED” and the second label may refer to the aforementioned label “FREE”.

In step 520, the computer device 400 corresponds to the number of frames per second (FPS) on the basis of the transition point of transition from the blocked state to the free state. n is a natural number) seconds later, a label can be assigned to define that lane changes are possible. For example, in step 520, the computer device 400 may assign a label indicating that lane change is possible after n seconds to the m-th frame before the switching point. In this case, m may be determined by a multiplication operation having n and the number of frames per second as parameters. As a more specific example, it is assumed that the index of a frame corresponding to a transition point at which the FPS is fixed to 10 and the transition point from the blocked state to the free state is 110. In this case, a label defining that lane change is possible after 1 second may be given to a frame of index 100, which is a 10 (1 (second) Х 10 (FPS))-th frame before the switching point. Similarly, a label defining that lane change is possible after 2 seconds may be given to a frame of index 90, which is the 20th (2 (second) Х 10 (FPS))-th frame before the switching point.

In operation 530, the computer device 400 may learn a lane change time point based on frames of the image and labels included in the frames. Depending on the embodiment, the computer device 400 performs weakly supervised learning for each frame of an image based on whether a vehicle or a moving object exists, and thus the vehicle or moving object occupies each frame. The area can be output in the form of a heat map. In this case, the computer device 400 may learn a lane change time point by further using the heat map and the feature vector of the image in step 430. In this case, the computer device 400 may increase a weight for an area occupied by a vehicle or a moving object in the image by adding the heat map value to all dimensions of the feature vector. In addition, as shown in FIG. 1, since images assigned the same label may have various situations, the features of images corresponding to a preset time interval (for example, 1 second) are connected with a single vector. Features can be encoded. For example, different situations shown in FIG. 1 can be distinguished according to a case in which an area occupied by a vehicle or a moving object in an image increases and decreases as time passes.

In addition, the computer device 400 may utilize multi-scale integration to obtain an importance map for objects that appear in different sizes in the image regardless of the size of the object. For example, the computer device 400 generates a set of images whose size has been adjusted for an image, and repeatedly inputs the set of images whose size has been adjusted as an attention model to provide multi-scale importance maps (saliency maps). maps), it is possible to obtain an importance map for objects that appear in different sizes in the image regardless of the size of the object by integrating the importance maps for all scales.

6 is a diagram illustrating an example of assigning a label according to an embodiment of the present invention. 6 shows a total of five frames 610 to 650, one for every 10 frames among frames included in an image. In this case, the first frame 610 of FIG. 6 represents a frame at a time point at which the block state is changed to the free state. The first frame 610 may be in a state in which a label'FREE', which is a label indicating a free state, is attached. Frames prior to the first frame 610 may be frames to which a label'BLOCKED', which is a label indicating a block state, is assigned. In this case, the second frame 620, the third frame 630, the fourth frame 640, and the fifth frame 650 shown in FIG. 6 may also be included in frames to which the label'BLOCKED' is assigned.

Assuming that the number of frames per second is 10, as previously described, the computer device 400 may assign a label defining that lane change is possible after n seconds to the m-th frame preceding from the first frame 610. Here, m may be determined through a multiplication operation between the number of frames per second and n.

For example, in FIG. 6, the second frame 620 represents a 10th frame prior to the first frame 610. In this case, a label defining that lane change is possible after 1 second may be given to the second frame 620, which is the 10th frame (the number of frames per second is 10 × 1 second) from the first frame 610.

Similarly, the third frame 630 represents the twentieth frame preceding from the first frame 610. In this case, a label defining that lane change is possible after 2 seconds may be given to a third frame 630 that is a previous 20th frame (the number of frames per second is 10 × 2 seconds) from the first frame 610.

It is easy to understand that the fourth frame 640 may represent the 30th frame preceding from the first frame 610, and a label defining that lane change is possible after 3 seconds can be given to the fourth frame 640. You can understand.

Tables 1 and 6 describe an example in which n is a natural number less than or equal to 3, but it will be easily understood that n can be set in more various ways. If n is a natural number less than or equal to 4, a label defining that lane change is possible after 4 seconds will be assigned to the fifth frame 650 of FIG. 6.

The label defining that lane change is possible after n seconds may be additionally assigned to the existing label'BLOCKED', but may be given by replacing the existing label'BLOCKED'.

Hereinafter, a method and system for providing notification of whether a lane change is possible will be described.

7 is a flowchart illustrating an example of a method for providing a lane change notification according to an embodiment of the present invention. The method of providing a lane change notification according to the present embodiment may be performed by the computer device 400 for implementing a lane change notification providing system. For example, the processor 420 of the computer device 400 may be implemented to execute a code of an operating system included in the memory 410 or a control instruction according to the code of at least one program. Here, the processor 420 may cause the computer device 400 to perform the steps 710 to 720 included in the method of FIG. 7 according to a control command provided by a code stored in the computer device 400. Can be controlled.

In step 710, when the request for lane change is recognized, the computer device 400 analyzes the image input through the camera in relation to the side-rear region of the vehicle and determines the current state related to the lane change as whether the lane change is possible. have. This determination may be made through the artificial intelligence model learned through FIG. 5 above. As an example, the AI model can divide the current state into a block state in which lane change is impossible and a free state in which lane change is possible. In this case, the block state may be subdivided according to the unit time, and may include a state in which lane change is possible after n (n is a natural number) seconds. For example, the block state may be classified into a 3seconds left state in which lane change is possible after 3 seconds, a 2seconds left state in which lane change is possible after 2 seconds, and a 1second left state in which lane change is possible after 1 second. In other words, the state related to lane change can consist of two or more depths, the upper level is divided into a block state and a free state, and the lower level is a state in which lane change is possible after n (n is a natural number) seconds, for example, 3 seconds. It can be classified into left state, 2seconds left state, and 1second left state.

The above-described step 710 is a step of acquiring an image input through a camera in relation to the side-rear region of the vehicle, and a block state in which lane change is impossible, and a free state in which lane change is possible. , And n (where n is a natural number) seconds later, the predicting step may be divided into a state in which lane change is possible, and the predicting step may be to use the model learned through FIG. 5.

In step 720, the computer device 400 may provide differentiated notifications in different ways according to the current state. The computer device 400 may provide different notifications for a current state, a block state in which lane change is impossible, a free state in which lane change is possible, and a state in which lane change is possible after n (where n is a natural number) seconds. . In the Block state, information on that lane change is impossible, and in the Free state, information on that lane change is possible may be provided. In addition, information on the remaining time until the lane change every second from n seconds to 1 second may be provided for a lower level in which the block state is subdivided (a state in which lane change is possible after n seconds). As a more specific example, when n is 3, the computer device 400 connected to the display device installed in the vehicle provides information on the state in which lane change is possible after n seconds, such as 3 seconds, 2 seconds, and 1 second, the remaining time until the lane change The display device may provide information on. In addition to providing a visual notification through a display device, providing an audible notification (sound notification) using a speaker may be considered. Meanwhile, in addition to providing the notification to the driver of the vehicle, it may be considered that the notification is provided to an artificial intelligence that controls autonomous driving of the vehicle.

The computer device 400 may provide a notification on whether a lane change is possible only for a certain period of time, and at this time, a starting point of a certain time may be defined as a time point of a lane change request, and an end point of a certain time may be defined as a time when lane change is completed. . In other words, the computer device 400 may provide a notification from the time when the lane change request is recognized to the time when the lane change completion is recognized. The computer device 400 is used when the driver of the vehicle turns on a direction indicator (a left or right blinker) for lane change on a multi-lane road, and when a lane change guide is output during route guidance in connection with a navigation system in use by the driver. The point in time can be recognized as the point of request for lane change. In addition, the computer device 400 determines that the lane has changed as a result of positioning using location coordinates (eg, GPS coordinate values) when the turn indicator that has been turned on is turned off, and when it is determined that the lane has changed through ADAS sensing. The time point can be recognized as the time when the lane change is completed. As a method of determining when a lane change request is made and when a lane change is completed, one of the above-described methods or a combination of a plurality of methods may be applied. In addition to the above method, a combination with other well-known technologies is also possible.

The computer device 400 is in a block state in which lane change is impossible, a free state in which lane change is free, and a state in which lane change is possible after n (where n is a natural number) seconds (eg, 3seconds left state, 2seconds left state, 1second). left state) can provide different notifications. For example, the computer device 400 may provide a differentiated notification according to a depth of information on whether a lane change is possible. In this case, the computer device 400 may be connected to a plurality of display devices installed in the vehicle, and different display devices may output notifications for the upper level and the lower level.

8 is a diagram illustrating an example of a display device that outputs a lane change notification according to an embodiment of the present invention.

8 shows display devices 810 to 830 installed in a vehicle, and these display devices 810 to 830 can be used as a medium for outputting a lane change notification. For example, the side mirrors 810 on both sides of the vehicle may be used as a first display device, and are installed on the inner side of the front display device 820 and the side mirror 810 such as a head up display (HUD) or a navigation terminal. At least one of the separate display elements 830 such as LEDs may be used as a second display device. At this time, the HUD can be applied to either a combiner type or a type using a windshield as a reflective surface without a separate combiner.

Hereinafter, the HUD and the LED will be described as representative examples of the second display device (the HUD is denoted by 820 and the LED is denoted by 830).

For example, the side mirror 810 corresponding to the first display device outputs information on the upper level, and the HUD 820 and the LED 830 corresponding to the second display device output information on the lower level. . In addition, the side mirror 810 may output information in a form that is more emphasized when in a block state, and the HUD 820 may output information in a form that is more emphasized in a free state. For example, information on the block state may be displayed in red, and information on the free state may be displayed in green. In the case of the LED 830, like the side mirror 810, it may be composed of a red LED as a means for emphasizing the block state, and at this time, the block state is divided into a number proportional to the number of sub-levels (eg, 3) Can be configured.

9 to 13 are diagrams showing an example of visualization of a lane change notification according to an embodiment of the present invention. In the examples of FIGS. 9 to 13, the information on whether the lane can be changed is divided into two depths, and the upper level is divided into a block state (10) and a free state (20), and the lower level subdividing the block state (10) is It is divided into 3seconds left state (11), 2seconds left state (12), and 1second left state (13).

Referring to FIG. 9, when in the Block state 10, the computer device 400 may display a Block icon 901 indicating the Block state 10 on the side mirror 810, and three LEDs ( 830) can be turned on.

Referring to FIG. 10, in the 3seconds left state (11), the computer device 400 maintains the lighting state of the Block icon 901 and the three LEDs 830 displayed on the side mirror 810, and the HUD ( Information on the remaining time until the lane change is possible, that is, information 1003 on the state in which the lane change is possible after 3 seconds may be displayed on the 820.

Referring to FIG. 11, in the 2seconds left state 12, the computer device 400 may light up two LEDs 830 while maintaining the Block icon 901 displayed on the side mirror 810, and In addition, on the HUD 820, information on the remaining time until the lane change is possible, that is, information 1105 on the state in which the lane change is possible after 2 seconds may be displayed.

Referring to FIG. 12, in the 1 second left state 13, the computer device 400 may turn on one LED 830 while maintaining the Block icon 901 displayed on the side mirror 810. In addition, on the HUD 820, information on the remaining time until the lane change is possible, that is, information 1207 on the state in which the lane change is possible after 1 second may be displayed.

Referring to FIG. 13, when in the Free state 20, the computer device 400 may display a Free icon 1309 indicating the Free state 20 on the side mirror 810, and three LEDs ( At the same time as turning off all the lights 830, information 1311 about a state in which lane change is free may be displayed on the HUD 820.

Accordingly, the computer device 400 provides a differentiated notification according to the depth of information on whether a lane change is possible, and the information on the upper level (block state and free state) is the side mirror 810 which is the first display device. The information on the lower level (3seconds left state, 2seconds left state, 1second left state) which is output to and subdivided the block state may be output through the HUD 820 and the LED 830 which are the second display devices. The computer device 400 visualizes time information until the lane change is possible as information on a lower level through the HUD 820 and the LED 830 to support the driver of the vehicle to prepare for the lane change in advance.

10 to 13 show examples of lighting 3, 2, and 1 LED 830 for each of the 3seconds left, 2seconds left, and 1second left states, but the number of LEDs 830 lighting in each state is different. It can also be set. For example, in the block state, three LEDs 830 are turned on, and two, one, and zero LEDs 830 may be set to light for each of the 3seconds left, 2seconds left, and 1second left states.

In the above, the use of the side mirror 810 as the first display device and the HUD 820 and the LED 830 as the second display device are described, but are not limited thereto, and the first display device and the second display The usage example of the device can be changed anytime. As an example, the LED 830 may be omitted and the side mirror 810 and the HUD 820 may be used as a medium for outputting a lane change notification. Referring to FIG. 14, side mirrors 810 on both sides of a vehicle may be used as a first display device and a HUD 820 may be used as a second display device. Similar to the method described above, the information 901 and 1309 on the upper level (block state and free state) is output to the side mirror 810 as the first display device, and the lower level (3seconds left state) is subdivided into the block state. , 2seconds left state, 1second left state) information (1003, 1105, 1207, 1311) may be output through the HUD 820, which is a second display device.

In addition, the computer device 400 may provide a lane change notification through sound in order to supplement information on whether a lane change is possible. The sound is used for the purpose of reinforcing information on the upper level (Block state and Free state). For example, the computer device 400 is information about the upper level and has different pitches for the Block state and Free state. Sound notification can be output.

Referring to FIG. 15, the computer device 400 outputs a sound notification of a low pitch when in the Block state 10, while a pitch higher than the Block state 10 in the Free state 20 ( higher pitch) sound notification can be output. For example, in the Block state (10), a low-pitched sound is repeatedly output at a certain interval (e.g., 0.5 seconds), and in the Free state (20), a continuous high-pitched sound is maintained while the Free state (20) is maintained. Sound can be output. As another example, in the free state 20, in addition to the continuous output form, a one-time output form or a sound having a different interval from the block state 10 may be output.

In addition to varying the pitch or interval of the sound for the Block and Free states, the basic elements of the sound, such as loudness, pitch, and tone timbre, or melody depending on the change or repetition of the sound. It is also possible to differentiate notifications by different design variables such as musical elements such as, harmony, and rhythm.

According to an embodiment, the computer device 400 may activate a lane change preparation function as a notification about a state in which a lane change is possible is provided after n seconds. In this case, the lane change preparation function may include a preparation function such as calculating detailed parameters for lane change or calculating a route.

In addition, the computer device 400 may store information on the driving tendency of the driver of the vehicle in step 720 and may set the intensity of the notification based on the driving tendency.

The computer apparatus 400 may determine the intensity of the notification based on the number of frames per label for frames to which a label defining that lane change is possible after n seconds is assigned. For example, if the number of frames assigned with a 3-second label is 10 while the number of frames assigned with a 2-second label is 1 or 0 based on 10 FPS, the presence of a vehicle or object accelerating rapidly in the lane to be changed is indicated. It can mean. In this case, the computer device 400 may provide a relatively stronger notification or provide a signal for driving a safety device preventing lane change.

Meanwhile, in the above, the embodiments of the present invention have been described focusing on an example in which the block state can be subdivided according to the unit time and the unit time is in seconds, but the embodiment of the present invention is not limited thereto. For example, a state related to a lane change according to an embodiment is divided into a state in which lane change is impossible (Block) and a lane change is possible (Free), and a state in which lane change is impossible is subdivided into n (n May be a natural number) including a lower level divided into a state in which lane change is possible after a unit time. For example, a state related to lane change is a state in which lane change is impossible, lane change is possible, lane change is possible after 0.1 seconds, lane change is possible after 0.5 seconds, lane change is possible after 1 second, and after 3 seconds. It may include a state in which lane change is possible. As such, in the case of states indicating that lane change is possible after a certain time, the interval between such specific times may not only be constant, but also may not be constant, and such an interval may be in seconds as well as in seconds. It may be a unit time other than that.

As described above, according to embodiments of the present invention, a differentiated notification may be provided using different media according to the depth of information on whether or not a lane change is possible, and it is possible to determine whether a lane change is possible using sounds of different pitches. You can effectively supplement information about.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable gate array (PLU). It may be implemented using one or more general purpose computers or special purpose computers, such as a logic unit), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be embodyed in any type of machine, component, physical device, computer storage medium or device to be interpreted by the processing device or to provide instructions or data to the processing device. have. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. In this case, the medium may be one that continuously stores a program executable by a computer, or temporarily stores a program for execution or download. In addition, the medium may be a variety of recording means or storage means in a form in which a single or several pieces of hardware are combined, but is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks, and And ROM, RAM, flash memory, and the like may be configured to store program instructions. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or a storage medium managed by a server.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (11)

In the lane change notification providing method executed in a computer device,
The computer device includes at least one processor configured to execute computer readable instructions contained in a memory,
The lane change notification providing method,
Determining, by the at least one processor, a state related to a lane change by analyzing an image input through a camera in relation to a side-rear region of the vehicle; And
Providing, by the at least one processor, a notification using different media according to a depth of information on whether a lane change is possible as a state related to the lane change
Lane change notification providing method comprising a. The method of claim 1,
The state related to the lane change is divided into a state in which lane change is impossible (Block) and a state in which lane change is possible (Free), and the state in which the lane change is impossible is subdivided, and after n (where n is a natural number) unit time Including sub-levels separated by lane changeable states
Lane change notification providing method, characterized in that. The method of claim 1,
The providing step,
Outputting information on a higher level through a first display device installed in the vehicle,
Outputting information on a lower level through a second display device installed in the vehicle
Lane change notification providing method, characterized in that. The method of claim 1,
The providing step,
Using a first display device installed in the vehicle as a means for emphasizing a state in which lane change is impossible,
Using a second display device installed in the vehicle as a means for highlighting the remaining time until lane change is possible
Lane change notification providing method, characterized in that. The method of claim 1,
The providing step,
Providing the notification for a certain period of time from the time when the lane change request is recognized to the time when the lane change completion is recognized
Lane change notification providing method, characterized in that. The method of claim 5,
Recognizing at least one of a case where the turn indicator of the vehicle is turned on and a case where a lane change guidance is output during navigation route guidance as the lane change request
Lane change notification providing method, characterized in that. The method of claim 5,
At least one of the case where the turn indicator of the vehicle is turned off, when it is determined that the lane has changed as a result of positioning for location coordinates, and when it is determined that the lane has changed through ADAS (Advanced Driving Assistance System) sensing What you perceive as perfect
Lane change notification providing method, characterized in that. The method of claim 1,
The providing step,
Outputting information on whether the lane change is possible through sound
Lane change notification providing method comprising a. The method of claim 8,
The outputting step,
Outputting sounds of different pitches according to a state in which lane change is impossible (block) and lane change is possible (free)
Lane change notification providing method, characterized in that. A computer program stored in a non-transitory computer-readable recording medium for executing the method of providing a lane change notification according to any one of claims 1 to 9 on the computer device. In the computer device,
At least one processor configured to execute computer readable instructions contained in memory
Including,
The at least one processor,
By analyzing the image input through the camera in relation to the side-rear area of the vehicle, it determines the state related to the lane change,
Providing a notification using different media according to the depth of information on whether the lane change is possible as a state related to the lane change
Computer device, characterized in that.

Images