KR102582056B1 - Surround view providing device and operating method of the same - Google Patents

Abstract

A surround view providing device installed in a vehicle to provide a surround view image of the vehicle is disclosed. The apparatus includes a data receiving unit configured to receive driving data related to driving of the vehicle from the vehicle, generating a driving feature vector for the vehicle from the received driving data, and generating a driving feature vector of the vehicle, each of which includes a plurality of driving feature vectors. A driving pattern determination unit configured to determine the driving pattern of the vehicle by classifying it into one of a plurality of clusters including, determining reference vehicles having the same driving pattern as the driving pattern of the vehicle, and converting parameters of the stored reference vehicle. It includes a parameter determination unit configured to determine conversion parameters for the vehicle using the conversion parameters for the vehicle and a surround view image providing unit configured to generate a surround view image using conversion parameters for the vehicle and camera images generated by a camera of the vehicle. .

Description

Surround view providing device and operating method thereof {SURROUND VIEW PROVIDING DEVICE AND OPERATING METHOD OF THE SAME}

The present invention relates to a surround view providing device and a method of operating the same.

With the development of vehicle imaging technology, multiple cameras are being mounted on vehicles. The Surround View Monitor uses multiple cameras mounted on the vehicle to generate images around the vehicle. At this time, in order to generate images around the vehicle that are similar to reality, calibration between images generated by each of a plurality of cameras mounted on the vehicle is required.

In the absence of such matching, errors may occur in the final surround view image due to deviations between images generated from cameras. Therefore, when the surround view system is first installed, a matching procedure is performed to match the cameras. However, as the vehicle drives, the positions of the cameras change little by little, causing a problem of misalignment.

The purpose of the present invention to solve the above problems is to provide a surround view providing device and a method of operating the same.

One aspect of the present invention for achieving the above object provides a surround view providing device and a method of operating the same.

A surround view providing device installed in a vehicle according to embodiments of the present invention to provide a surround view image for the vehicle includes a data receiver configured to receive driving data related to the driving of the vehicle from the vehicle, and a data receiving unit configured to receive driving data related to the driving of the vehicle from the vehicle. Driving pattern determination configured to determine the driving pattern of the vehicle by generating a driving feature vector for the vehicle and classifying the driving feature vector of the vehicle into one of a plurality of clusters, each of which includes a plurality of driving feature vectors. A parameter determination unit configured to determine reference vehicles having the same driving pattern as the driving pattern of the vehicle and determine conversion parameters for the vehicle using stored conversion parameters of the reference vehicle, conversion parameters for the vehicle, and a camera for the vehicle. and a surround view image providing unit configured to generate a surround view image using camera images generated by.

A method of operating a surround view image providing device (i.e., a method of providing a surround view image) according to embodiments of the present invention is implemented as a program consisting of a plurality of instructions, stored in a computer-readable non-transitory medium, and executed by a computer. You can.

According to embodiments of the present invention, conversion parameters for camera images of the vehicle in the current state are determined using driving data related to the vehicle's driving, and accurate surround view images are provided to the user through the determined conversion parameters. There is a possible effect.

Figure 1 shows a surround view image using a plurality of cameras mounted on a vehicle.
Figure 2 shows a surround view providing system including a surround view providing device according to an embodiment.
Figure 3 shows a surround view providing device according to embodiments of the present invention.
FIG. 4 is a diagram illustrating a process for determining conversion parameters for a vehicle according to a clustering engine according to embodiments of the present invention.
Figure 5 is a diagram for explaining a method of estimating transformation parameters according to embodiments of the present invention.
Figure 6 is a diagram for explaining calculating rotation parameters of a vehicle according to embodiments of the present invention.
Figure 7 is a diagram illustrating the hardware configuration of a surround view providing device according to embodiments of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

Figure 1 shows a surround view image using a plurality of cameras mounted on a vehicle. Referring to Figure 1, a vehicle 1 is parked within a parking line. A plurality of cameras may be mounted on the vehicle 1, and each of the plurality of cameras may capture images corresponding to its viewing angle at its installed location. By combining images generated by a plurality of cameras, a surround view image 3 representing the surroundings of the vehicle 1 can be generated. For example, the surround view image 3 may be generated by images captured from a total of four cameras of the vehicle 1: a front camera, a rear camera, and two side cameras.

On the other hand, since the images generated by each of the cameras mounted on the vehicle 1 are images acquired in different coordinate systems, if the coordinate systems of each of the images do not match each other (that is, the images of the cameras, such as the optical axes of each of the cameras, etc. If there is no correction considering the parameters or the correction is not performed accurately), a misfit section (2) may occur between the images of each camera, so matching is necessary.

Figure 2 shows a surround view providing system including a surround view providing device according to an embodiment. Referring to FIG. 2 , the surround view providing system 1000 may include a vehicle 1 and a surround view providing device 100.

The vehicle 1 may be a device that travels on a road surface according to a driving force generated by a driving device such as a motor. Depending on the embodiments, the vehicle 100 may be a car, motorcycle, bus, trailer, excavator, wheel loader, excavator, crane, or truck, but the embodiments of the present invention are not limited thereto.

Vehicle 1 may include a camera 11, sensors 12, and display 13.

The camera 11 can photograph the surroundings of the vehicle 1 and generate an image. Depending on the embodiment, there may be a plurality of cameras 11, and the plurality of cameras 11 are disposed on the front, rear, left, and right sides of the vehicle 1 to generate images around the vehicle 1. You can.

The sensor 12 may measure the driving environment of the vehicle 1 and generate driving data DD related to the driving of the vehicle 1 according to the measurement results. Depending on embodiments, the sensor 12 may measure the angular acceleration of the vehicle 1 and generate driving data DD including angular acceleration information accordingly. At this time, the driving data DD may be time-series data and may be generated periodically or in real time.

For example, the sensor 12 may be a 6-axis inertial sensor, but is not limited thereto.

The display 13 may display a surround view image (SVV) provided from the surround view providing device 100. Alternatively, the surround view image SVV may be displayed directly by a display included in the surround view providing device 100 rather than by the display 13 of the vehicle 1.

The surround view providing device 100 is linked with the vehicle 1, generates a surround view image (SVV) for the vehicle 1, and provides the generated surround view image (SVV) to the driver of the vehicle 1. You can. Depending on the embodiment, the surround view providing device 100 transmits the generated surround view image (SVV) to the vehicle 1 to be displayed through the display 13, or is included in the surround view providing device 100. Surround view video (SVV) can be directly displayed and provided through a display (not shown).

The surround view providing device 100 is mounted in the vehicle 1 and is electrically connected to the vehicle 1 to exchange data.

The surround view providing device 100 may generate a surround view image for the vehicle 1 using a camera image (CV) captured from the camera 11 of the vehicle 1. According to embodiments, the surround view providing device 100 represents the front, right, left, and rear of the vehicle 1, respectively, generated by the cameras 11 installed at the front, right, left, and rear of the vehicle 1. Camera images (CV) may be received, and a surround view image (SVV) representing the surroundings of the vehicle 1 may be generated using the received camera images (CV).

According to embodiments of the present invention, the surround view providing device 100 may generate a surround view image (SVV) by combining camera images (CV) generated from the cameras 11 of the vehicle 1. For example, the surround view providing device 100 (1) converts camera images (CV) into top view images, and (2) synthesizes camera images (CV) converted into top view images according to the boundary line. This creates a surround view video (SVV).

At this time, the surround view providing device is,

(1) To convert to a top view image, the rotation angle formed by the shooting vector indicating the shooting direction of each camera 11 and the ground is determined, and rotation conversion according to the determined angle is performed.

(2) An affine transformation is performed on the top view image to determine the area to be used in the surround view image, and the top view image to which the affine transformation has been applied is synthesized to create a surround view image.

The above-described transformation parameters may include information about rotation angles in each direction (roll, pitch, and yaw) representing rotation transformation, and coefficients and constant terms of linear terms representing affine transformation.

Meanwhile, upon initial installation of the surround view providing device 100, an expert or technician can determine the installation environment and characteristics of each camera 11 of the vehicle 1, and an accurate surround view image (SVV) can be generated. there is. However, when the vehicle 1 is driving, force (or impact) in various directions may be applied to the camera 11, and accordingly, there are cases where the correct surround view image (SVV) is not generated according to the existing settings. . In particular, the shooting vector of the camera 11 may change, causing the angle between the shooting vector used during rotation conversion and the ground to differ from the initial setting. In the case of affine transformation, the degree of deviation from the initial setting is relatively small, and as a result, the influence of driving has a relatively large effect on rotation transformation.

According to embodiments of the present invention, a conversion parameter (CPM) for camera images (CV) of the vehicle 1 in the current state is determined using driving data related to the driving of the vehicle 1, and the determined conversion is performed. There is an effect of providing accurate surround view video (SVV) to the user through parameters (CPM). In particular, in this specification, a transformation parameter may mean a rotation angle.

The conversion parameter (CPM) may be inferred from data for a vehicle other than vehicle 1 (i.e., a reference vehicle). Depending on embodiments, the conversion parameter (CPM) may be determined as an output of a learning model that inputs data related to the driving of the vehicle 1 and data related to the driving of the reference vehicle.

Depending on embodiments, the surround view providing device 100 may store data necessary to determine a conversion parameter (CPM) for each camera image (CV). For example, the surround view providing device 100 may store training data necessary to infer a conversion parameter (CPM). For example, the surround view providing device 100 may include driving data DD for vehicles other than the vehicle 1 (i.e., reference vehicles), camera position data related to the positions of cameras installed in the reference vehicles, and Transformation parameters for the corresponding cameras can be held as learning data. The learning data can be updated periodically or in real time through a wireless communication network.

According to embodiments, the surround view providing device 100 determines a conversion parameter (CPM) corresponding to the current driving state of the vehicle 1 based on a predetermined learning model and provides a surround view using the determined conversion parameter (CPM). It can be provided as device 100.

A detailed description of the method for determining the conversion parameter (CPM) for vehicle 1 will be described later. Additionally, in this specification, it is assumed that the surround view providing device 100 determines or generates a conversion parameter (CPM), but depending on the embodiment, a separate cloud server 200 is installed outside the surround view providing device 100. It exists, and the conversion parameters may be determined (or created) by the cloud server 200. This is to reduce the amount of computation of the surround providing device 100. For example, the surround view providing system 10 may further include a cloud server 200, and the cloud server 200 may exchange data with the surround view providing device 100 and surround view video (SVV). ) The conversion parameter (CPM) used during generation can be determined, and information about the conversion parameter (CPM) can be transmitted to the surround view providing device 100.

Figure 3 shows a surround view providing device according to embodiments of the present invention. Referring to FIG. 3 , the surround view providing device 100 may include a data receiving unit 101, a driving pattern determining unit 103, a parameter determining unit 105, and a surround view image providing unit 107. Meanwhile, the components 101, 103, 105, and 107 shown in FIG. 2 represent functional components of the surround view providing device 100 according to embodiments of the present invention, and each of the components 101, 103, and 105 , 107) may be implemented with a memory and a processor configured to perform a predetermined operation by executing instructions stored in the memory.

The data receiver 101 may receive data from the vehicle 1. To this end, the data receiver 101 may receive data by periodically communicating with the vehicle 1 through a wired or wireless network. Depending on embodiments, the data receiver 101 may receive driving data DD related to the driving of the vehicle from the vehicle 1.

The driving pattern determination unit 103 may determine the driving pattern of the vehicle 1 using the driving data DD transmitted from the vehicle 1.

The driving pattern determination unit 103 may generate a driving feature vector from the driving data DD of the vehicle 1 and determine the driving pattern of the vehicle 1 using the generated driving feature vector. According to embodiments, the driving pattern determination unit 103 determines a cluster corresponding to the driving characteristic vector of the vehicle 1 using a clustering engine, and applies the driving pattern assigned to the determined cluster to the driving pattern corresponding to the vehicle 1. It can be predicted by pattern.

For example, the clustering engine may perform clustering according to a Gaussian Mixture Model (GMM) or clustering according to the K-means algorithm. Here, the K-means algorithm, unlike the Gaussian mixture model, may be a machine learning-based clustering technique belonging to unsupervised learning.

Specifically, the K-means algorithm sets a vector value corresponding to the initial center point for each of the K clusters and then assigns vehicles to clusters with an initial center point close to the driving feature vector representing each vehicle. Next, when cluster assignment to all vehicles is completed, the center point of each cluster is reset to the median or average value of the driving feature vectors of the vehicles belonging to the cluster, and cluster assignment of all vehicles is performed again based on the reset center point. do. Vehicles can be assigned to clusters by repeating the above-described center point resetting and cluster reallocation until there is no change in the center point.

The parameter determination unit 105 may determine (or estimate) a conversion parameter (CPM) for the vehicle 1 based on the driving pattern determined by the driving pattern determination unit 103. Depending on embodiments, the parameter determination unit 105 may determine conversion parameters for the vehicle 1 based on conversion parameters of reference vehicles belonging to a group corresponding to the driving pattern of the vehicle 1.

The surround view image providing unit 107 receives camera images (CV) from the vehicle 1, and determines the image quality of the vehicle 1 according to the conversion parameter (CPM) for the vehicle 1 by the parameter determination unit 105. Camera images (CV) generated by the cameras 11 may be converted, and a surround view image (SVV) may be generated using the converted camera images.

FIG. 4 is a diagram illustrating a process for determining conversion parameters for a vehicle according to a clustering engine according to embodiments of the present invention. Referring to FIG. 4, the driving pattern determination unit 103 uses the driving data DD received by the data receiving unit 101 to generate a driving feature vector (DFV) representing the driving characteristics of the vehicle 1. . For example, the driving pattern determination unit 103 may generate a driving feature vector (DFV) from driving data DD collected during a period from a predetermined reference point to the current point in time (hereinafter referred to as driving period). At this time, the predetermined reference point in time may mean the point in time when matching for the cameras 11 of the vehicle 1 was last achieved or a point in time thereafter.

The driving pattern determination unit 103 may calculate a driving feature vector (DFV) using the angular acceleration data of the vehicle 1 included in the driving data DD. Depending on embodiments, the driving pattern determination unit 103 may calculate a driving feature vector (DFV) including the average and maximum angular velocity of the vehicle 1 during the driving period as components. For example, the driving pattern determination unit 103 may calculate the driving characteristic vector (DFV) for the vehicle 1 according to Equation 1 below.

Here, E(α) is the average angular acceleration vector representing the average of each of the angular acceleration vectors (α) during the driving period, and M(α) is the maximum angular acceleration representing the maximum of each of the angular acceleration vectors (α) during the driving period. It is a vector.

At this time, the angular acceleration vector α can be expressed according to Equation 2 below.

Here, α _r , α _p , and α _y represent angular acceleration components in the roll, pitch, and yaw directions in that order. Meanwhile, according to the above notation, the average angular acceleration vector E(α) can be defined according to Equation 3 below.

Here, T represents the driving period. In other words, each component of the average angular acceleration vector E(α) becomes the average of each component of the angular acceleration vector during the driving period.

Meanwhile, the maximum angular acceleration vector M(α) represents the angular acceleration vector with the largest size among the angular acceleration vectors.

The driving pattern determination unit 103 uses a clustering engine to classify the driving feature vector (DFV) of the vehicle 1 and the driving feature vectors of previously stored reference vehicles into one of a plurality of clusters (CLU), and the driving feature vector ( The driving pattern of the vehicle 1 can be determined by determining the cluster to which the DFV belongs.

In other words, the driving feature vectors of the vehicle 1 and the reference vehicles can be viewed as representing the driving pattern of the corresponding vehicle, and the driving pattern determination unit 103 clusters these driving feature vectors according to a predetermined standard to have similar properties. Driving feature vectors are classified into one cluster. Accordingly, it is understood that each cluster represents a driving pattern of a vehicle, and that driving feature vectors belonging to each cluster can be categorized into identical (or similar) driving patterns.

At this time, driving feature vectors of reference vehicles may be calculated and stored in advance from driving data of reference vehicles. Like the driving characteristic vector of the vehicle 1, the driving characteristic vectors of the reference vehicles may be generated from driving data of the reference vehicle during a period from a predetermined reference point to the current time (hereinafter, driving period). In other words, the driving feature vector is created to correspond to the driving period of the reference vehicle. In this case, in fact, multiple driving feature vectors may be generated for one reference vehicle, and in the clustering process described herein, such driving Each of the feature vectors is treated as being generated from a separate reference vehicle.

At this time, the driving period may vary for each reference vehicle.

Accordingly, the surround view providing device 100 may only store driving data during the driving period for each reference vehicle.

Meanwhile, the clustering engine may be based on a Gaussian mixture model. For example, in the case of Gaussian mixture model (GMM), each of K pre-specified Gaussian distributions (K is a natural number greater than 1) corresponds to a cluster. It operates to classify vehicles into one of K Gaussian distributions. Specifically, in the case of a clustering engine, classification can be done by selecting the Gaussian distribution with the highest result value of the given Gaussian distribution selection function (γ) among K Gaussian distributions using Bayes' theorem.

After clustering, the driving pattern determination unit 103 determines a reference vehicle having the same driving pattern as that of vehicle 1, and determines conversion parameters for vehicle 1 based on the conversion parameters of the determined reference vehicles. You can. In other words, if the driving pattern is similar, the impact (amount of impact) on the cameras installed in the vehicle is likely to be similar, and therefore, the conversion parameters of the vehicle 1 can be estimated by using the conversion parameters of a reference vehicle with a similar driving pattern. You can.

At this time, information about conversion parameters of reference vehicles may be stored in advance. For example, information about conversion parameters calculated upon actual matching of reference vehicles may be stored. Meanwhile, the conversion parameters of the reference vehicles may be values actually measured or calculated at the end of the driving period of each of the reference vehicles. Accordingly, the stored conversion parameters of the reference vehicle correspond to the driving period, and as a result, there is a high possibility that the driving data (or driving pattern) of the reference vehicles will be reflected.

Figure 5 is a diagram for explaining a method of estimating transformation parameters according to embodiments of the present invention. The conversion parameters described in FIG. 5 may be rotation parameters associated with the rotation conversion required to convert the camera image (CV) of each camera installed in the vehicle into a top-view image. For example, the transformation parameter may mean the rotation angle formed between the shooting vector of each camera and the ground.

Referring to FIG. 5, the parameter determination unit 105 performs conversion on vehicle 1 using conversion parameters of other vehicles (i.e., reference vehicles) having the same driving pattern as that of vehicle 1. Parameters can be determined.

Transformation parameters described with reference to FIG. 5 may mean rotation parameters. The rotation parameter is a parameter necessary to convert the images of each camera installed in the vehicle from the viewpoint of the corresponding camera to the top view viewpoint. For example, when converting the viewpoint (or shooting vector) of each camera to the top view viewpoint, each axis ( It may be a parameter indicating how much to rotate around the x-axis, y-axis, and z-axis. For example, the rotation parameter for each vehicle may include information about the angle (i.e., rotation angle) to be rotated around three axes (x-axis, y-axis, z-axis) for each of the cameras installed in each vehicle. You can. If 4 cameras are installed in the vehicle, the rotation parameter may include information about 4 * 3 = 12 rotation angles. However, depending on embodiments, angle information for some of the three axes may be omitted.

_The parameter determination unit ₁₀₅ sets a rotation parameter ( R ₁ ~R _N ) can be determined. At this time, the rotation parameters (R ₁ to R _N ) are transformation parameters used when producing surround view images of other vehicles (i.e., reference vehicles) 2 corresponding to each of the other driving feature vectors (DFV ₁ to DFV _N ). According to embodiments, the parameter determination unit 105 determines other driving feature vectors (DFV 1 to DFV N) included in the cluster (CLU) to which the driving feature vector (DFV) of the vehicle 1 belongs, and determines other driving feature vectors (DFV ₁ to DFV _N ) Identifiers of the reference vehicles 2 corresponding to each of the feature vectors DFV ₁ to DFV _N can be determined. Afterwards, the parameter determination unit 105 may determine rotation parameters (R ₁ to R _N ) corresponding to the reference vehicles 2 using the identifiers of the identified reference vehicles 2 . Rotation parameters (R ₁ ~R _N ) may be stored in advance.

The parameter determination unit 105 may calculate the rotation parameter (R) of the vehicle 1 using the determined rotation parameters (R ₁ to R _N ) of the reference vehicles 2 . According to embodiments, the parameter determination unit 105 determines the vehicle based on the rotation angle corresponding to each axis (x, y, and z) included in the rotation parameters (R ₁ to R _N ) of the reference vehicles 2. The rotation parameter (R) can be determined by calculating the rotation angle for the cameras 11 in (1).

Figure 6 is a diagram for explaining calculating rotation parameters of a vehicle according to embodiments of the present invention. Referring to FIG. 6 , as seen in FIG. 5 , rotation parameters (R ₁ to R _N ) of reference vehicles 2 having the same driving pattern as that of the vehicle 1 are shown. Depending on embodiments, the surround view providing device 100 may store information on rotation parameters of various vehicles, and at this time, the rotation parameters may be stored and used in the form of a table. For example, as shown in FIG. 6, rotation angles corresponding to the x-axis, y-axis, and z-axis of each camera installed in the reference vehicles 2 are shown in order.

Depending on the embodiment, the parameter determination unit 105 may determine the rotation parameter by calculating the rotation angle for each camera of the vehicle 1 based on Equation 4 below.

Here, R _j,x , R _j,y , and R _j,z are each in order of the jth camera of vehicle 1 (j=1, 2, 3, 4; j=1 is the front, j=2 is the rear) , j=3 represents the left side and j=4 represents the right side). a _ij,x , a _ij,z and a _ij,z are positive constants as weights (i=1, 2, ?? N), where for all i and j the sum of a _ij,x, a _ij,z The sum of and the sum of a _ij,z may both be 1. R _ij,x , R _ij,y and R _ij,z represent the rotation angles of the x-axis, y-axis and z-axis with respect to the j-th camera of the ith reference vehicle (2) in that order. For example, in the case of Figure 6, R _11,x =6 and R _13,y =3.

The weights a _ij,x , a _ij,z, and a _ij,z, described in Equation 4 above, may be determined based on the vehicle characteristics of the corresponding i-th reference vehicle and the driving period of the corresponding i-th reference vehicle.

For example, a _ij,x , a _ij,z, and a _ij,z can be determined according to Equation 5 below.

Here, L is an index indicating any one of x, y and z, T is the driving period of vehicle 1, T _i is the driving period corresponding to the ith reference vehicle (driving feature vector), and D _i represents the total driving distance of the i-th reference vehicle, and k _i,j,L is a constant, a value representing the difference between the installation characteristics of the j-th camera of the i-th reference vehicle and the installation characteristics of the j-th camera of the vehicle, It is a value determined in advance.

As described above, according to the embodiments of the present invention, the parameter determination unit 105 has the effect of estimating the rotation parameter of the vehicle 1 using the driving data of the vehicle 1 and the driving data and rotation parameter of the reference vehicle. there is.

The surround view image providing unit 107 may generate a surround view image (SVV) for the vehicle 1 by applying the determined rotation parameter to the received camera image (CV) of the vehicle 1. Depending on embodiments, the surround view image providing unit 107 may directly display the generated surround view image SVV or transmit it to the vehicle 1.

Figure 7 is a diagram illustrating the hardware configuration of a surround view providing device according to embodiments of the present invention.

Referring to FIG. 7, the surround view providing device 100 includes at least one processor 110 and instructions that instruct the at least one processor 110 to perform at least one operation. It may include a storage memory (memory, 120).

Here, at least one operation is interpreted as including at least some of the operations or functions of the above-described surround view providing device 100, and detailed descriptions are omitted to prevent redundant description.

Here, the at least one processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. You can. The memory 120 may be comprised of at least one of a volatile storage medium and a non-volatile storage medium.

The surround view providing device 100 may further include a storage device 160 for non-temporarily storing input data, intermediate processing data, temporary data, output data, etc. resulting from performing at least one operation.

For example, the memory 120 may be one of read only memory (ROM) and random access memory (RAM), and the storage device 160 may be flash memory. , a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, micro SD card).

Additionally, the surround view providing device 100 may further include a transceiver 130 that performs communication through a wireless network. Additionally, the surround view providing device 100 may further include an input interface device 140, an output interface device 150, etc. Each component included in the surround view providing device 100 is connected by a bus 170 and can communicate with each other.

Examples of the surround view providing device 100 include a communication capable desktop computer, a laptop computer, a laptop, a smart phone, a tablet PC, and a mobile phone ( mobile phone, smart watch, smart glass, e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia) broadcasting) player, digital audio recorder, digital audio player, digital video recorder, digital video player, PDA (Personal Digital Assistant), etc.

Methods according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the computer software art.

Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

Additionally, the above-described method or device may be implemented by combining all or part of its components or functions, or may be implemented separately.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

delete

delete

Claims (5)

In a surround view providing device installed in a vehicle to provide a surround view image of the vehicle,
a data receiver configured to receive driving data related to driving of the vehicle from the vehicle;
By generating a driving feature vector for the vehicle from received driving data and classifying the driving feature vector of the vehicle into one of a plurality of clusters, each of which includes a plurality of driving feature vectors, the driving feature vector of the vehicle is determined. a driving pattern determination unit configured to determine a pattern;
a parameter determination unit configured to determine reference vehicles having the same driving pattern as that of the vehicle and determine conversion parameters for the vehicle using stored conversion parameters of the reference vehicle; and
A surround view image providing unit configured to generate the surround view image using conversion parameters for the vehicle and camera images generated by a camera of the vehicle,
The driving data includes information about the angular acceleration measured in the vehicle,
The driving pattern determination unit,
During a predetermined driving period, a driving feature vector representing the average angular acceleration of the vehicle and the maximum angular acceleration is generated according to the equation below,


(Here, DFV is the driving characteristic vector of the vehicle, E(α) is the average angular acceleration vector representing the average of the angular acceleration of the vehicle, and α _r , α _p and α _y are the roll and pitch of the vehicle ( pitch), represents the angular acceleration component in the yaw direction, T is the driving period of the vehicle, and M(α) is the maximum angular acceleration vector with the largest size among the angular acceleration vectors.)
The transformation parameter is a rotation parameter indicating a rotation angle associated with a rotation transformation used to transform camera images of the vehicle into a top-view image,
The parameter determination unit,
Calculating rotation parameters for the vehicle based on rotation angles corresponding to each axis included in rotation parameters of the reference vehicles having the same driving pattern as that of the vehicle,
Surround view providing device. delete delete delete delete