WO2018097356A1 - Vehicle parking assist device and method therefor - Google Patents

Vehicle parking assist device and method therefor Download PDF

Info

Publication number
WO2018097356A1
WO2018097356A1 PCT/KR2016/013633 KR2016013633W WO2018097356A1 WO 2018097356 A1 WO2018097356 A1 WO 2018097356A1 KR 2016013633 W KR2016013633 W KR 2016013633W WO 2018097356 A1 WO2018097356 A1 WO 2018097356A1
Authority
WO
WIPO (PCT)
Prior art keywords
parking
vehicle
line
avm
image
Prior art date
Application number
PCT/KR2016/013633
Other languages
French (fr)
Korean (ko)
Inventor
기석철
이재설
윤희상
Original Assignee
충북대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 충북대학교 산학협력단 filed Critical 충북대학교 산학협력단
Priority to PCT/KR2016/013633 priority Critical patent/WO2018097356A1/en
Publication of WO2018097356A1 publication Critical patent/WO2018097356A1/en

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00Conjoint control of vehicle sub-units of different type or different function
    • B60W10/20Conjoint control of vehicle sub-units of different type or different function including control of steering systems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/06Automatic manoeuvring for parking
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention

Definitions

  • the present embodiment relates to a vehicle parking assist apparatus and a method thereof. More specifically, by using the AVM image to generate and output a parking space located in the surrounding area of the vehicle and a parking map indicating the current position of the vehicle to assist the driver to park the vehicle more efficiently.
  • a vehicle parking assist device and a method thereof by using the AVM image to generate and output a parking space located in the surrounding area of the vehicle and a parking map indicating the current position of the vehicle to assist the driver to park the vehicle more efficiently.
  • Figure 1 (a) is a view showing a parking assistance system using an ultrasonic sensor.
  • the parking assist system scans a space to park using an ultrasonic sensor mounted on a vehicle, and determines whether parking is possible by checking an obstacle located in a corresponding area.
  • both sides of the parking space must be blocked at all times so that the space can be recognized as a parking space.
  • the distance of the parking space can be accurately recognized.
  • FIG. 1B is a diagram illustrating an around view system using a camera.
  • the around view system is provided to support a driver's parking by installing a plurality of cameras photographing the surroundings of the vehicle and providing the driver with omnidirectional images of the surroundings of the vehicle photographed through the plurality of cameras.
  • the surround view system has difficulty in providing automatic parking assistance as the omnidirectional image provided through the monitor provides only a small area around the vehicle, and furthermore, parking is blocked when the parking line is blocked by the surrounding obstacle. The disadvantage is that it doesn't help much.
  • the present embodiment generates and outputs a parking map indicating the current location of the parking space and the parking space located in the surrounding area by using the AVM image to assist the driver to park the vehicle more efficiently. It is a main object to provide a parking assist device and a method thereof.
  • a vehicle parking assist device for assisting parking of a vehicle by using an AVM image, an image obtained by synthesizing the captured images collected from a plurality of cameras provided in the vehicle to obtain an AVM (Around View Monitoring) image Acquisition unit;
  • a calculator configured to calculate at least one image feature point associated with parking of the vehicle from the AVM image;
  • a controller for matching the image feature points to a plurality of sequentially obtained AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result. It provides a vehicle parking assistance device comprising a.
  • the vehicle parking assist apparatus acquires an AVM image by synthesizing captured images collected from a plurality of cameras provided in the vehicle. Process of doing; Calculating at least one image feature point associated with parking of the vehicle from the AVM image; And matching the image feature points to a plurality of sequentially obtained AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result. It provides a vehicle parking assistance method comprising a.
  • the conventional vehicle assistance system generates and outputs a parking map indicating the current location of the parking space and the parking space located in the surrounding area of the vehicle using the AVM image.
  • Overcoming the disadvantages of the driver has the effect of allowing the driver to park the vehicle more efficiently.
  • FIG. 1 is a view schematically showing a conventional parking assistance system.
  • FIG. 2 is a view for explaining the operation of the vehicle parking assisting apparatus according to the present embodiment.
  • FIG. 3 is a block diagram schematically illustrating a vehicle parking assisting apparatus according to the present embodiment.
  • FIG. 4 is a flowchart illustrating a parking assisting method of the vehicle runner assisting apparatus according to the present embodiment.
  • FIG. 5 is an exemplary diagram illustrating an implementation form of a parking map according to the present embodiment.
  • FIG. 6 is an exemplary diagram illustrating a process of generating a parking map according to the present embodiment.
  • FIG. 7 is an exemplary view illustrating predefined parking line pattern information for each type of parking line according to the present embodiment.
  • FIG. 8 is an exemplary view for explaining a parking assist method by the vehicle parking assist apparatus according to the present embodiment.
  • FIG. 9 is an exemplary view for explaining a parking state notification method by the vehicle parking assisting apparatus according to the present embodiment.
  • FIG. 2 is a view for explaining the operation of the vehicle parking assisting apparatus according to the present embodiment.
  • the vehicle parking assistance apparatus 200 refers to a device that provides a parking assistance service for a driver of a vehicle.
  • the vehicle parking assistance apparatus 200 is preferably implemented in the form of vehicle infotainment (IVI) that provides various functions related to the vehicle, but is not necessarily limited thereto.
  • IVI vehicle infotainment
  • the implementation form of the vehicle parking assist apparatus 200 is not limited to a specific form.
  • the vehicle parking assist device 200 is interlocked with various devices provided in the vehicle to provide parking assistance service.
  • the vehicle parking assisting apparatus 200 may collect related images required for generating an AVM image by interworking with an around view monitoring (AVM) camera system implemented in a vehicle.
  • AVM around view monitoring
  • the vehicle parking assisting device 200 may be interlocked with an ultrasonic sensor mounted on the vehicle to receive sensing information measured by the ultrasonic sensor.
  • the vehicle parking assist device 200 may receive vehicle driving information in association with each sensor provided in the vehicle, and the steering system may be used. It may be linked with the transmission and reception of steering information for controlling the movement of the vehicle.
  • the vehicle parking assisting apparatus 200 generates and provides a parking map for assisting the parking of the vehicle by using the AVM image. That is, the vehicle parking assisting apparatus 200 calculates image feature points of the plurality of AVM images obtained sequentially, and compares the calculated image feature points with each other. Subsequently, the vehicle parking assisting apparatus 200 generates a parking space (eg, a dotted line region) located in the peripheral area of the vehicle and a parking map indicating the current position of the vehicle according to a matching result of matching the feature points of the plurality of AVM images. To provide.
  • a parking space eg, a dotted line region
  • the parking map generated by the vehicle parking assisting apparatus 200 is generated by matching the plurality of AVM images based on a matching result of matching the feature points of the plurality of AVM images. That is, the vehicle parking assistance apparatus 200 generates a parking map indicating the current location of the parking space located in the peripheral area of the vehicle by continuously matching the AVM images according to the movement of the vehicle.
  • Such a parking map according to the present embodiment has an effect of providing an image covering a wider area than an omnidirectional image provided through a conventional around view system.
  • the current position of the vehicle is additionally calculated based on a matching result of matching the feature points of the plurality of AVM images, and provided together on the parking map, thereby accurately positioning the vehicle in the conventional parking assistance system.
  • the method for generating a parking map by the vehicle parking assist apparatus 200 according to the present embodiment will be described in detail later in the process of describing the operation of each component of the vehicle parking assist apparatus 200 in FIG. 2.
  • the vehicle parking assistance apparatus 200 searches for a parking space available for parking of an actual vehicle in a parking space located in a peripheral area of the vehicle based on the generated parking map, displays the information on the parking map, and provides the driver's parking. Can support
  • the vehicle parking assist apparatus 200 may calculate and provide a vehicle movement trajectory, that is, a parking path, from the current vehicle position to the parking available space based on the found available parking space and the parking map.
  • the vehicle parking assist apparatus 200 may provide a control signal for controlling the steering means of the vehicle based on the calculated parking path and the parking map when the vehicle having the vehicle parking assist apparatus 200 provides the automatic parking assistance function. By creating and providing a vehicle, the vehicle can be automatically parked in a parking space.
  • FIG. 3 is a block diagram schematically illustrating a vehicle parking assisting apparatus according to the present embodiment.
  • the vehicle parking assisting apparatus 200 includes a communication unit 300, an image acquisition unit 310, a calculation unit 320, a control unit 330, a storage unit 340, and a search unit.
  • the components included in the vehicle parking assist apparatus 200 are not necessarily limited thereto.
  • the communication unit 300 communicates with various devices provided in the vehicle, and thereby transmits and receives data necessary for the vehicle parking assist device 200 to provide a parking assistance service.
  • Data transmitted and received by the vehicle parking assisting apparatus 200 according to the present embodiment using the communication unit 300 may include photographed images captured by a plurality of cameras included in the AVM camera system, sensing information measured by an ultrasonic sensor, and a steering system. Steering information of the vehicle may be collected.
  • the communication unit 200 may include at least one communication module among a wired communication module and a wireless communication module according to a communication method supported by the vehicle.
  • the wireless communication module may support Wi-Fi, Zigbee, Bluetooth, WAVE and the like.
  • the image acquisition unit 310 generates an AVM image by synthesizing the captured images collected from the plurality of cameras provided in the vehicle.
  • the vehicle parking assistance apparatus 200 provides a parking assistance service using the AVM image.
  • an AVM camera system is mounted on a vehicle in which the vehicle parking assist apparatus 200 is implemented, and the vehicle parking assist apparatus 200 collects related images necessary for generating an AVM image through interworking with the AVM camera system. do.
  • the AVM camera system includes a plurality of cameras (not shown), so that the image acquisition unit 310 captures the captured images of the front, rear, left, and right rooms of the vehicle photographed using the plurality of cameras. It is provided as a related video.
  • the image acquisition unit 310 generates an AVM image by converting and synthesizing the captured images collected from the plurality of cameras into a top view, that is, Bird's Eye View.
  • the vehicle parking assist The apparatus 200 may be implemented to receive an AVM image directly from an AVM camera system mounted in a vehicle. In this case, the image acquisition unit 310 serves to deliver the AVM image provided from the AVM camera system to the calculation unit 320.
  • the calculator 320 calculates at least one image feature point associated with the parking of the vehicle from the AVM image provided by the image acquirer 310.
  • the image feature point calculated by the calculator 320 is illustrated as being a parking line in the AVM image, but is not necessarily limited thereto.
  • the calculator 320 may further calculate an object corresponding to the obstacle in the AVM image as an image feature point when there is an obstacle including a parking vehicle in the peripheral area of the vehicle.
  • the calculator 320 may calculate an image feature point in the AVM image by measuring a luminance average value and a luminance dispersion value of the AVM image, and may calculate an image feature point in the VM image by image processing the AVM image.
  • the method of calculating the image feature point in the AVM image is not limited to a specific method.
  • the controller 330 generates and provides a parking map for assisting the parking of the vehicle based on the AVM image generated using the image acquisition unit 310 and the image feature point calculated using the calculation unit 320.
  • the controller 330 matches and compares the image feature points with respect to each of a plurality of sequentially obtained AVM images.
  • the controller 330 may use a scan matching technique as a method for matching image feature points of a plurality of AVM images.
  • the scan matching technique compares an image of a current view with an image of a view immediately before the current view, and finds and matches matching points in overlapping portions according to a comparison result.
  • the control unit 330 uses the scan matching technique described above to normalize and compare two consecutive images changed by the movement of the vehicle.
  • the scan matching technique is a known technique and detailed description thereof will be omitted.
  • the controller 330 determines a moving trajectory of the image feature point according to the matching result of the image feature points of the plurality of AVM images, and generates a parking map by matching the plurality of AVM images according to the identified moving trajectory. Meanwhile, in the present embodiment, the parking line in the AVM image is extracted as the image feature point. Accordingly, the controller 330 may recognize a parking line located near the vehicle according to the matching result of the image feature points of the plurality of AVM images, and generate a parking map indicating the parking space based on the recognized parking line. .
  • the controller 330 further detects the movement trajectory of the vehicle through the movement trajectory of the image feature point, and estimates and provides the current position of the vehicle.
  • the controller 330 shows the estimated current position of the vehicle together on the parking map and provides the driver's parking more efficiently.
  • the controller 330 calculates parameter values for correcting displacement and angle differences between the plurality of AVM images according to image feature point matching results of the plurality of AVM images, and matches the plurality of AVM images based on the calculated parameter values. . This is to correct an error between two consecutive AVM images that may occur according to the moving position of the vehicle. Through this, the controller 330 according to the present embodiment may cause an error that may occur in the process of generating a parking map. There is an effect that can minimize the occurrence.
  • the controller 330 may additionally use predetermined parking line pattern information for each type of parking line in the process of generating the parking map.
  • the parking line pattern information is information defining a standard of the corresponding parking line so that the specific parking line can be distinguished from other parking lines, and may include a corner shape and a line shape of the parking line.
  • the parking line recognized by the controller 330 may not be recognized by a vehicle or an obstacle, and part of the parking line may be out of the field of view of the AVM camera system. Areas may also not be recognized.
  • the parking map must provide a parking space based on an incomplete parking line, which is a problem that the accuracy of the parking assistance function of the vehicle using the vehicle parking assistance device 200 may be degraded. do.
  • the controller 330 uses the parking line pattern information as described above when the parking line recognized according to the image feature point matching result of the plurality of AVM images is a partial parking line. Correct the line and generate a parking map based on the corrected parking line.
  • the incomplete parking line means a parking line in which only a part of the parking line recognized by the controller 330 is recognized, not the entire area.
  • the controller 330 calculates the characteristics of the parking line from the partial region of the recognized incomplete parking line.
  • the characteristic of the parking line is preferably the shape of the corner portion of the incomplete parking line, but is not necessarily limited thereto.
  • the controller 330 calculates parking line pattern information corresponding to the above-mentioned characteristic of the parking line from among the predetermined parking line pattern information for each type of parking line, and uses the calculated parking line pattern information to partially or all the remaining areas of the incomplete parking line. Predict.
  • the controller 330 corrects the incomplete parking line by merging the predicted remaining area into a partial area of the previously recognized incomplete parking line, and generates a parking map based on the corrected parking line.
  • the controller 330 may reflect information generated from other constituent means of the vehicle runner assisting device 200 on the parking map.
  • the storage unit 340 stores information necessary for the vehicle parking assistance apparatus 200 according to the present embodiment to perform the parking assistance function of the vehicle.
  • the storage unit 340 may store the predetermined parking line pattern information for each type of parking line.
  • the storage unit 340 may store the parking line pattern information in any one of a template and learning data.
  • Pre-defined vehicle type information for each type of vehicle in the storage unit 340 may be stored.
  • Vehicle shape information may include information about the shape and size of the vehicle.
  • the search unit 350 searches for and provides a parking available space of an actual vehicle among the parking spaces located in the peripheral area of the vehicle based on the parking map generated using the controller 330.
  • the search unit 350 checks the relative position between the current position of the vehicle and the parking space and the presence of obstacles in the parking space by using the parking map, and the parking space where the actual vehicle can be parked among the parking spaces according to the check result.
  • a method of searching for a parking space of a vehicle by the search unit 350 is not limited to a specific method.
  • the search unit 350 may search for a parking space of the vehicle by further utilizing ultrasonic sensing information collected from the ultrasonic sensor provided in the vehicle.
  • the search unit 350 assures the reliability of the searched parking space by searching for a parking space of the vehicle using a parking map whose accuracy is guaranteed through a calibration procedure using the parking line pattern information. It has the effect of making it possible.
  • the search unit 350 transmits the searched parking space to the controller 330 so that the parking space can be additionally displayed on the parking map. According to an exemplary embodiment, the search unit 350 may calculate an optimal parking path of the vehicle to the searched parking space and additionally provide the same.
  • the automatic parking support unit 360 serves to assist the automatic parking support function of the vehicle through linkage with the steering system of the vehicle.
  • the automatic parking support unit 360 controls a control signal for controlling the vehicle steering means based on the parking map generated using the controller 330 and the optimum parking path calculated using the search unit 350. Generates and transmits the generated control signal to the steering means.
  • the automatic parking assistance unit 360 may collect vehicle driving information from each sensor provided in the vehicle in advance.
  • the display 370 receives and displays the parking map generated from the controller 210.
  • the display unit 370 includes at least one display means for displaying a parking map.
  • the display means may be an audio visual navigation (AVN) mounted in a vehicle.
  • APN audio visual navigation
  • the parking state notification unit 380 checks the parking state of the vehicle based on the parking line displayed on the parking map and the current position of the vehicle, and recognizes the confirmed parking state of the vehicle in a form that can be recognized by the user. Output
  • the parking state notification unit 380 determines the parking state of the vehicle by using the parking map whose accuracy is guaranteed through the calibration procedure using the parking line pattern information, thereby allowing the driver to determine the exact parking state of the vehicle. It is effective to confirm.
  • FIG. 4 is a flowchart illustrating a parking assisting method of the vehicle runner assisting apparatus according to the present embodiment.
  • the vehicle parking assist apparatus 200 obtains an AVM image by synthesizing the captured images collected from the plurality of cameras provided in the vehicle (S402).
  • the vehicle parking assist apparatus 200 calculates at least one image feature point related to the parking of the vehicle from the AVM image acquired in step S402 (S404). In operation S404, the vehicle parking assisting apparatus 200 calculates a parking line in the AVM image as an image feature point.
  • the vehicle parking assist apparatus 200 matches the image feature points calculated in step S404 with respect to the plurality of AVM images obtained sequentially (S406).
  • the vehicle parking assistance apparatus 200 generates and outputs a parking space indicating a parking space located in a peripheral area of the vehicle and a current map of the vehicle according to the matching result of step S406 (S408).
  • the vehicle parking assisting apparatus 200 may recognize a parking line located near the vehicle according to the matching result of step S406, and generate a parking map indicating the parking space based on the recognized parking line.
  • the vehicle parking assisting device 200 corrects the parking line using previously stored parking line pattern information when the parking line recognized according to the matching result of step S406 is an incomplete parking line, and based on the corrected parking line. Create a parking map.
  • the vehicle parking assist apparatus 200 searches for a parking available space of an actual vehicle in the parking space located in the surrounding area of the vehicle based on the parking map generated in step S408, and optimizes the search result and the available parking space.
  • the parking route is calculated and provided (S410).
  • the vehicle parking assist apparatus 200 checks whether the automatic parking function is selected in the vehicle (S412), and when the automatic parking function is selected, the vehicle parking assist apparatus 200 adjusts the steering means of the vehicle based on the parking map of step S408 and the optimal parking path of step S410.
  • a control signal for controlling is generated and output (S412).
  • each process is described as being sequentially executed, but is not necessarily limited thereto. In other words, since the process described in FIG. 4 may be applied by changing or executing one or more processes in parallel, FIG. 4 is not limited to the time series order.
  • FIG. 5 is an exemplary diagram illustrating an embodiment of a parking map according to the present embodiment
  • FIG. 6 is an exemplary diagram illustrating a process of generating a parking map according to the present embodiment.
  • the vehicle parking assisting apparatus 200 recognizes a parking line located near the vehicle according to a matching result of matching the feature points of the plurality of AVM images, and indicates a parking space based on the recognized parking line. Create and provide a parking map.
  • the vehicle parking assisting apparatus 200 corrects the parking line using previously stored parking line pattern information when the recognized parking line is an incomplete parking line, and based on the corrected parking line, the parking map.
  • the vehicle parking assist apparatus 200 generates and provides a parking map indicating a parking space based on the recognized parking line, but uses previously stored parking line pattern information for an incomplete parking line. To correct the shape (dotted line area).
  • FIG. 7 is an exemplary view illustrating predefined parking line pattern information for each type of parking line according to the present embodiment.
  • the vehicle parking assistance apparatus 200 when the parking line recognized in the process of generating the parking map is an incomplete parking line, the shape and size of the first half of the corresponding parking line using the predetermined parking line pattern information for each type of parking line Predict. Thereafter, the vehicle parking assistance apparatus 200 corrects the incomplete parking line based on the prediction result, and generates a parking map based on the corrected parking line.
  • the parking line pattern information defines and provides unique features, for example, a corner shape and a line shape, for the parking lines currently being used.
  • FIG. 8 is an exemplary view for explaining a parking assist method by the vehicle parking assist apparatus according to the present embodiment.
  • the vehicle parking assisting apparatus 200 searches for a parking available space of an actual vehicle in a parking space located in a peripheral area of the vehicle based on a parking map. By displaying this on the parking map, it can be confirmed that the driver supports the parking.
  • the vehicle parking assisting apparatus 200 may additionally calculate and provide the current position of the vehicle based on the parking map, by using the parking path of the vehicle to the previously available parking space using the same It can also calculate and provide.
  • the vehicle parking assistance apparatus 200 supports the vehicle to be parked automatically in the available parking space based on the calculated parking path and the parking map.
  • FIG. 9 is an exemplary view for explaining a parking state notification method by the vehicle parking assisting apparatus according to the present embodiment.
  • the vehicle parking assisting apparatus 200 in the case of the vehicle parking assisting apparatus 200 according to the present embodiment, it is possible to check the parking state of the vehicle based on the parking line displayed on the parking map and the current position of the vehicle when the parking of the vehicle is completed. Able to know. Furthermore, the vehicle parking assisting apparatus 200 according to the present embodiment operates to enable the driver to check the parking state of the vehicle by using the parking map whose accuracy is guaranteed through the correction procedure using the parking line pattern information. Reliability can also be ensured.
  • control unit 340 storage unit
  • search unit 360 automatic parking support unit

Abstract

The present embodiment relates to: a vehicle parking assist device, which generates and outputs a parking map indicating a parking space located in a surrounding area of the vehicle by using an around view monitoring (AVM) image, thereby assisting a driver to park a vehicle more efficiently; and a method therefor. Specifically, provided is the vehicle parking assist device comprising: an image acquisition unit for acquiring an AVM image by synthesizing captured images collected from a plurality of cameras included in the vehicle; a calculation unit for calculating at least one image feature point related to parking of the vehicle from the AVM image; and a control unit for matching the image feature point with respect to a plurality of AVM images sequentially acquired, and generating and outputting a parking map indicating a parking space located in a surrounding area of the vehicle and a current location of the vehicle according to the matching result.

Description

차량 주차보조 장치 및 그 방법Vehicle parking assist device and method
본 실시예는 차량 주차보조 장치 및 그 방법에 관한 것이다. 보다 상세하게는, AVM 영상을 활용하여 차량의 주변 영역에 위치하는 주차공간 및 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력함으로써 운전자로 하여금 보다 효율적으로 차량의 주차를 수행할 수 있도록 보조하는 차량 주차보조 장치 및 그 방법에 관한 것이다.The present embodiment relates to a vehicle parking assist apparatus and a method thereof. More specifically, by using the AVM image to generate and output a parking space located in the surrounding area of the vehicle and a parking map indicating the current position of the vehicle to assist the driver to park the vehicle more efficiently. A vehicle parking assist device and a method thereof.
이하에 기술되는 내용은 단순히 본 실시예와 관련되는 배경 정보만을 제공할 뿐 종래기술을 구성하는 것이 아니다.The contents described below merely provide background information related to this embodiment and do not constitute a prior art.
자동차는 현대 도시생활에서 빠질 수 없는 대표적인 이동수단으로 자리 잡게 되면서, 더욱 빠른 속도로 보급이 확대되고 있다. 한편, 대부분의 운전자에게 있어서 자동차의 주차는 가장 큰 어려움으로써 손꼽히고 있다. 이러한, 주차 문제에 도움을 제공하기 위하여 주차 보조 시스템이 개발된 바 있으며, 이미 여러 가지 형태로 제품에 적용되어 판매되고 있다. 이하, 도 1을 참조하여 종래의 주차 보조 시스템을 설명하도록 한다.As automobiles become a representative means of transportation indispensable in modern urban life, they are being expanded at a faster rate. On the other hand, for most drivers, parking of cars is one of the biggest difficulties. Parking assistance system has been developed to help with the parking problem, and has already been applied to products in various forms. Hereinafter, a conventional parking assistance system will be described with reference to FIG. 1.
도 1의 (a)는 초음파 센서를 이용한 주차 보조 시스템을 나타낸 도면이다. 이러한, 주차 보조 시스템(Parking Assist System)은 차량에 장착된 초음파 센서를 이용하여 주차할 공간을 스캔하고, 이를 통해, 해당 영역에 자리하고 있는 장애물을 확인함으로써 주차 가능 여부를 판단하게 된다. 하지만, 초음파 센서를 이용한 주차 보조 시스템의 경우 주차할 공간의 양 옆이 항상 막혀 있어야 해당 공간을 주차 가능 공간으로서 인식할 수 있다는 한계를 가지며, 더욱이, 초음파의 특성상 주차할 공간의 거리를 정밀하게 인식할 수 없다는 단점이 존재한다. 또한, 주차 과정에서 차량의 위치를 정확하게 인식하지 못함에 따라 자동 주차 지원기능을 제공하는 데 있어서 어려움이 존재한다는 문제점이 존재한다.Figure 1 (a) is a view showing a parking assistance system using an ultrasonic sensor. The parking assist system scans a space to park using an ultrasonic sensor mounted on a vehicle, and determines whether parking is possible by checking an obstacle located in a corresponding area. However, in the case of the parking assistance system using the ultrasonic sensor, both sides of the parking space must be blocked at all times so that the space can be recognized as a parking space. Furthermore, due to the nature of the ultrasonic waves, the distance of the parking space can be accurately recognized. There is a disadvantage of not being able to. In addition, there is a problem in that there is a difficulty in providing an automatic parking assistance function as the location of the vehicle is not accurately recognized in the parking process.
도 1의 (b)는 카메라를 이용한 어라운드 뷰 시스템을 나타낸 도면이다. 어라운드 뷰 시스템(Around View System)은 차량 주변을 촬영하는 카메라를 복수 개 설치하고, 복수의 카메라를 통하여 촬영된 차량 주변의 전방향 영상을 운전자에게 제공함으로써 운전자의 주차를 보조하도록 이루어진다. 하지만, 어라운드 뷰 시스템은 모니터를 통해 제공되는 전방향 영상이 차량 주변의 좁은 영역만을 제공함에 따라 자동 주차 지원기능을 제공하는데 있어서 어려움이 존재하며, 더욱이, 주변 장애물로 인해 주차선이 가려지는 경우 주차를 하는데 있어서 크게 도움이 되지 못한다는 단점이 존재한다.1B is a diagram illustrating an around view system using a camera. The around view system is provided to support a driver's parking by installing a plurality of cameras photographing the surroundings of the vehicle and providing the driver with omnidirectional images of the surroundings of the vehicle photographed through the plurality of cameras. However, the surround view system has difficulty in providing automatic parking assistance as the omnidirectional image provided through the monitor provides only a small area around the vehicle, and furthermore, parking is blocked when the parking line is blocked by the surrounding obstacle. The disadvantage is that it doesn't help much.
따라서, 종래의 주차 보조 시스템의 단점을 극복하면서도 운전자로 하여금 보다 효율적으로 차량을 주차 가능토록 보조하는 새로운 기술을 필요로 한다.Thus, there is a need for a new technology that assists drivers to park their vehicles more efficiently while overcoming the disadvantages of conventional parking assistance systems.
본 실시예는 AVM 영상을 활용하여 차량의 주변 영역에 위치하는 주차공간 및 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력함으로써 운전자로 하여금 보다 효율적으로 차량의 주차를 수행할 수 있도록 보조하는 차량 주차보조 장치 및 그 방법을 제공하는 데 주된 목적이 있다.The present embodiment generates and outputs a parking map indicating the current location of the parking space and the parking space located in the surrounding area by using the AVM image to assist the driver to park the vehicle more efficiently. It is a main object to provide a parking assist device and a method thereof.
본 실시예는, AVM 영상을 활용하여 차량의 주차를 보조하는 차량 주차보조 장치에 있어서, 상기 차량에 구비된 복수 개의 카메라로부터 수집한 촬영 영상을 합성하여 AVM(Around View Monitoring) 영상을 획득하는 영상 획득부; 상기 AVM 영상에서 상기 차량의 주차와 관련된 적어도 하나의 영상 특징점을 산출하는 산출부; 및 순차적으로 획득된 복수의 AVM 영상에 대하여 상기 영상 특징점을 매칭하고, 매칭결과에 따라 상기 차량의 주변 영역에 위치하는 주차공간 및 상기 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력하는 제어부를 포함하는 것을 특징으로 하는 차량 주차보조 장치를 제공한다.In the present embodiment, a vehicle parking assist device for assisting parking of a vehicle by using an AVM image, an image obtained by synthesizing the captured images collected from a plurality of cameras provided in the vehicle to obtain an AVM (Around View Monitoring) image Acquisition unit; A calculator configured to calculate at least one image feature point associated with parking of the vehicle from the AVM image; And a controller for matching the image feature points to a plurality of sequentially obtained AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result. It provides a vehicle parking assistance device comprising a.
또한, 본 실시예의 다른 측면에 의하면, 차량 주차보조 장치가 AVM 영상을 활용하여 차량의 주차를 보조하는 방법에 있어서, 상기 차량에 구비된 복수 개의 카메라로부터 수집한 촬영 영상을 합성하여 AVM 영상을 획득하는 과정; 상기 AVM 영상에서 상기 차량의 주차와 관련된 적어도 하나의 영상 특징점을 산출하는 과정; 및 순차적으로 획득된 복수의 AVM 영상에 대하여 상기 영상 특징점을 매칭하고, 매칭결과에 따라 상기 차량의 주변 영역에 위치하는 주차공간 및 상기 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력하는 과정을 포함하는 것을 특징으로 하는 차량 주차보조 방법을 제공한다.According to another aspect of the present embodiment, in a method for assisting parking of a vehicle by using the AVM image, the vehicle parking assist apparatus acquires an AVM image by synthesizing captured images collected from a plurality of cameras provided in the vehicle. Process of doing; Calculating at least one image feature point associated with parking of the vehicle from the AVM image; And matching the image feature points to a plurality of sequentially obtained AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result. It provides a vehicle parking assistance method comprising a.
이상에서 설명한 바와 같이 본 실시예에 의하면, 차량 주자보조 장치가 AVM 영상을 활용하여 차량의 주변 영역에 위치하는 주차공간 및 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력함으로써 종래의 주차 보조 시스템의 단점을 극복하면서도 운전자로 하여금 보다 효율적으로 차량을 주차 가능토록 하는 효과가 있다.As described above, according to the present embodiment, the conventional vehicle assistance system generates and outputs a parking map indicating the current location of the parking space and the parking space located in the surrounding area of the vehicle using the AVM image. Overcoming the disadvantages of the driver has the effect of allowing the driver to park the vehicle more efficiently.
도 1은 종래의 주차 보조 시스템을 개략적으로 나타낸 도면이다.1 is a view schematically showing a conventional parking assistance system.
도 2는 본 실시예에 따른 차량 주차보조 장치의 동작을 설명하기 위한 도면이다.2 is a view for explaining the operation of the vehicle parking assisting apparatus according to the present embodiment.
도 3은 본 실시예에 따른 차량 주차보조 장치를 개략적으로 나타낸 블록 구성도이다.3 is a block diagram schematically illustrating a vehicle parking assisting apparatus according to the present embodiment.
도 4는 본 실시예에 따른 차량 주자보조 장치의 주차보조 방법을 설명하기 위한 순서도이다.4 is a flowchart illustrating a parking assisting method of the vehicle runner assisting apparatus according to the present embodiment.
도 5는 본 실시예에 따른 주차 맵의 구현 형태를 예시한 예시도이다.5 is an exemplary diagram illustrating an implementation form of a parking map according to the present embodiment.
도 6은 본 실시예에 따른 주차 맵의 생성과정을 예시한 예시도이다.6 is an exemplary diagram illustrating a process of generating a parking map according to the present embodiment.
도 7은 본 실시예에 따른 주차선의 종류별 기 정의된 주차선 패턴정보를 예시한 예시도이다.7 is an exemplary view illustrating predefined parking line pattern information for each type of parking line according to the present embodiment.
도 8은 본 실시예에 따른 차량 주차보조 장치에 의한 주차보조 방법을 설명하기 위한 예시도이다.8 is an exemplary view for explaining a parking assist method by the vehicle parking assist apparatus according to the present embodiment.
도 9는 본 실시예에 따른 차량 주차보조 장치에 의한 주차상태 알림 방법을 설명하기 위한 예시도이다.9 is an exemplary view for explaining a parking state notification method by the vehicle parking assisting apparatus according to the present embodiment.
이하, 본 실시예를 첨부된 도면을 참조하여 상세하게 설명한다.Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.
도 2는 본 실시예에 따른 차량 주차보조 장치의 동작을 설명하기 위한 도면이다.2 is a view for explaining the operation of the vehicle parking assisting apparatus according to the present embodiment.
차량 주차보조 장치(200)는 차량의 운전자를 위한 주차 보조 서비스를 제공하는 장치를 의미한다. 이러한, 차량 주차보조 장치(200)는 바람직하게는 차량과 관련된 각종 기능을 제공하는 차량 인포테인먼트(IVI: In Vehicle Infotainment) 형태로 구현되는 것이 바람직하나 반드시 이에 한정되는 것은 아니다. 본 실시예에서는 차량 주차보조 장치(200)의 구현 형태에 대해서 특정 형태로 한정하지는 않는다.The vehicle parking assistance apparatus 200 refers to a device that provides a parking assistance service for a driver of a vehicle. The vehicle parking assistance apparatus 200 is preferably implemented in the form of vehicle infotainment (IVI) that provides various functions related to the vehicle, but is not necessarily limited thereto. In the present embodiment, the implementation form of the vehicle parking assist apparatus 200 is not limited to a specific form.
차량 주차보조 장치(200)는 주차 보조 서비스를 제공하기 위해 차량 내 구비된 각종 장치와 연동된다. 예컨대, 차량 주차보조 장치(200)는 차량 내 구현된 AVM(Arround View Monitoring) 카메라 시스템과 연동되어 AVM 영상을 생성하는 데 있어서 필요한 관련 영상을 수집할 수 있다.The vehicle parking assist device 200 is interlocked with various devices provided in the vehicle to provide parking assistance service. For example, the vehicle parking assisting apparatus 200 may collect related images required for generating an AVM image by interworking with an around view monitoring (AVM) camera system implemented in a vehicle.
차량 주차보조 장치(200)는 차량에 장착된 초음파 센서와 연동되어, 초음파 센서에 의해 측정된 센싱정보를 전달받을 수 있다.The vehicle parking assisting device 200 may be interlocked with an ultrasonic sensor mounted on the vehicle to receive sensing information measured by the ultrasonic sensor.
차량 주차보조 장치(200)는 차량 주차보조 장치(200)를 구비한 차량이 자동 주차 지원기능을 제공하는 경우, 차량 내 구비된 각 센서들과 연동되어 차량 주행 정보를 수신할 수 있으며, 조향 시스템과 연동되어 차량의 움직임을 제어하기 위한 조향 정보를 송수신할 수 있다.When the vehicle equipped with the vehicle parking assist device 200 provides an automatic parking assistance function, the vehicle parking assist device 200 may receive vehicle driving information in association with each sensor provided in the vehicle, and the steering system may be used. It may be linked with the transmission and reception of steering information for controlling the movement of the vehicle.
본 실시예에 따른 차량 주차보조 장치(200)는 AVM 영상을 활용하여 차량의 주차를 보조하기 위한 주차 맵을 생성하여 제공한다. 즉, 차량 주차보조 장치(200)는 순차적으로 획득된 복수의 AVM 영상에 대하여 영상 특징점을 산출하고, 산출된 영상 특징점들을 서로 매칭하여 비교한다. 이후, 차량 주차보조 장치(200)는 복수의 AVM 영상의 특징점들을 매칭한 매칭결과에 따라 차량의 주변 영역에 위치하는 주차공간(ex: 점선 영역) 및 차량의 현재 위치를 지시하는 주차 맵을 생성하여 제공한다.The vehicle parking assisting apparatus 200 according to the present embodiment generates and provides a parking map for assisting the parking of the vehicle by using the AVM image. That is, the vehicle parking assisting apparatus 200 calculates image feature points of the plurality of AVM images obtained sequentially, and compares the calculated image feature points with each other. Subsequently, the vehicle parking assisting apparatus 200 generates a parking space (eg, a dotted line region) located in the peripheral area of the vehicle and a parking map indicating the current position of the vehicle according to a matching result of matching the feature points of the plurality of AVM images. To provide.
한편, 본 실시예에 따른 차량 주차보조 장치(200)에 의해 생성되는 주차 맵은 복수의 AVM 영상의 특징점을 매칭한 매칭결과에 기반하여 복수의 AVM 영상들을 정합하는 단계를 거쳐 생성된다. 즉, 차량 주차보조 장치(200)는 연속되는 AVM 영상들을 차량의 이동에 따라 계속 정합함으로써 차량의 주변 영역에 위치하는 주차공간 및 차량의 현재 위치를 지시하는 주차 맵을 생성한다. 이러한, 본 실시예에 따른 주차 맵의 경우 종래의 어라운드 뷰 시스템을 통해 제공되는 전방향 영상 대비 보다 넓은 영역을 커버하는 영상을 제공할 수 있는 효과가 있다. 또한, 본 실시예의 경우 복수의 AVM 영상의 특징점을 매칭한 매칭결과에 기반하여 차량의 현재 위치를 추가로 산출하고, 이를 주차 맵 상에 함께 제공함에 따라 종래의 주차 보조 시스템에서 차량의 위치를 정확하게 인식하지 못함에 따라 야기되는 문제점을 사전에 방지할 수 있는 효과가 있다.Meanwhile, the parking map generated by the vehicle parking assisting apparatus 200 according to the present embodiment is generated by matching the plurality of AVM images based on a matching result of matching the feature points of the plurality of AVM images. That is, the vehicle parking assistance apparatus 200 generates a parking map indicating the current location of the parking space located in the peripheral area of the vehicle by continuously matching the AVM images according to the movement of the vehicle. Such a parking map according to the present embodiment has an effect of providing an image covering a wider area than an omnidirectional image provided through a conventional around view system. In addition, in the present embodiment, the current position of the vehicle is additionally calculated based on a matching result of matching the feature points of the plurality of AVM images, and provided together on the parking map, thereby accurately positioning the vehicle in the conventional parking assistance system. There is an effect that can prevent in advance the problems caused by the failure to recognize.
본 실시예에 따른 차량 주차보조 장치(200)가 주차 맵을 생성하는 방법은 도 2에서 차량 주차보조 장치(200)의 각 구성요소의 동작을 설명하는 과정에서 보다 자세하게 후술토록 한다.The method for generating a parking map by the vehicle parking assist apparatus 200 according to the present embodiment will be described in detail later in the process of describing the operation of each component of the vehicle parking assist apparatus 200 in FIG. 2.
차량 주차보조 장치(200)는 생성된 주차 맵에 기반하여 차량의 주변 영역에 위치하는 주차공간 내 실제 차량의 주차가 가능한 주차 가능공간을 탐색하고, 이를 주차 맵 상에 표시하여 제공함으로써 운전자의 주차를 지원할 수 있다. The vehicle parking assistance apparatus 200 searches for a parking space available for parking of an actual vehicle in a parking space located in a peripheral area of the vehicle based on the generated parking map, displays the information on the parking map, and provides the driver's parking. Can support
차량 주차보조 장치(200)는 탐색된 주차 가능공간 및 주차 맵에 기반하여 현재 차량의 위치로부터 상기의 주차 가능공간까지의 차량 이동 궤적, 즉, 주차 경로를 산출하여 운전자에게 제공할 수 있다.The vehicle parking assist apparatus 200 may calculate and provide a vehicle movement trajectory, that is, a parking path, from the current vehicle position to the parking available space based on the found available parking space and the parking map.
차량 주차보조 장치(200)는 차량 주차보조 장치(200)를 구비한 차량이 자동 주차 지원기능을 제공하는 경우, 산출된 주차 경로 및 주차 맵에 기반하여 차량의 조향 수단을 제어하기 위한 제어신호를 생성하여 제공함으로써 차량이 주차 가능공간에 자동 주차될 수 있도록 지원할 수 있다.The vehicle parking assist apparatus 200 may provide a control signal for controlling the steering means of the vehicle based on the calculated parking path and the parking map when the vehicle having the vehicle parking assist apparatus 200 provides the automatic parking assistance function. By creating and providing a vehicle, the vehicle can be automatically parked in a parking space.
도 3은 본 실시예에 따른 차량 주차보조 장치를 개략적으로 나타낸 블록 구성도이다.3 is a block diagram schematically illustrating a vehicle parking assisting apparatus according to the present embodiment.
도 3에 도시하듯이, 본 실시예에 따른 차량 주차보조 장치(200)는 통신부(300), 영상 획득부(310), 산출부(320), 제어부(330), 저장부(340), 탐색부(350), 자동주차 지원부(360), 디스플레이부(370) 및 주차상태 알림부(380)를 포함한다. 여기서, 차량 주차보조 장치(200)에 포함되는 구성요소는 반드시 이에 한정되는 것은 아니다.As shown in FIG. 3, the vehicle parking assisting apparatus 200 according to the present embodiment includes a communication unit 300, an image acquisition unit 310, a calculation unit 320, a control unit 330, a storage unit 340, and a search unit. The unit 350, the automatic parking support unit 360, the display unit 370 and the parking state notification unit 380. Here, the components included in the vehicle parking assist apparatus 200 are not necessarily limited thereto.
통신부(300)는 차량 내 구비된 각종 장치와 통신을 수행하며, 이를 통해, 차량 주차보조 장치(200)가 주차 보조 서비스를 제공하는 데 있어서 필요한 데이터들을 송수신한다. 본 실시예에 따른 차량 주차보조 장치(200)가 통신부(300)를 이용하여 송수신하는 데이터는 AVM 카메라 시스템에 포함된 복수 개의 카메라로부터 촬영된 촬영 영상, 초음파 센서에 의해 측정된 센싱정보 및 조향 시스템에서 수집된 차량의 조향 정보 등일 수 있다.The communication unit 300 communicates with various devices provided in the vehicle, and thereby transmits and receives data necessary for the vehicle parking assist device 200 to provide a parking assistance service. Data transmitted and received by the vehicle parking assisting apparatus 200 according to the present embodiment using the communication unit 300 may include photographed images captured by a plurality of cameras included in the AVM camera system, sensing information measured by an ultrasonic sensor, and a steering system. Steering information of the vehicle may be collected.
통신부(200)는 차량에서 지원하는 통신 방식에 따라 유선통신 모듈 및 무선통신 모듈 중 적어도 하나의 통신모듈을 구비할 수 있다. 이때, 무선통신 모듈은 Wi-Fi, 지그비, 블루투스, WAVE 등을 지원할 수 있다.The communication unit 200 may include at least one communication module among a wired communication module and a wireless communication module according to a communication method supported by the vehicle. At this time, the wireless communication module may support Wi-Fi, Zigbee, Bluetooth, WAVE and the like.
영상 획득부(310)는 차량에 구비된 복수 개의 카메라로부터 수집한 촬영 영상을 합성하여 AVM 영상을 생성한다. 한편, 본 실시예에 따른 차량 주차보조 장치(200)는 AVM 영상을 활용한 주차 보조 서비스를 제공한다. 이를 위해, 차량 주차보조 장치(200)가 구현되는 차량에는 AVM 카메라 시스템이 탑재되며, 차량 주차보조 장치(200)는 AVM 카메라 시스템과의 연동을 통해 AVM 영상을 생성하는 데 있어서 필요한 관련 영상을 수집한다. AVM 카메라 시스템은 복수 개의 카메라(미도시)를 포함하며, 이에, 영상 획득부(310)는 복수 개의 카메라를 이용하여 촬영된 차량의 전방, 후방, 좌측방, 우측방에 대한 촬영 영상을 상기의 관련 영상으로서 제공받는다.The image acquisition unit 310 generates an AVM image by synthesizing the captured images collected from the plurality of cameras provided in the vehicle. On the other hand, the vehicle parking assistance apparatus 200 according to the present embodiment provides a parking assistance service using the AVM image. To this end, an AVM camera system is mounted on a vehicle in which the vehicle parking assist apparatus 200 is implemented, and the vehicle parking assist apparatus 200 collects related images necessary for generating an AVM image through interworking with the AVM camera system. do. The AVM camera system includes a plurality of cameras (not shown), so that the image acquisition unit 310 captures the captured images of the front, rear, left, and right rooms of the vehicle photographed using the plurality of cameras. It is provided as a related video.
영상 획득부(310)는 복수의 카메라로부터 수집한 촬영 영상을 탑뷰(Top View), 즉 버드아이뷰(Bird's Eye View)로 변환 및 합성하여 AVM 영상을 생성한다.다른 실시예에서, 차량 주차보조 장치(200)는 차량 내 탑재된 AVM 카메라 시스템으로부터 직접 AVM 영상을 제공받는 형태로 구현될 수 있다. 이 경우, 영상 획득부(310)는 AVM 카메라 시스템으로부터 제공받은 AVM 영상을 산출부(320)로 전달하는 역할을 수행한다.The image acquisition unit 310 generates an AVM image by converting and synthesizing the captured images collected from the plurality of cameras into a top view, that is, Bird's Eye View. In another embodiment, the vehicle parking assist The apparatus 200 may be implemented to receive an AVM image directly from an AVM camera system mounted in a vehicle. In this case, the image acquisition unit 310 serves to deliver the AVM image provided from the AVM camera system to the calculation unit 320.
산출부(320)는 영상 획득부(310)를 이용하여 제공되는 AVM 영상에서 차량의주차와 관련된 적어도 하나의 영상 특징점을 산출한다. 이하, 본 실시예에서는 산출부(320)가 산출하는 영상 특징점이 AVM 영상 내 주차선인 것으로 예시하여 설명하나 반드시 이에 한정되는 것은 아니다. 예컨대, 산출부(320)는 차량의 주변 영역에 주차 차량을 포함한 장애물이 존재하는 경우 AVM 영상 내 해당 장애물에 대응되는 객체를 영상 특징점으로써 추가로 산출할 수 있다.The calculator 320 calculates at least one image feature point associated with the parking of the vehicle from the AVM image provided by the image acquirer 310. Hereinafter, in the present exemplary embodiment, the image feature point calculated by the calculator 320 is illustrated as being a parking line in the AVM image, but is not necessarily limited thereto. For example, the calculator 320 may further calculate an object corresponding to the obstacle in the AVM image as an image feature point when there is an obstacle including a parking vehicle in the peripheral area of the vehicle.
산출부(320)는 AVM 영상의 휘도 평균값 및 휘도 분산값을 측정하여 AVM 영상 내 영상 특징점을 산출할 수 있으며, AVM 영상을 이미지 프로세싱하여 VM 영상 내 영상 특징점을 산출할 수도 있다. 본 실시예에서는 산출부(320)가 AVM 영상 내 영상 특징점을 산출하는 방법에 대하여 특정 방법으로 한정하지는 않는다.The calculator 320 may calculate an image feature point in the AVM image by measuring a luminance average value and a luminance dispersion value of the AVM image, and may calculate an image feature point in the VM image by image processing the AVM image. In the present exemplary embodiment, the method of calculating the image feature point in the AVM image is not limited to a specific method.
제어부(330)는 영상 획득부(310)를 이용하여 생성된 AVM 영상 및 산출부(320)를 이용하여 산출된 영상 특징점에 기반하여 차량의 주차를 보조하기 위한 주차 맵을 생성하여 제공한다.The controller 330 generates and provides a parking map for assisting the parking of the vehicle based on the AVM image generated using the image acquisition unit 310 and the image feature point calculated using the calculation unit 320.
이하, 제어부(330)가 차량의 주차를 보조하기 위한 주차 맵을 생성하는 방법에 대하여 설명하도록 한다.Hereinafter, a method of generating a parking map for assisting parking of the vehicle by the controller 330 will be described.
제어부(330)는 순차적으로 획득된 복수의 AVM 영상 각각에 대하여 상기의 영상 특징점을 서로 매칭하여 비교한다. 본 실시예의 경우, 제어부(330)는 복수의 AVM 영상의 영상 특징점들을 매칭하기 위한 방법으로써 스캔 매칭(Scan Matching) 기술을 이용할 수 있다. 스캔 매칭 기술은 현재 시점의 영상과 현재 시점의 바로 이전 시점의 영상을 비교하고, 비교결과에 따라 서로 간 겹쳐지는 부분에서의 일치점들을 찾아 매칭하는 기술이다. 본 실시예에 따른 제어부(330)는 차량의 이동에 의하여 달라지는 연속된 두 영상을 정규화하여 비교하기 위하여 상기의 스캔 매칭 기술을 사용한다. 스캔 매칭 기술은 공지의 기술로서 자세한 설명은 생략하도록 한다.The controller 330 matches and compares the image feature points with respect to each of a plurality of sequentially obtained AVM images. In the present embodiment, the controller 330 may use a scan matching technique as a method for matching image feature points of a plurality of AVM images. The scan matching technique compares an image of a current view with an image of a view immediately before the current view, and finds and matches matching points in overlapping portions according to a comparison result. The control unit 330 according to the present exemplary embodiment uses the scan matching technique described above to normalize and compare two consecutive images changed by the movement of the vehicle. The scan matching technique is a known technique and detailed description thereof will be omitted.
제어부(330)는 복수의 AVM 영상의 영상 특징점들의 매칭결과에 따라 영상 특징점의 이동 궤적을 파악하고, 파악된 이동 궤적에 따라 복수의 AVM 영상들을 정합하여 주차 맵을 생성한다. 한편, 본 실시예의 경우 영상 특징점으로서 AVM 영상 내 주차선이 추출된다. 이에, 제어부(330)는 복수의 AVM 영상의 영상 특징점들의 매칭결과에 따라 차량의 주변에 위치하는 주차선을 인식하고, 인식된 주차선을 기반으로 주차공간을 지시하는 주차 맵을 생성할 수 있다.The controller 330 determines a moving trajectory of the image feature point according to the matching result of the image feature points of the plurality of AVM images, and generates a parking map by matching the plurality of AVM images according to the identified moving trajectory. Meanwhile, in the present embodiment, the parking line in the AVM image is extracted as the image feature point. Accordingly, the controller 330 may recognize a parking line located near the vehicle according to the matching result of the image feature points of the plurality of AVM images, and generate a parking map indicating the parking space based on the recognized parking line. .
마찬가지로, 제어부(330)는 영상 특징점의 이동 궤적을 통해 차량의 이동 궤적을 추가로 파악하고, 이를 통해, 차량의 현재 위치를 추정하여 제공한다. 제어부(330)는 추정된 차량의 현재 위치를 주차 맵 상에 함께 도시하여 제공함으로써 운전자의 주차가 보다 효율적으로 지원 가능토록 동작한다.Similarly, the controller 330 further detects the movement trajectory of the vehicle through the movement trajectory of the image feature point, and estimates and provides the current position of the vehicle. The controller 330 shows the estimated current position of the vehicle together on the parking map and provides the driver's parking more efficiently.
제어부(330)는 복수의 AVM 영상의 영상 특징점 매칭결과에 따라 복수의 AVM 영상 간의 변위차 및 각도차를 보정하기 위한 파라미터값을 산출하고, 산출된 파라미터값에 기반하여 복수의 AVM 영상들을 정합한다. 이는, 차량의 이동 위치에 따라 발생할 수 있는 연속된 두 AVM 영상 사이의 오차를 보정하기 위함이며, 이를 통해, 본 실시예에 따른 제어부(330)는 주차 맵을 생성하는 과정에서 야기될 수 있는 오류 발생을 최소화할 수 있는 효과가 있다.The controller 330 calculates parameter values for correcting displacement and angle differences between the plurality of AVM images according to image feature point matching results of the plurality of AVM images, and matches the plurality of AVM images based on the calculated parameter values. . This is to correct an error between two consecutive AVM images that may occur according to the moving position of the vehicle. Through this, the controller 330 according to the present embodiment may cause an error that may occur in the process of generating a parking map. There is an effect that can minimize the occurrence.
본 실시예에 따른 제어부(330)는 주차 맵을 생성하는 과정에서 주차선의 종류별 기 정의된 주차선 패턴정보를 추가로 이용할 수 있다. 주차선 패턴정보는 특정 주차선에 대하여 다른 주차선과의 구분이 가능하도록 해당 주차선의 규격을 정의한 정보로서 주차선의 코너 형태 및 라인 형태 등이 포함될 수 있다.The controller 330 according to the present exemplary embodiment may additionally use predetermined parking line pattern information for each type of parking line in the process of generating the parking map. The parking line pattern information is information defining a standard of the corresponding parking line so that the specific parking line can be distinguished from other parking lines, and may include a corner shape and a line shape of the parking line.
한편, 본 실시예에 있어서, 제어부(330)를 통해 인식되는 주차선은 차량 또는 장애물에 의해 주차선 중 일부가 인식되지 않을 수 있으며, AVM 카메라 시스템의 시야각(Field of View)을 벗어나는 주차선의 일부 영역 또한 인식이 되지 않을 수 있다. 이 경우, 주차 맵은 불완전한 주차선에 기반으로 한 주차공간을 제공할 수 밖에 없으며, 이는 곧, 차량 주차보조 장치(200)를 이용한 차량의 주차 보조 기능에 대하여 그 정확성이 떨어질 수 있다는 문제점이 존재한다. Meanwhile, in the present embodiment, the parking line recognized by the controller 330 may not be recognized by a vehicle or an obstacle, and part of the parking line may be out of the field of view of the AVM camera system. Areas may also not be recognized. In this case, the parking map must provide a parking space based on an incomplete parking line, which is a problem that the accuracy of the parking assistance function of the vehicle using the vehicle parking assistance device 200 may be degraded. do.
이 점을 고려하여, 본 실시예에 따른 제어부(330)는 복수의 AVM 영상의 영상 특징점 매칭결과에 따라 인식된 주차선이 불완전(Partial) 주차선인 경우 상기의 주차선 패턴정보를 이용하여 해당 주차선을 보정하고, 보정된 주차선을 기반으로 주차 맵을 생성한다. 이때, 불완전 주차선은 제어부(330)에 의해 인식된 주차선 중 전체 영역이 아닌 일부 영역만이 인식된 주차선을 의미한다In consideration of this, the controller 330 according to the present embodiment uses the parking line pattern information as described above when the parking line recognized according to the image feature point matching result of the plurality of AVM images is a partial parking line. Correct the line and generate a parking map based on the corrected parking line. In this case, the incomplete parking line means a parking line in which only a part of the parking line recognized by the controller 330 is recognized, not the entire area.
제어부(330)는 인식된 불완전 주차선의 일부 영역으로부터 해당 주차선의 특징을 산출한다. 이때, 해당 주차선의 특징은 불완전 주차선의 코너 부분의 형상인 것이 바람직하나 반드시 이에 한정되는 것은 아니다.The controller 330 calculates the characteristics of the parking line from the partial region of the recognized incomplete parking line. At this time, the characteristic of the parking line is preferably the shape of the corner portion of the incomplete parking line, but is not necessarily limited thereto.
제어부(330)는 주차선의 종류별 기 정의된 주차선 패턴정보 중 상기의 주차선 특징에 대응되는 주차선 패턴정보를 산출하고, 산출된 주차선 패턴정보를 이용하여 불완전 주차선의 나머진 영역의 일부 또는 전부를 예측한다.The controller 330 calculates parking line pattern information corresponding to the above-mentioned characteristic of the parking line from among the predetermined parking line pattern information for each type of parking line, and uses the calculated parking line pattern information to partially or all the remaining areas of the incomplete parking line. Predict.
제어부(330)는 예측된 나머지 영역을 기존에 인식된 불완전 주차선의 일부 영역에 병합시켜 불완전 주차선을 보정하고, 보정된 주차선을 기반으로 주차 맵을 생성한다.The controller 330 corrects the incomplete parking line by merging the predicted remaining area into a partial area of the previously recognized incomplete parking line, and generates a parking map based on the corrected parking line.
실시예에 따라 제어부(330)는 차량 주자보조 장치(200)의 다른 구성수단으로부터 생성된 정보를 주차 맵 상에 반영할 수 있다.According to an exemplary embodiment, the controller 330 may reflect information generated from other constituent means of the vehicle runner assisting device 200 on the parking map.
저장부(340)는 본 실시예에 따른 차량 주차보조 장치(200)가 차량의 주차 보조 기능을 수행함에 있어서 필요한 정보들을 저장한다. 예컨대, 저장부(340)는 주차선의 종류별 기 정의된 주차선 패턴정보를 저장할 수 있다. 이때, 저장부(340)는 상기의 주차선 패턴정보를 탬플릿 및 학습 데이터 중 어느 하나의 형태로 저장할 수 있다.The storage unit 340 stores information necessary for the vehicle parking assistance apparatus 200 according to the present embodiment to perform the parking assistance function of the vehicle. For example, the storage unit 340 may store the predetermined parking line pattern information for each type of parking line. In this case, the storage unit 340 may store the parking line pattern information in any one of a template and learning data.
저장부(340)의 차량의 종류별 기 정의된 차량 형태정보를 저장할 수도 있다. 차량 형태정보는 차량의 모양 및 크기에 대한 정보가 포함될 수 있다.Pre-defined vehicle type information for each type of vehicle in the storage unit 340 may be stored. Vehicle shape information may include information about the shape and size of the vehicle.
탐색부(350)는 제어부(330)를 이용하여 생성된 주차 맵에 기반하여 차량의 주변 영역에 위치하는 주차공간 중 실제 차량의 주차가 가능한 주차 가능공간을 탐색하여 제공한다. 탐색부(350)는 주차 맵을 이용하여 차량의 현재 위치와 주차 공간 사이의 상대적 위치 및 주차공간 내 장애물의 존재 여부를 확인하고, 확인결과에 따라 주차공간 중 실제 차량의 주차가 가능한 주차 가능공간을 탐색할 수 있다. 본 실시예에서는 탐색부(350)가 차량의 주차 가능공간을 탐색하는 방법에 대해서는 특정 방법으로 한정하지는 않는다. 예컨대, 탐색부(350)는 차량에 구비된 초음파 센서로부터 수집된 초음파 센싱정보를 추가로 활용하여 차량의 주차 가능공간을 탐색할 수도 있다.The search unit 350 searches for and provides a parking available space of an actual vehicle among the parking spaces located in the peripheral area of the vehicle based on the parking map generated using the controller 330. The search unit 350 checks the relative position between the current position of the vehicle and the parking space and the presence of obstacles in the parking space by using the parking map, and the parking space where the actual vehicle can be parked among the parking spaces according to the check result. You can explore In the present embodiment, a method of searching for a parking space of a vehicle by the search unit 350 is not limited to a specific method. For example, the search unit 350 may search for a parking space of the vehicle by further utilizing ultrasonic sensing information collected from the ultrasonic sensor provided in the vehicle.
본 실시예에 있어서 탐색부(350)는 주차선 패턴정보를 이용한 보정절차를 거쳐 그 정확성이 보장된 주차 맵을 이용하여 차량의 주차 가능공간을 탐색함에 따라 탐색된 주차 가능공간에 대한 신뢰성 또한 보장할 수 있도록 하는 효과가 있다.In this embodiment, the search unit 350 assures the reliability of the searched parking space by searching for a parking space of the vehicle using a parking map whose accuracy is guaranteed through a calibration procedure using the parking line pattern information. It has the effect of making it possible.
탐색부(350)는 탐색된 주차 가능공간을 제어부(330)로 전달하여 상기의 주차 가능공간이 주차 맵 상에 추가로 표시될 수 있도록 동작한다. 실시예에 따라 탐색부(350)는 탐색된 주차 가능공간으로의 차량의 최적 주차 경로를 산출하고, 이를 추가로 제공할 수도 있다.The search unit 350 transmits the searched parking space to the controller 330 so that the parking space can be additionally displayed on the parking map. According to an exemplary embodiment, the search unit 350 may calculate an optimal parking path of the vehicle to the searched parking space and additionally provide the same.
자동주차 지원부(360)는 차량의 조향 시스템과의 연동을 통해 차량의 자동 주차 지원기능을 보조하는 역할을 수행한다. 본 실시예에 따른 자동주차 지원부(360)는 제어부(330)를 이용하여 생성된 주차 맵 및 탐색부(350)를 이용하여 산출된 최적 주차 경로에 기반하여 차량의 조향 수단을 제어하기 위한 제어신호를 생성하고, 생성된 제어신호를 조향 수단으로 전달한다. 이를 위해, 자동주차 지원부(360)는 사전에 차량 내 구비된 각 센서들로부터 차량 주행 정보를 수집할 수 있다.The automatic parking support unit 360 serves to assist the automatic parking support function of the vehicle through linkage with the steering system of the vehicle. The automatic parking support unit 360 according to the present embodiment controls a control signal for controlling the vehicle steering means based on the parking map generated using the controller 330 and the optimum parking path calculated using the search unit 350. Generates and transmits the generated control signal to the steering means. To this end, the automatic parking assistance unit 360 may collect vehicle driving information from each sensor provided in the vehicle in advance.
디스플레이부(370)는 제어부(210)로부터 생성된 주차 맵을 수신하여 표시한다. 디스플레이부(370)는 주차 맵을 표시하기 위한 적어도 하나의 디스플레이 수단을 구비하며, 이러한, 디스플레이 수단은 차량에 장착된 AVN(Audio Visual Navigation)일 수 있다.The display 370 receives and displays the parking map generated from the controller 210. The display unit 370 includes at least one display means for displaying a parking map. The display means may be an audio visual navigation (AVN) mounted in a vehicle.
주차상태 알림부(380)는 차량의 주차가 완료된 경우 주차 맵 상에 표시된 주차선 및 차량의 현재 위치에 기반하여 차량의 주차 상태를 확인하고, 확인된 차량의 주차 상태를 사용자에게 인지 가능한 형태로 출력한다. 본 실시예에 따른 주차상태 알림부(380)는 주차선 패턴정보를 이용한 보정절차를 거쳐 그 정확성이 보장된 주차 맵을 이용하여 차량의 주차 상태를 판단함에 따라 운전자로 하여금 차량의 정확한 주차 상태를 확인 가능토록 하는 효과가 있다.When the parking of the vehicle is completed, the parking state notification unit 380 checks the parking state of the vehicle based on the parking line displayed on the parking map and the current position of the vehicle, and recognizes the confirmed parking state of the vehicle in a form that can be recognized by the user. Output The parking state notification unit 380 according to the present embodiment determines the parking state of the vehicle by using the parking map whose accuracy is guaranteed through the calibration procedure using the parking line pattern information, thereby allowing the driver to determine the exact parking state of the vehicle. It is effective to confirm.
도 4는 본 실시예에 따른 차량 주자보조 장치의 주차보조 방법을 설명하기 위한 순서도이다.4 is a flowchart illustrating a parking assisting method of the vehicle runner assisting apparatus according to the present embodiment.
차량 주차보조 장치(200)는 차량에 구비된 복수 개의 카메라로부터 수집된 촬영 영상을 합성하여 AVM 영상을 획득한다(S402).The vehicle parking assist apparatus 200 obtains an AVM image by synthesizing the captured images collected from the plurality of cameras provided in the vehicle (S402).
차량 주차보조 장치(200)는 단계 S402에서 획득한 AVM 영상에서 차량의 주차와 관련된 적어도 하나의 영상 특징점을 산출한다(S404). 단계 S404에서 차량 주차보조 장치(200)는 영상 특징점으로서 AVM 영상 내 주차선을 산출한다.The vehicle parking assist apparatus 200 calculates at least one image feature point related to the parking of the vehicle from the AVM image acquired in step S402 (S404). In operation S404, the vehicle parking assisting apparatus 200 calculates a parking line in the AVM image as an image feature point.
차량 주차보조 장치(200)는 순차적으로 획득된 복수의 AVM 영상에 대하여 단계 S404에서 산출한 영상 특징점을 매칭한다(S406).The vehicle parking assist apparatus 200 matches the image feature points calculated in step S404 with respect to the plurality of AVM images obtained sequentially (S406).
차량 주차보조 장치(200)는 단계 S406의 매칭결과에 따라 차량의 주변 영역게 위치하는 주차공간 및 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력한다(S408). 단계 S408에서 차량 주차보조 장치(200)는 단계 S406의 매칭결과에 따라 차량의 주변에 위치하는 주차선을 인식하고, 인식된 주차선을 기반으로 주차공간을 지시하는 주차 맵을 생성할 수 있다.The vehicle parking assistance apparatus 200 generates and outputs a parking space indicating a parking space located in a peripheral area of the vehicle and a current map of the vehicle according to the matching result of step S406 (S408). In operation S408, the vehicle parking assisting apparatus 200 may recognize a parking line located near the vehicle according to the matching result of step S406, and generate a parking map indicating the parking space based on the recognized parking line.
단계 S408에서 차량 주차보조 장치(200)는 단계 S406의 매칭결과에 따라 인식된 주차선이 불완전 주차선인 경우 기 저장된 주차선 패턴정보를 이용하여 해당 주차선을 보정하고, 보정된 주차선을 기반으로 주차 맵을 생성한다.In operation S408, the vehicle parking assisting device 200 corrects the parking line using previously stored parking line pattern information when the parking line recognized according to the matching result of step S406 is an incomplete parking line, and based on the corrected parking line. Create a parking map.
차량 주차보조 장치(200)는 단계 S408에서 생성한 주차 맵에 기반하여 차량의 주변 영역에 위치하는 주차공간 내 실제 차량의 주차가 가능한 주차 가능공간을 탐색하고, 탐색결과 및 주차 가능공간으로의 최적 주차 경로를 산출하여 제공한다(S410).The vehicle parking assist apparatus 200 searches for a parking available space of an actual vehicle in the parking space located in the surrounding area of the vehicle based on the parking map generated in step S408, and optimizes the search result and the available parking space. The parking route is calculated and provided (S410).
차량 주차보조 장치(200)는 차량에서 자동 주차 기능이 선택되었는지 여부를 확인하고(S412), 자동 주차 기능이 선택된 경우 단계 S408의 주차 맵 및 단계 S410의 최적 주차 경로에 기반하여 차량의 조향 수단을 제어하기 위한 제어신호를 생성하여 출력한다(S412).The vehicle parking assist apparatus 200 checks whether the automatic parking function is selected in the vehicle (S412), and when the automatic parking function is selected, the vehicle parking assist apparatus 200 adjusts the steering means of the vehicle based on the parking map of step S408 and the optimal parking path of step S410. A control signal for controlling is generated and output (S412).
도 4에서는 각각의 과정을 순차적으로 실행하는 것으로 기재하고 있으나, 반드시 이에 한정되는 것은 아니다. 다시 말해, 도 4에 기재된 과정을 변경하여 실행하거나 하나 이상의 과정을 병렬적으로 실행하는 것으로 적용 가능할 것이므로, 도 4는 시계열적인 순서로 한정되는 것은 아니다.In FIG. 4, each process is described as being sequentially executed, but is not necessarily limited thereto. In other words, since the process described in FIG. 4 may be applied by changing or executing one or more processes in parallel, FIG. 4 is not limited to the time series order.
도 5는 본 실시예에 따른 주차 맵의 구현 형태를 예시한 예시도이며, 도 6은 본 실시예에 따른 주차 맵의 생성과정을 예시한 예시도이다.5 is an exemplary diagram illustrating an embodiment of a parking map according to the present embodiment, and FIG. 6 is an exemplary diagram illustrating a process of generating a parking map according to the present embodiment.
본 실시예에 따른 차량 주차보조 장치(200)는 복수의 AVM 영상의 특징점을 매칭한 매칭결과에 따라 차량의 주변에 위치하는 주차선을 인식하고, 인식된 주차선을 기반으로 주차공간을 지시하는 주차 맵을 생성하여 제공한다.The vehicle parking assisting apparatus 200 according to the present embodiment recognizes a parking line located near the vehicle according to a matching result of matching the feature points of the plurality of AVM images, and indicates a parking space based on the recognized parking line. Create and provide a parking map.
이와 더불어, 본 실시예에 따른 차량 주차보조 장치(200)는 인식된 주차선이 불완전 주차선인 경우 기 저장된 주차선 패턴정보를 이용하여 해당 주차선을 보정하고, 보정된 주차선을 기반으로 주차 맵을 생성한다.In addition, the vehicle parking assisting apparatus 200 according to the present embodiment corrects the parking line using previously stored parking line pattern information when the recognized parking line is an incomplete parking line, and based on the corrected parking line, the parking map. Create
도 5 및 도 6을 참조하면, 차량 주차보조 장치(200)가 인식된 주차선을 기반으로 주차공간을 지시하는 주차 맵을 생성하여 제공하되, 불완전 주차선에 대해서는 기 저장된 주차선 패턴정보를 이용하여 그 형태를 보정(점선 영역)하여 제공하는 것을 확인할 수 있다.5 and 6, the vehicle parking assist apparatus 200 generates and provides a parking map indicating a parking space based on the recognized parking line, but uses previously stored parking line pattern information for an incomplete parking line. To correct the shape (dotted line area).
도 7은 본 실시예에 따른 주차선의 종류별 기 정의된 주차선 패턴정보를 예시한 예시도이다.7 is an exemplary view illustrating predefined parking line pattern information for each type of parking line according to the present embodiment.
본 실시예에 따른 차량 주차보조 장치(200)는 주차 맵을 생성하는 과정에서 인식된 주차선이 불완전 주차선인 경우 주차선의 종류별 기 정의된 주차선 패턴정보를 이용하여 해당 주차선의 전반전인 모양 및 크기를 예측한다. 이후, 차량 주차보조 장치(200)는 예측결과에 기반하여 불완전 주차선을 보정하고, 보정된 주차선을 기반으로 주차 맵을 생성한다.In the vehicle parking assist apparatus 200 according to the present embodiment, when the parking line recognized in the process of generating the parking map is an incomplete parking line, the shape and size of the first half of the corresponding parking line using the predetermined parking line pattern information for each type of parking line Predict. Thereafter, the vehicle parking assistance apparatus 200 corrects the incomplete parking line based on the prediction result, and generates a parking map based on the corrected parking line.
도 7을 참조하면, 본 실시예에 따른 주차선 패턴정보가 현재 사용되고 있는 주차선들에 대한 고유의 특징 예컨대, 코너 형태 및 라인 형태 등을 각각 정의하여 제공하는 것을 확인할 수 있다.Referring to FIG. 7, it can be seen that the parking line pattern information according to the present embodiment defines and provides unique features, for example, a corner shape and a line shape, for the parking lines currently being used.
도 8은 본 실시예에 따른 차량 주차보조 장치에 의한 주차보조 방법을 설명하기 위한 예시도이다.8 is an exemplary view for explaining a parking assist method by the vehicle parking assist apparatus according to the present embodiment.
도 8 (a)을 참조하면, 본 실시예에 따른 차량 주차보조 장치(200)가 주차 맵에 기반하여 차량의 주변 영역에 위치하는 주차공간 내 실제 차량의 주차가 가능한 주차 가능공간을 탐색하고, 이를 주차 맵 상에 표시하여 제공함으로써 운전자의 주차를 지원하는 것을 확인할 수 있다.Referring to FIG. 8 (a), the vehicle parking assisting apparatus 200 according to the present embodiment searches for a parking available space of an actual vehicle in a parking space located in a peripheral area of the vehicle based on a parking map. By displaying this on the parking map, it can be confirmed that the driver supports the parking.
한편, 본 실시예에 따른 차량 주차보조 장치(200)는 주차 맵에 기반하여 차량의 현채 위치를 추가로 산출하여 제공할 수 있으며, 이를 이용하여 앞서 탐색된 주차 가능공간으로의 차량의 주차 경로를 산출하여 제공할 수도 있다.On the other hand, the vehicle parking assisting apparatus 200 according to the present embodiment may additionally calculate and provide the current position of the vehicle based on the parking map, by using the parking path of the vehicle to the previously available parking space using the same It can also calculate and provide.
도 8의 (b)를 참조하면, 차량 주차보조 장치(200)가 산출된 주차 경로 및 주차 맵에 기반하여 차량이 주차 가능공간에 자동 주차될 수 있도록 지원하는 것을 확인할 수 있다.Referring to FIG. 8 (b), it can be seen that the vehicle parking assistance apparatus 200 supports the vehicle to be parked automatically in the available parking space based on the calculated parking path and the parking map.
도 9는 본 실시예에 따른 차량 주차보조 장치에 의한 주차상태 알림 방법을 설명하기 위한 예시도이다.9 is an exemplary view for explaining a parking state notification method by the vehicle parking assisting apparatus according to the present embodiment.
도 9를 참조하면, 본 실시예에 따른 차량 주차보조 장치(200)에 의하는 경우 차량의 주차 완료 시 주차 맵 상에 표시된 주차선 및 차량의 현재 위치에 기반하여 차량의 주차 상태를 확인 가능한 것을 알 수 있다. 더욱이, 본 실시예에 따른 차량 주차보조 장치(200)는 주차선 패턴정보를 이용한 보정절차를 거쳐 정확성이 보장된 주차 맵을 이용하여 차량의 주차 상태를 확인 가능토록 동작함으로써 운전자로 하여금 그 결과에 대한 신뢰성 또한 보장될 수 있도록 하는 효과가 있다.Referring to FIG. 9, in the case of the vehicle parking assisting apparatus 200 according to the present embodiment, it is possible to check the parking state of the vehicle based on the parking line displayed on the parking map and the current position of the vehicle when the parking of the vehicle is completed. Able to know. Furthermore, the vehicle parking assisting apparatus 200 according to the present embodiment operates to enable the driver to check the parking state of the vehicle by using the parking map whose accuracy is guaranteed through the correction procedure using the parking line pattern information. Reliability can also be ensured.
이상의 설명은 본 실시예의 기술 사상을 예시적으로 설명한 것에 불과한 것으로서, 본 실시예가 속하는 기술 분야에서 통상의 지식을 가진 자라면 본 실시예의 본질적인 특성에서 벗어나지 않는 범위에서 다양한 수정 및 변형이 가능할 것이다. 따라서, 본 실시예들은 본 실시예의 기술 사상을 한정하기 위한 것이 아니라 설명하기 위한 것이고, 이러한 실시예에 의하여 본 실시예의 기술 사상의 범위가 한정되는 것은 아니다. 본 실시예의 보호 범위는 아래의 청구범위에 의하여 해석되어야 하며, 그와 동등한 범위 내에 있는 모든 기술 사상은 본 실시예의 권리범위에 포함되는 것으로 해석되어야 할 것이다.The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to describe the present invention, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.
(부호의 설명)(Explanation of the sign)
200: 차량 주차보조 장치 300: 통신부200: vehicle parking assist device 300: communication unit
310: 영상 획득부 320: 산출부310: image acquisition unit 320: calculator
330: 제어부 340: 저장부330: control unit 340: storage unit
350: 탐색부 360: 자동주차 지원부350: search unit 360: automatic parking support unit
370: 디스플레이부 380: 주차상태 알림부370: display unit 380: parking state notification unit

Claims (14)

  1. AVM 영상을 활용하여 차량의 주차를 보조하는 차량 주차보조 장치에 있어서,In the vehicle parking assist device that assists the parking of the vehicle using the AVM image,
    상기 차량에 구비된 복수 개의 카메라로부터 수집한 촬영 영상을 합성하여 AVM(Around View Monitoring) 영상을 획득하는 영상 획득부;An image acquisition unit for synthesizing captured images collected from a plurality of cameras provided in the vehicle to obtain an AVM (Around View Monitoring) image;
    상기 AVM 영상에서 상기 차량의 주차와 관련된 적어도 하나의 영상 특징점을 산출하는 산출부; 및A calculator configured to calculate at least one image feature point associated with parking of the vehicle from the AVM image; And
    순차적으로 획득된 복수의 AVM 영상에 대하여 상기 영상 특징점을 매칭하고, 매칭결과에 따라 상기 차량의 주변 영역에 위치하는 주차공간 및 상기 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력하는 제어부A controller for matching the image feature points to a plurality of sequentially acquired AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result
    를 포함하는 것을 특징으로 하는 차량 주차보조 장치.Vehicle parking assist device comprising a.
  2. 제 1항에 있어서,The method of claim 1,
    상기 제어부는,The control unit,
    상기 매칭결과에 따라 상기 영상 특징점의 이동 궤적을 파악하고, 상기 이동 궤적에 따라 상기 복수의 AVM 영상들을 정합하여 상기 주차 맵을 생성하는 것을 특징으로 하는 차량 주차보조 장치.And a moving map of the image feature point based on the matching result, and matching the plurality of AVM images according to the moving trajectory to generate the parking map.
  3. 제 2항에 있어서,The method of claim 2,
    상기 제어부는,The control unit,
    상기 매칭결과에 따라 상기 복수의 AVM 영상 간의 변위차 및 각도차를 보정하기 위한 파라미터값을 산출하고, 상기 파라미터값에 기반하여 상기 복수의 AVM 영상들을 정합하는 것을 특징으로 하는 차량 주차보조 장치.And a parameter value for correcting a displacement difference and an angle difference between the plurality of AVM images according to the matching result, and matching the plurality of AVM images based on the parameter value.
  4. 제 1항에 있어서,The method of claim 1,
    상기 산출부는, 상기 영상 특징점으로서 상기 AVM 영상 내 주차선을 산출하며,The calculating unit calculates a parking line in the AVM image as the image feature point,
    상기 제어부는, 상기 매칭결과에 따라 상기 차량의 주변에 위치하는 상기 주차선을 인식하고, 인식된 주차선을 기반으로 상기 주차공간을 지시하는 상기 주차 맵을 생성하는 것을 특징으로 차량 주차보조 장치.And the control unit recognizes the parking line located around the vehicle according to the matching result and generates the parking map indicating the parking space based on the recognized parking line.
  5. 제 4항에 있어서,The method of claim 4, wherein
    상기 주차선의 종류별로 코너 형태 및 라인 형태를 정의한 주차선 패턴정보를 템플릿 및 학습 데이터 중 어느 하나의 형태로 저장하는 저장부를 더 포함하며,Further comprising a storage unit for storing the parking line pattern information defining the corner shape and the line shape for each type of the parking line in any one form of a template and learning data,
    상기 제어부는, 상기 주차선의 종류별 기 정의된 주차선 패턴정보를 추가로 이용하여 상기 주차 맵을 생성하는 것을 특징으로 하는 차량 주차보조 장치.The control unit, the vehicle parking assist device, characterized in that for generating the parking map by further using the predetermined parking line pattern information for each type of the parking line.
  6. 제 5항에 있어서,The method of claim 5,
    상기 제어부는, The control unit,
    상기 매칭결과에 따라 인식된 주차선이 전체 영역이 아닌 일부 영역만이 인식된 불완전 주차선인 경우 상기 저장부에 저장된 상기 주차선 패턴정보를 이용하여 상기 불완전 주차선을 보정하고, 보정된 주차선을 기반으로 상기 주차 맵을 생성하는 것을 특징으로 하는 차량 주차보조 장치.When the parking line recognized according to the matching result is an incomplete parking line in which only a part of the area is recognized instead of the entire area, the incomplete parking line is corrected using the parking line pattern information stored in the storage unit, and the corrected parking line is corrected. Vehicle parking assistance device, characterized in that for generating the parking map based.
  7. 제 6항에 있어서,The method of claim 6,
    상기 제어부는,The control unit,
    상기 일부 영역에서 상기 불완전 주차선의 특징을 산출하고, 산출된 특징에 대응되는 상기 저장부 내 주차선 패턴정보를 이용하여 상기 일부 영역을 제외한 나머지 영역의 일부 또는 전부를 예측하고, 예측된 나머지 영역을 상기 일부 영역에 병합시켜 상기 불완전 주차선을 보정하는 것을 특징으로 하는 차량 주차보조 장치.Computing a feature of the incomplete parking line in the partial region, using the parking line pattern information in the storage unit corresponding to the calculated feature to predict some or all of the remaining region except for the partial region, and predict the remaining region Vehicle parking assist device, characterized in that for correcting the incomplete parking line by merging in the partial area.
  8. 제 4항에 있어서,The method of claim 4, wherein
    상기 차량의 주차가 완료된 경우 상기 매칭결과에 따라 인식된 상기 주차선의 인식결과 및 상기 차량의 현재 위치에 기반하여 상기 차량의 주차 상태를 확인하고, 상기 차량의 주차 상태에 대한 확인결과를 사용자에게 인지 가능한 형태로 출력하는 주차상태 알림부를 더 포함하는 것을 특징으로 하는 차량 주차보조 장치.When the parking of the vehicle is completed, the parking state of the vehicle is checked based on the recognition result of the parking line recognized according to the matching result and the current position of the vehicle, and the user is aware of the result of checking the parking state of the vehicle Vehicle parking assist device further comprises a parking state notification unit for outputting in a possible form.
  9. 제 1항에 있어서,The method of claim 1,
    상기 주차 맵에 기반하여 상기 주차공간 내 실제 상기 차량의 주차가 가능한 주차 가능공간을 탐색하고, 탐색결과 및 상기 주차 가능공간으로의 상기 차량의 최적 주차 경로를 산출하여 제공하는 탐색부를 더 포함하는 것을 특징으로 하는 차량 주차보조 장치.And a search unit for searching for a parking available space of the actual parking of the vehicle in the parking space based on the parking map, and calculating and providing a search result and an optimal parking path of the vehicle to the available parking space. Vehicle parking assist device.
  10. 제 9항에 있어서,The method of claim 9,
    상기 탐색부는,The search unit,
    상기 차량에 구비된 초음파 센서로부터 수집된 초음파 센싱정보를 추가로 활용하여 상기 주차 가능공간을 탐색하는 것을 특징으로 하는 차량 주차보조 장치.Vehicle parking assistance device, characterized in that for further exploring the available parking space by further utilizing the ultrasonic sensing information collected from the ultrasonic sensor provided in the vehicle.
  11. 제 9항에 있어서,The method of claim 9,
    상기 주차 맵 및 상기 최적 주차 경로에 기반하여 상기 차량의 조향 수단을 제어하기 위한 제어신호를 생성하고, 상기 제어신호를 상기 조향 수단으로 전달하여 상기 차량이 상기 주차 가능공간에 자동 주차될 수 있도록 지원하는 자동주차 지원부를 더 포함하는 것을 특징으로 하는 차량 주차보조 장치.Generate a control signal for controlling the steering means of the vehicle based on the parking map and the optimal parking path, and transmit the control signal to the steering means so that the vehicle can be automatically parked in the parking space. Vehicle parking assistance device further comprises an automatic parking support.
  12. 차량 주차보조 장치가 AVM 영상을 활용하여 차량의 주차를 보조하는 방법에 있어서,In the vehicle parking assist device to assist the parking of the vehicle using the AVM image,
    상기 차량에 구비된 복수 개의 카메라로부터 수집한 촬영 영상을 합성하여 AVM 영상을 획득하는 과정;Obtaining an AVM image by synthesizing a captured image collected from a plurality of cameras provided in the vehicle;
    상기 AVM 영상에서 상기 차량의 주차와 관련된 적어도 하나의 영상 특징점을 산출하는 과정; 및Calculating at least one image feature point associated with parking of the vehicle from the AVM image; And
    순차적으로 획득된 복수의 AVM 영상에 대하여 상기 영상 특징점을 매칭하고, 매칭결과에 따라 상기 차량의 주변 영역에 위치하는 주차공간 및 상기 차량의 현재 위치를 지시하는 주차 맵을 생성하여 출력하는 과정Matching the image feature points to a plurality of sequentially obtained AVM images, and generating and outputting a parking space located in a peripheral area of the vehicle and a parking map indicating a current position of the vehicle according to a matching result
    을 포함하는 것을 특징으로 하는 차량 주차보조 방법.Vehicle parking assistance method comprising a.
  13. 제 12항에 있어서,The method of claim 12,
    상기 산출하는 과정은, 상기 영상 특징점으로서 상기 AVM 영상 내 주차선을 산출하며,The calculating may include calculating a parking line in the AVM image as the image feature point.
    상기 출력하는 과정은, 상기 매칭결과에 따라 상기 차량의 주변에 위치하는 상기 주차선을 인식하고, 인식된 주차선을 기반으로 상기 주차공간을 지시하는 상기 주차 맵을 생성하는 것을 특징으로 하는 차량 주차보조 방법.The outputting process may include: recognizing the parking line located around the vehicle according to the matching result, and generating the parking map indicating the parking space based on the recognized parking line. Secondary method.
  14. 제 13항에 있어서,The method of claim 13,
    상기 출력하는 과정은,The process of outputting,
    상기 주차선의 인식결과에 따라, 상기 주차선을 상기 주차선의 종류별 코너 형태 및 라인 형태를 정의한 주차선 패턴정보를 이용하여 보정하고, 보정된 주차선에 기반하여 상기 주차 맵을 생성하는 것을 특징으로 하는 차량 주차보조 방법.According to the recognition result of the parking line, the parking line is corrected by using the parking line pattern information defining the corner shape and line shape for each type of the parking line, and generates the parking map based on the corrected parking line. Vehicle parking assistance.
PCT/KR2016/013633 2016-11-24 2016-11-24 Vehicle parking assist device and method therefor WO2018097356A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/KR2016/013633 WO2018097356A1 (en) 2016-11-24 2016-11-24 Vehicle parking assist device and method therefor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2016/013633 WO2018097356A1 (en) 2016-11-24 2016-11-24 Vehicle parking assist device and method therefor

Publications (1)

Publication Number Publication Date
WO2018097356A1 true WO2018097356A1 (en) 2018-05-31

Family

ID=62196201

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2016/013633 WO2018097356A1 (en) 2016-11-24 2016-11-24 Vehicle parking assist device and method therefor

Country Status (1)

Country Link
WO (1) WO2018097356A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006096312A (en) * 2004-09-30 2006-04-13 Clarion Co Ltd Parking support system
US20110210868A1 (en) * 2009-08-31 2011-09-01 Katsutoshi Yano Parking assistance apparatus
KR20150017095A (en) * 2013-08-06 2015-02-16 현대자동차주식회사 Apparatus and method for providing image
KR101498976B1 (en) * 2013-12-19 2015-03-05 현대모비스(주) Parking asistance system and parking asistance method for vehicle
US20150078624A1 (en) * 2012-03-30 2015-03-19 Panasonic Corporation Parking assistance device and parking assistance method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006096312A (en) * 2004-09-30 2006-04-13 Clarion Co Ltd Parking support system
US20110210868A1 (en) * 2009-08-31 2011-09-01 Katsutoshi Yano Parking assistance apparatus
US20150078624A1 (en) * 2012-03-30 2015-03-19 Panasonic Corporation Parking assistance device and parking assistance method
KR20150017095A (en) * 2013-08-06 2015-02-16 현대자동차주식회사 Apparatus and method for providing image
KR101498976B1 (en) * 2013-12-19 2015-03-05 현대모비스(주) Parking asistance system and parking asistance method for vehicle

Similar Documents

Publication Publication Date Title
WO2014163307A1 (en) Automatic driving system for vehicle
WO2010134680A1 (en) Lane departure sensing method and apparatus using images that surround a vehicle
JP5523448B2 (en) Driving support system, information display device, and information display program
JP6257989B2 (en) Driving assistance device
WO2013018962A1 (en) Traffic lane recognizing apparatus and method thereof
WO2015102275A1 (en) Traffic lane change warning apparatus
KR100917330B1 (en) Top view monitor system and method of vehicle
JP2005290813A (en) Parking guidance robot
KR20150044874A (en) Depth based context identification
WO2011093604A2 (en) Vehicle navigation and method for inducing normal black box position in navigation system
JP2010030427A (en) Parking support system and parking support device
WO2017188523A1 (en) Traffic information big data operation system using recognition of license plate of transportation means, server therefor, and user terminal therefor
WO2014027478A1 (en) Road environment recognition device
JP2010237792A (en) Driving assist device, driving assist method, and driving assist program
JP2007153031A (en) Vehicle position calculation method and onboard device
US20170180701A1 (en) Information processing system
JPH11351895A (en) Car navigation system
EP3470780A1 (en) Object distance detection device
WO2017195965A1 (en) Apparatus and method for image processing according to vehicle speed
US9984296B2 (en) Misaligned tire detection method and apparatus
JPH10177699A (en) Voice guidance device for automobile utilizing image information
JP4848644B2 (en) Obstacle recognition system
WO2017095116A1 (en) Navigation device and path guidance method thereof
WO2019132504A1 (en) Destination guide apparatus and method
WO2021194109A1 (en) Method, device, and computer program for providing driving guide by using vehicle position information and signal light information

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16922262

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16922262

Country of ref document: EP

Kind code of ref document: A1