KR20210072213A - Apparatus for controlling a parking, system having the same and method thereof - Google Patents

Abstract

The present invention relates to a parking control apparatus, a system including the same, and a method therefor. The parking control apparatus according to one embodiment of the present invention may comprise: a processor for learning a parking lot environment based on image data and generating a parking lot map, and controlling a vehicle to automatically search and park a parking space based on the learned parking lot map; and a storage unit for storing the map. An embodiment of the present invention is to provide the parking control apparatus, the system including the same, and the method therefor, which are capable of learning the parking lot environment frequently sought by a user and generating the parking lot map.

Description

Apparatus for controlling a parking, system having the same and method thereof

The present invention relates to a parking control apparatus, a system including the same, and a method thereof, and more particularly, to a technology for performing automatic parking control by generating and learning a parking lot map.

A driving assistance system for a vehicle is being developed for user convenience. As one of the driving assistance systems of a vehicle, a parking assistance system exists.

In the conventional parking assistance system, in perpendicular parking or parallel parking, the user directly searches for a parking space, selects a parking space, and controls the vehicle through a remote control device near the vehicle until the vehicle parks itself in the parking space. In addition, in parallel parking, parking control is possible only when the user is in the vehicle, and in perpendicular parking, the user can control parking only near the vehicle, and the vehicle is pulled out of the parking space.

In addition, when using GPS, it is impossible to control parking assistance indoors without a GPS signal, and when parking assistance is performed using communication equipment, additional V2X terminals need to be installed in the vehicle and parking lot infrastructure.

An embodiment of the present invention provides a parking control device capable of learning a parking lot environment frequently visited by a user and generating a parking lot map using a sensing device mounted on a vehicle without a separate wireless communication (V2X) device, a system including the same, and a method therefor would like to provide

In addition, an embodiment of the present invention is to provide a parking control device, a system including the same, and a method thereof, which can perform a space search and parking of a vehicle on its own based on a learned parking lot environment after a user gets off at a desired location.

In addition, in the embodiment of the present invention, when the user wants to get into the vehicle after parking, the user selects the location where the user got off before or the location entered in advance, and the vehicle is controlled to move to the location where the user wants to exit and board the vehicle by itself to the corresponding location. An object of the present invention is to provide a parking control device capable of performing the same, a system including the same, and a method therefor.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

A parking control apparatus according to an embodiment of the present invention learns a parking lot environment based on image data, generates a parking lot map, and controls the vehicle to automatically search a parking space and park based on the learned parking lot map. ; and a storage unit configured to store the map.

In an embodiment, the processor may include generating a parking lot map based on parking lot driving history information.

In an embodiment, the processor may include determining a map generation or learning time based on a parking lot determination condition.

In an embodiment, the processor is configured such that the vehicle travels below a certain vehicle speed, changes a direction every time it travels a certain distance, recognizes a parking lot related phrase, a parking line, or a parking lot pictogram in the image data, or receives a GPS signal. In case of interruption, it may include starting the map creation or learning.

In one embodiment, the processor records the parking lot driving history when the vehicle travels over a certain vehicle speed, travels over a certain distance without changing directions, a GPS signal is received again, or an exit related phrase is recognized in the image data It may include terminating.

In an embodiment, the processor is configured to, when the searched area is greater than or equal to a predetermined reference value by proximate to both sides within the drivable road width, when the ratio of obstacles that do not exist in the parking space is less than or equal to the predetermined reference value, the generated The method may include determining that the map is in a learned state when the occupancy ratio of the left and right fixed obstacles on the total driving path of the map is equal to or greater than a predetermined reference value.

In an embodiment, the processor may include performing manual map learning when there is a request by a user.

In an embodiment, the processor may include determining the inter-floor movement of the parking space according to the slope of the parking lot road, and performing learning on the inter-floor layer.

In one embodiment, the processor is, based on the image data by the front camera, the number of floors displayed on the wall, parking space number, parking area number, entrance and exit guidance display, progress direction guidance, entry prohibition display, disabled parking area display , may include recognizing at least one or more of the display of the electric vehicle parking area as a feature point and reflecting it on the map.

In an embodiment, the processor is, based on the image data from the SVM camera, at least one or more of a parking space number, a traveling direction, an entrance and an exit, a parking space type, a parking space size, a fixed obstacle, and drivable space information may include recognizing as a feature point and reflecting it on the map.

In an embodiment, the processor may include generating by reflecting the boarding and disembarking zone, the parking available zone, the no-parking zone, the parking preference zone, and the parking avoidance zone on the map.

In an embodiment, the processor may include, after the user gets off, storing the user's getting off position in the storage unit, and performing automatic parking control of the vehicle.

In an embodiment, the processor may include moving the vehicle to a previous disembarkation position when receiving a ride call from the user after parking is complete.

In an embodiment, the processor may include transmitting the user's getting off position and the parking position of the vehicle to the user's mobile communication terminal.

In an embodiment, the processor may include receiving a boarding request through a telematics terminal, a home LOT, and a mobile communication terminal.

In an embodiment, when an obstacle that does not exist in the existing learning result exists during the search for the parking space or the vehicle exiting, the processor may stop and wait until the obstacle disappears on the path or avoid the obstacle. It may include controlling to avoid driving through the avoidance route when there is an existing route, or to reverse backward when the avoidance route does not exist.

A vehicle system according to an embodiment of the present invention includes a sensing device for photographing a parking lot and sensing an obstacle; and a parking control device that learns a parking lot environment based on the image data by the sensing device and generates a parking lot map, and controls the vehicle to automatically search and park a parking space based on the learned parking lot map. can

In one embodiment, the sensing device, a front camera for photographing a distance in front of the vehicle; an SVM camera that shoots short distances around the vehicle; and an ultrasonic sensor that detects obstacles around the vehicle.

In one embodiment, a server that uploads or downloads the learned parking lot map; may further include.

A parking control method according to an embodiment of the present invention includes the steps of photographing around a parking lot and sensing an obstacle; learning a parking lot environment based on the sensed data and generating a parking lot map; and controlling the vehicle to automatically search for and park a parking space based on the learned parking lot map.

This technology uses a sensing device mounted on a vehicle without a separate wireless communication (V2X) device to learn the parking lot environment frequently visited by users and create a parking lot map.

This technology allows the vehicle to perform space search and parking by itself based on the learned parking lot environment after getting off at the location desired by the user.

In this technology, when the user wants to get into the vehicle after parking, it is possible to select a location where the user got off before or a location entered in advance to control the vehicle to move to the location where the user wants to get out and board the vehicle by itself to the corresponding location.

In addition, various effects directly or indirectly identified through this document may be provided.

1 is a block diagram illustrating a configuration of a vehicle system including a parking control apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a map learning-based parking control method according to an embodiment of the present invention.
3 is a flowchart illustrating a method for learning a parking lot environment and building a map according to an embodiment of the present invention.
4 is an exemplary view of a parking lot map according to an embodiment of the present invention.
5 is an exemplary diagram for explaining an example of extracting a feature point using a front camera according to an embodiment of the present invention.
6 is an exemplary diagram for explaining an example of extracting a feature point using a Surround View Monitoring (SVM) system according to an embodiment of the present invention.
7 is an exemplary diagram for explaining a method of extracting a feature point based on image data according to an embodiment of the present invention.
8 is an exemplary diagram of data obtained based on a sensing device according to an embodiment of the present invention.
9 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
10 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
11 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
12 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
13 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
14 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
15 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
16 is an exemplary diagram of a map learning-based parking lot environment according to an embodiment of the present invention.
17 illustrates a computing system according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function interferes with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the components from other components, and the essence, order, or order of the components are not limited by the terms. In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

The present invention learns the parking lot environment frequently found by the user through the vehicle's imaging device, automatically or manually builds a map, allows the vehicle to search for a parking space and control the parking by itself based on the learned parking map, and allows the user to ride If desired, a configuration in which the vehicle is automatically moved to the riding position is disclosed.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 17 .

1 is a block diagram showing the configuration of a vehicle system 10 including a parking control device according to an embodiment of the present invention.

Referring to FIG. 1 , a vehicle system according to an embodiment of the present invention may communicate with an external server 500 and a user's mobile communication terminal 600 . The vehicle system may download or upload a parking lot map from the server 500 , and may transmit map information, a disembarkation position, a boarding position, and the like to the mobile communication terminal 600 . The vehicle system may include a parking control device 100 , a sensing device 200 , a navigation device 300 , and a telematics device 400 .

The parking control apparatus 100 according to an embodiment of the present invention may be implemented inside the vehicle 10 . In this case, the parking control apparatus 100 may be integrally formed with the vehicle's internal control units, or may be implemented as a separate device and connected to the vehicle's control units by a separate connection means.

The parking control apparatus 100 may learn a parking lot environment based on the image data and generate a parking lot map, and may control the vehicle to automatically search a parking space and park the vehicle based on the learned parking lot map. In addition, the parking control apparatus 100 controls the vehicle to automatically move to a user's previous disembarkation position or a position selected by the user to wait when the user wants to board the vehicle.

The parking control apparatus 100 may include a communication unit 110 , a storage unit 120 , a display unit 130 , and a processor 130 .

The communication unit 110 is a hardware device implemented with various electronic circuits to transmit and receive signals through a wireless or wired connection. In the present invention, an in-vehicle network communication technology can be performed. In-vehicle communication may be performed through Controller Area Network) communication, LIN (Local Interconnect Network) communication, Flex-Ray communication, and the like. As an example, the communication unit 110 may communicate with in-vehicle devices such as the navigation device 300 and the telematics device 400 .

The storage 120 may store a sensing result (eg, image data, etc.) of the sensing device 200 and data and/or an algorithm required for the processor 140 to operate.

As an example, the storage unit 120 may store parking lot environment information sensed through image data of a parking lot environment, an ultrasonic sensor, and the like.

The storage unit 120 includes a flash memory type, a hard disk type, a micro type, and a card type (eg, SD card (Secure Digital Card) or XD card (eXtream Digital) Card)), RAM (Random Access Memory), SRAM (Static RAM), ROM (Read-Only Memory), PROM (Programmable ROM), EEPROM (Electrically Erasable PROM), magnetic memory (MRAM) , a magnetic RAM), a magnetic disk, and an optical disk-type memory may include at least one type of storage medium.

The display unit 130 may include an input unit for receiving a control command from a user and an output unit for outputting an operation state and result of the apparatus 100 . Here, the input means may include a key button, and may include a mouse, a joystick, a jog shuttle, a stylus pen, and the like. In addition, the input means may include a soft key implemented on the display. The output means may include a display, and may include an audio output means such as a speaker. In this case, when a touch sensor such as a touch film, a touch sheet, or a touch pad is provided in the display, the display operates as a touch screen, and the input means and the output means are integrated. In the present invention, the output means may output parking map information, parking control information, and the like.

In this case, the display includes a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (Flexible Display). , a field emission display (FED), and a three-dimensional display (3D display) may include at least one.

The processor 140 may be electrically connected to the communication unit 110, the storage unit 120, the display unit 130, etc., and may electrically control each component, and may be an electrical circuit that executes a software command, Thereby, various data processing and calculations described later can be performed.

The processor 140 may process a signal transmitted between each component of the parking control apparatus 100 . The processor 140 may be, for example, an electronic control unit (ECU), a micro controller unit (MCU), or other sub-controller mounted in a vehicle.

The processor 140 learns the parking lot environment frequently found by the user through the sensing device 200, builds a map automatically or manually, and based on the learned parking lot environment information, the vehicle searches for a parking space by itself and performs parking control can do.

In addition, after the user gets out of the vehicle, the processor 140 may search for a parking space in the corresponding parking lot and perform parking control based on the parking environment information that the vehicle has automatically learned, and the get-off location is stored in the storage unit 120 . ) can be controlled.

In addition, the processor 140 may control the vehicle to move to a previous disembarkation position or a previously input position when the user wants to get on (board) after parking.

The processor 140 may perform automatic map learning based on the parking driving route.

The processor 140 may calculate the trajectory of the vehicle by using a wheel pulse sensor, a wheel speed sensor, and a steering wheel angle sensor of the vehicle. In addition, when receiving the GPS signal, the processor 140 may calculate a trajectory by using the GPS signal together with the sensing results of the vehicle wheel pulse sensor, the wheel speed sensor, and the steering wheel angle sensor.

The processor 140 may 1) the vehicle travels below a certain vehicle speed, 2) change the direction every time the vehicle travels a certain distance, or 3) the SVM camera or the front camera detects "parking lot", "parking", "entrance", When phrases such as "Parking", "Park", "Entrance", parking lines, and parking lot pictograms are recognized, or 4) the GPS signal is interrupted, it can be determined that the vehicle has entered the parking lot.

The processor 140 detects: 1) a point in time when the direction was not repeatedly changed before the determined point in time, 2) a point in time when the GPS signal was cut off (the point in time when entering the underground parking lot), 3) when the SVM camera or the front camera "parked", " At least one of the time points at which the phrases "entrance" and "Entrance" are recognized may be determined as the time of recording parking lot driving history.

The processor 140 detects that 1) the vehicle travels over a certain vehicle speed, 2) travels more than a certain distance without changing direction, 3) the GPS signal is received again, 4) the SVM/front camera is recognizing the phrases “exit” and “exit” When at least one is performed, the parking lot driving history recording may be terminated.

When the user drives again in the area where the map is generated, the processor 140 may repeatedly learn the parking lot driving history by adding a drivable area by accumulating the trajectory of the user's driving.

The processor 140 may perform automatic map learning based on the user's accumulated driving history information.

The processor 140 may determine whether the following condition is satisfied in order to determine whether a map for stable parking is learned by accumulating driving trajectories through repeated learning by the user driving several times or more.

1) When as many areas are searched as possible by searching close to both sides within the drivable road width

1-1) Must have a history of learning the map within X[m] from both ends of the drivable road width

1-2) A certain distance X from both ends of the road can be set differently depending on the road width and straight section/corner section

2) When the ratio of temporary obstacles such as cars and people not in the parking space is below a certain level

2-1) When the ratio of each left/right occupied by the temporary obstacle in the straight section distance Y[m] in the straight section is less than or equal to Z[%]

2-2) When the ratio of each left/right occupied by the temporary obstacle in the curve section distance Y'[m] in the curve section is less than Z'[%]

3) When the occupancy ratio of the left/right fixed obstacles (wall/pillar/floor markers) on the total driving route T[m] of the created map is S[%] or more

4) If a temporary obstacle that was not found during previous map learning or temporary obstacle discovered during previous map learning disappears, it is counted as not occupied.

When the conditions 1), 2), 3), and 4) are satisfied, that is, when it is determined that a map sufficient for stable parking is generated, the processor 140 informs the user of the generated map.

After determining whether to learn the map, the processor 140 may receive a selection from the user whether to learn the additional map.

The subsequent steps are the same as the manual map learning steps, and the manual map learning process will be described below.

When the processor 140 receives a manual map learning request from the user, that is, when the user wants to learn the parking lot map, the manual map learning starts through the parking lot map manual learning command of the navigation device 300 .

The processor 140 may use the driving history from the point in time when the GPS signal recorded by automatic map learning is cut off (the point in time of entering the underground parking lot) to generate the map.

The processor 140 guides the user to drive the driving route that the user wants to learn at least once, and guides the user to the map generated through learning. In this case, the map may be displayed through the display unit 130 .

While the user is learning the parking lot map, the user may request to confirm whether the map is generated as intended by the navigation device 300, and after confirming whether the map is generated as the user intended, the processor 140 is generated In order to secure the reliability of the map, the driving trajectory can be accumulated through repeated learning by driving the map several times or more and reflected in the map data.

4 is an exemplary view of a parking lot map according to an embodiment of the present invention. 401 and 402 of FIG. 4 are exemplary views of a map of a parking lot, and 403 is an example of displaying a disembarkation location, a boarding location, an avoidance area, a prohibited area, a preferred area, and the like on the map.

The processor 140 generates a map to have a layer structure so that it can learn about parking spaces on multiple floors, estimates the slope of the road using a longitudinal acceleration sensor, a wheel speed sensor, and an image sensor, and generates a map of the road. When the slope changes abruptly, it is determined that the floor of the parking space changes. In addition, when moving to a parking space on a different floor, the processor 140 may learn all of the area in which the floor changes so as to facilitate processing between maps of different layers in a separate layer.

When generating an automatic map or a manual map, the processor 140 may ID the feature points recognized using the front camera and reflect the corresponding information, location information, and direction information on the parking lot map.

The processor 140 may reflect the 3D coordinates of the feature points recognized based on the image data to the map, and generate a 3D-based map.

The processor 140 recognizes the number of floors/parking space number/parking area number, entrance/exit guide display, progress direction guidance, entry prohibition display, disabled parking area display, electric vehicle parking area display, etc. as characteristic points displayed on the wall on the image data. can

5 is an exemplary diagram for explaining an example of extracting a feature point using a front camera according to an embodiment of the present invention. Reference numerals 501 and 502 of FIG. 5 indicate positions and shooting ranges of the front camera. 503 discloses an example in which the number of the parking lot wall is recognized as a feature point, 504 is an example in which "entry prohibited" and "parking lot entrance" of the parking lot entrance are recognized as a feature point, and 505 is an "exit" and "underground 1 entrance" as a feature point. This is an example recognized as

The processor 140 may use image data captured using the SVM camera 210 when generating an automatic map or a manual map. The processor 140 uses the SVM camera 210 to ID the recognized feature points and reflects the corresponding information and location information on the 2D map of the parking lot. In addition, the processor 140 generates a 6D map including 6D information of the feature points.

The processor 140 may recognize a parking space number, a traveling direction, an entrance/exit indication, a parking space type indication (a parking area for the disabled, an electric vehicle, etc.) as a feature point.

The processor 140 may reflect the parking space type and size information on the parking lot map based on the recognized parking line using the SVM camera 210 . In addition, the processor 140 reflects the fixed obstacle (wall, column, curb) information recognized using the SVM camera 210 on the parking lot map, and reflects the recognized drivable space information (free space) information on the parking lot map. do.

6 is an exemplary diagram for explaining an example of extracting a feature point using a Surround View Monitoring (SVM) system according to an embodiment of the present invention. An example of recognizing an arrow indicating a driving direction on a road as a feature point is disclosed in 601 and 605 of FIG. 6 , an example of recognizing a disabled sign of a parking space as a feature point is disclosed in 602, and parking on a parking line in 603 and 604 An example of recognizing an obstacle such as a vehicle is disclosed, 606 discloses an example in which a number indicating a parking position on a road is recognized as a feature point, and 607 discloses an example in which an electric vehicle parking space display of a parking space is recognized as a feature point and 608, 609, and 610 also disclose examples of recognizing the marks (OUT, IN, entrance, etc.) in the parking lot road as a feature point.

7 is an exemplary diagram for explaining a method of extracting a feature point based on image data according to an embodiment of the present invention. Referring to FIG. 7 , it can be seen that the processor 140 distinguishes and displays feature points on each image data.

As described above, the processor 140 performs map learning based on image data captured by the front camera or the SVM camera, but image information of the front camera and the SVM camera may be different. That is, when an obstacle was recognized from a long distance using the front camera, but the image was taken with the SVM camera at a short distance, the obstacle may not be recognized. Accordingly, the processor 140 may perform the following correction for each situation.

1) Fixed obstacle: Correct the SVM image recognition result based on the result recognized by the front camera

2) Moving obstacles: Update map data of temporary obstacles such as vehicles/people

3) Unrecognized/misrecognized by light sources such as reflection/light/shadow: use of ultrasonic sensor value

4) Floor marker that looks like a three-dimensional object: using the ultrasonic sensor value

5) Use the front camera for obstacles in areas that the SVM cannot recognize (e.g., obstacles away from the floor such as fire hydrants/blocking gates)

Then, the processor 140 may process obstacles during map learning. When a vehicle that is not parked within the parking dividing line is learned, the processor 140 checks whether a vehicle passes in the corresponding area through re-learning of the corresponding area, and when the vehicle is learned at the same location even during re-learning, the corresponding vehicle can learn from the map with illegally parked vehicles.

In addition, when an object other than a fixed obstacle, such as a person/bicycle, is learned on the map, the processor 140 checks whether the vehicle passes through the corresponding area through re-learning of the area, and checks whether the obstacle is at the same location during re-learning. If learned, the corresponding obstacle may be learned on the map as a temporary obstacle.

In addition, the processor 140 may remove the illegally parked vehicle/temporary obstacle from the map if it disappears after re-checking during map re-learning or map learning in the future.

In addition, the processor 140 may designate and display an on/off zone, a parking available/prohibited zone, and a preferred zone/avoidance zone on the map as shown in 403 of FIG. 4 .

When the automatic map or manual map generation is completed, the processor 140 checks whether the user's desired boarding/alighting area has been learned.

When it is confirmed that the user-desired boarding/unloading zone has not been learned, the processor 140 may guide the user to drive the desired boarding/unloading zone in the same manner as in the manual map learning procedure and reflect it on the map. In addition, when learning, the processor 140 also stores route information that the user has manually driven in the map, and when learning, searches for a parking space located near the driving route and stores it in the map. In addition, when it is confirmed that the user-desired embarkation / disembarkation zone has been learned, or upon completion of additional learning of the embarkation / disembarkation zone, the processor 140 guides the user to the map generated through learning, and the user rides on the generated map Zones and drop-off zones can be specified. In addition, the processor 140 sets the inter-floor movement layer as a prohibited zone, but allows the user to manually release it. The generated map may be stored in an in-vehicle system or transmitted to the user's mobile communication terminal 600 through the telematics device 400 .

In addition, the processor 140 may receive, if necessary, the user's vehicle information (general/disabled person/electricity, etc.) designated through the navigation device 300 or the display unit 130 screen, the mobile communication terminal 600, and the learned map. Parking allowed/prohibited zones, preferred/avoided zones, and non-drivable zones can be designated on the top. In addition, the processor 140 controls so that when a vehicle enters a zone in the map learned according to a preset, autonomous parking is performed without a separate operation after getting off for each zone or through manipulation of a smart key or mobile phone terminal or other methods. can do.

In addition, the processor 140 may guide the user through the navigation device 300 or the display unit 130 that autonomous parking is possible when the GPS coordinates of the own vehicle pass the learned GPS coordinates of the parking lot.

In addition, the processor 140 corrects the location of the vehicle by comparing the 3D coordinates of the learned feature points with the 3D coordinates of the feature points recognized by the front camera or the SVM camera after the vehicle enters the learned parking map.

Also, the processor 140 may control the learned map to be displayed on the navigation device 300 or the display unit 130 and the location of the own vehicle to be displayed on the map.

In addition, the processor 140 displays on the screen of the navigation device 300 or the display unit 130 that the autonomous parking function can be used after the vehicle enters the learned parking map, so that the user can use the navigation device 300 or the display unit 130 . ) to perform autonomous parking, and after getting off, according to a preset, autonomous parking can be performed without a separate operation or through operation of a smart key or a mobile communication terminal.

In addition, when the user enters a pre-set alighting zone, the processor 140 alights without a separate setting and then performs autonomous parking according to the pre-set without a separate operation or through the operation of a smart key or a mobile phone terminal. .

Also, the processor 140 may record the location where the user got off the system in the system. After parking is complete, when the user wants to get on again later, the location where he got off can be stored so that he can call the vehicle to that location.

After the user gets off, the processor 140 performs parking control using the map learned in advance and the driving path used for learning, and considers the passable width and the turning radius in the parking lot using the path traveled by the user during manual learning Thus, a parking space search trajectory may be generated.

In addition, the processor 140 determines whether there is a path that can be avoided by using the pre-learned map information for obstacles that are not in the existing learning result during parking space search, and if there is a path that can be avoided, avoid in the corresponding direction. And, if there is no route that can be avoided, the control can be performed to reverse the route traveled until it can travel on another route.

In addition, the processor 140 searches for a parking space when following the space search trajectory, and when the learned parking space is empty or a new parking space is found, the process proceeds to the next parking space selection process.

In addition, the processor 140 uses the telematics device 400 to warn the user's mobile communication terminal 600 (eg, a smartphone) when a parking space is not searched even after searching all the previously learned maps. can be sent.

When a parking space is searched for, the processor 140 satisfies the condition of the searched parking space based on the user's vehicle information (general/disabled/electric vehicle, etc.) input by the user in advance and the set preferred zone/avoid zone information , parks in the corresponding parking space, and returns to the parking space search step if not satisfied.

In addition, when the processor 140 transmits a warning to the user's mobile communication terminal through the telematics device 400 because the parking space is not searched even after searching all the previously learned maps, the user can park the vehicle on the map. The location can be set manually.

The processor 140 may control the vehicle to be parked in the selected parking space by using the ultrasonic sensor and SVM camera information mounted on the vehicle. Also, after parking is completed, the processor 140 may store the parked location on the map and transmit the parking location to the user's mobile communication terminal through the telematics device 400 .

In addition, when receiving a boarding request from the telematics device 400 or the home IOT, the processor 140 may control the vehicle to move to a previous disembarkation location or a location input by the user. In addition, the processor 140 may receive arbitrary location information on a location selected by the user or a map as a location to call a vehicle through an IOT terminal or a smart phone terminal. Then, the processor 140 may perform driving control by generating a driving trajectory to the previous disembarkation position or a position selected by the user.

In addition, when a pedestrian exists on a path that is not in the existing learning result during parking space search/exit, the processor 140 may stop and continue to follow the path until the pedestrian disappears from the path.

In addition, the processor 140 determines whether it is possible to avoid an oncoming vehicle using pre-learned map information after stopping the own vehicle when an obstacle on the path that is not in the existing learning result during parking space search/exit is a vehicle facing and approaching. , when there is an avoidable route, it avoids in the corresponding direction, and when there is no avoidable route, the control can be controlled to reverse the route traveled until it can travel on another route.

In addition, in the case of a vehicle preceding an obstacle on a path that is not in the existing learning result during parking space search/exit, the processor 140 may follow the path while maintaining a predetermined distance from the preceding vehicle. In addition, the processor 140 recognizes the reversing light of the vehicle preceding the obstacle on the path that is not in the existing learning result during parking space search/exit, and the preceding vehicle has the will to park the reversing light or forward traffic. It is determined that the vehicle is reversing due to the congestion, and using the map information learned in advance, it can be controlled to reverse the route traveled until it can travel to another route.

In addition, when there is a vehicle following when reversing, the processor 140 may perform reverse path tracking while maintaining a predetermined distance from the following vehicle.

The processor 140 may update the map when parking/exiting by using the automatic parking map learning function.

Such an automatic parking map may be purchased or downloaded from the server 500 through the navigation device 300 , the telematics device 400 , the mobile communication terminal 600 , and the like without the need for learning about the parking lot. In addition, as in the present invention, the learned parking lot information may be sold or uploaded to the server 500 .

When a vehicle enters the parking space area stored in the server 500 , the server 500 purchases/downloads a map of the parking space through the navigation device 300 , the telematics device 400 , or the mobile communication terminal 600 . You can check with your doctor.

The sensing device 200 includes information such as a parking lot entrance or exit sign, a parking line in the parking lot, a driving direction line, etc., obstacles located in or around the parking lot, for example, a parked vehicle, a person walking in the parking lot, a parking lot wall It may include one or more sensors that measure the back.

The sensing device 200 may include a plurality of sensors to detect an object external to the vehicle, and the position of the external object (obstacle), the speed of the external object, the moving direction of the external object, and/or the type of the external object (eg: information about vehicles, pedestrians, bicycles or motorcycles, etc.) can be obtained. To this end, the sensing device 200 includes an SVM camera 210, a front camera 220, an ultrasonic sensor 230, etc., which is not shown in FIG. 1, but includes a radar, a camera, a laser scanner and/or a corner radar; It may further include a lidar, an acceleration sensor, a yaw rate sensor, a torque measuring sensor and/or a wheel speed sensor, a steering angle sensor, and the like. In the present invention, a parking space may be searched using an ultrasonic sensor, and the presence or absence of an obstacle in the parking lot may be determined using the SVM camera 210 and the front camera 220 .

The SVM camera 210 may include a camera applied to a surround view monitor system, and may be mounted on the front of the vehicle, the rear of the vehicle, or on the side of the vehicle, and transmit image data captured around the vehicle to the parking control device ( 100) can be provided. The SVM camera 210 can see the vehicle omnidirectionally, but it is disadvantageous to a three-dimensional object, it is easy to recognize a floor marker, and has characteristics similar to a view viewed on a map.

The front camera 220 may be mounted on the windshield glass of the vehicle to photograph the front of the vehicle and provide image data to the parking control apparatus 100 . The front camera 220 is capable of recognizing the front of a distant area, similar to an actual user's field of view, and is advantageous for recognizing a stereoscopic object and recognizing a wall marker. Accordingly, the front camera 220 may recognize feature points such as path recognition for autonomous space search, obstacle recognition on a path, and a marker for map learning/own vehicle location estimation.

The ultrasonic sensor 230 may detect an obstacle, a lane, a parking line, etc. around the vehicle and provide it to the parking control apparatus 100 . The ultrasonic sensor 230 may compensate for image limitations such as extremely low illumination, road reflection, and camouflage pattern.

The navigation device 300 may provide map information and parking lot map information around the parking lot to the parking control device 100 .

The telematics apparatus 400 may provide a map generated by communicating with the user's mobile communication terminal 600 to the user's mobile communication terminal 600 .

Hereinafter, a map learning-based parking control method according to an embodiment of the present invention will be described in detail with reference to FIG. 2 . 2 is a flowchart illustrating a map learning-based parking control method according to an embodiment of the present invention.

Hereinafter, it is assumed that the parking control apparatus 100 of FIG. 1 performs the process of FIG. 2 . Also, in the description of FIG. 2 , an operation described as being performed by the device may be understood as being controlled by the processor 140 of the parking control device 100 .

Referring to FIG. 2 , the parking control device 100 uses the image data around the vehicle acquired by the sensing device 200 to learn the parking lot environment frequently sought by the user and build a map ( S201 ). That is, the parking control device 100 can learn the parking lot environment frequently found by the user by using the SVM camera 210, the front camera 220, and the telematics device 400 mounted on the vehicle without a separate V2V communication device. have.

The parking control apparatus 100 may perform space search and parking by itself based on the learned parking lot environment (S201).

After the user gets off at a desired location, the parking control device 100 searches for an empty parking space in the parking lot based on the learned parking lot environment and performs parking control so that the vehicle moves to and parks in the searched parking space by itself.

After the parking is completed, when the user wants to board the vehicle, the parking control apparatus 100 performs vehicle movement control to the get-off position or the pre-entered get-off position during parking (S203). Accordingly, when the user selects a disembarkation position or a pre-entered disembarkation position during parking, the vehicle moves to the corresponding position, so that the user can directly get into the vehicle at the desired position.

As such, the present invention does not park the user by directly searching for parking sympathy, and even if the user gets off after entering the parking lot or gets off the parking lot, the vehicle itself searches for a parking space and automatically parks the vehicle, and after parking, the user selects the parked vehicle for boarding the vehicle. By moving the vehicle to the user's previous disembarkation position without moving to a searched and parked position, the user can get on directly from the previous disembarkation position, thereby increasing the user's convenience.

Hereinafter, a method for learning a parking lot environment and building a map according to an embodiment of the present invention will be described in detail with reference to FIG. 3 . 3 is a flowchart illustrating a method for learning a parking lot environment and building a map according to an embodiment of the present invention.

Hereinafter, it is assumed that the parking control apparatus 100 of FIG. 1 performs the process of FIG. 3 . In addition, in the description of FIG. 3 , an operation described as being performed by the apparatus may be understood as being controlled by the processor 140 of the parking control apparatus 100 .

Referring to FIG. 2 , the parking control apparatus 100 accumulates a user driving history ( S301 ), and performs automatic map learning based on the accumulated user driving history ( S302 ).

Then, the parking control device 100 determines whether a map is generated (S303), and if the map is generated, suggests parking lot learning to the user (S304).

The parking control device 100 determines whether the boarding/unloading location is included in the parking lot information of the map (S305), and if the boarding/unloading location is included, learns the user's boarding/unloading location (S306).

On the other hand, when the parking lot information of the map does not include the getting on/off position, the parking control device 100 additionally registers the user's getting on/off position (S307).

In addition, when the user requests parking lot learning (S401), the parking control device 100 performs user manual map learning based on GPS coordinates (S402).

The parking control device 100 determines whether a map is generated (S403), and if the map is generated, determines whether the boarding/alighting location is included in the parking lot information of the map (S305), and the boarding/alighting location If is included, the user's getting on/off position is learned (S306).

On the other hand, when the parking lot information of the map does not include the getting on/off position, the parking control device 100 additionally registers the user's getting on/off position (S307).

8 to 16 are exemplary views of a map learning-based parking lot environment according to an embodiment of the present invention.

In FIG. 8 , as an example of creating a map of a parking lot on the first basement level and a parking lot on the second basement floor, an example in which a feature point recognized based on image data of a front camera and an SVM camera is reflected on the map is disclosed.

In FIGS. 9 and 10 , a map of the parking path from the first floor to the first basement level is disclosed. Referring to FIG. 10 , when it is determined that the parking lot is entered at the time when the GPS signal is cut off, map learning is started, and a section in which the road slope changes abruptly occurs, it is determined as an inter-floor movement layer, and the section where the road slope is stabilized is in the parking lot. It can be determined that the inter-floor movement has been completed.

In FIG. 11 , an example of searching for a space in a parking lot on the first basement floor after completion of interfloor movement is disclosed, and in FIG. 12 , an example of displaying a parked vehicle, a parking line, and characteristic points is disclosed. 1301 and 1302 of FIG. 13 show an example of performing learning (-0.5 layer compared to the entry area layer) on the inter-floor movement layer when entering a section in which the road slope changes rapidly, and completing the learning of the inter-floor movement layer when entering the section where the road slope is stabilized. start

14 discloses an example of performing inter-floor movement layer learning in the 2nd basement parking lot, and FIG. 15 discloses a 2nd basement parking lot map. 16 shows the entire parking lot map of the entrance, the first basement floor, and the second basement floor.

As described above, the present invention generates an automatic map for the parking lot environment by accumulating the user's driving history. In addition, according to the present invention, the map generation and learning time can be performed through the parking lot determination condition and the map generation can be performed by determining the driving history recording time/end time. In addition, the present invention can determine parking fixtures/temporary obstacles, etc. through repeated driving learning and select whether to be reflected on the map, and can perform a manual parking lot map learning command by the user. In addition, the present invention can determine the characteristics and recognition marks of the parking lot when creating a map and reflect it on the map.

17 illustrates a computing system according to an embodiment of the present invention.

Referring to FIG. 17 , the computing system 1000 includes at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , and storage connected through a bus 1200 . 1600 , and a network interface 1700 .

The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and/or the storage 1600 . The memory 1300 and the storage 1600 may include various types of volatile or nonvolatile storage media. For example, the memory 1300 may include read only memory (ROM) and random access memory (RAM).

Accordingly, the steps of a method or algorithm described in connection with the embodiments disclosed herein may be directly implemented in hardware, a software module, or a combination of the two executed by the processor 1100 . A software module resides in a storage medium (ie, memory 1300 and/or storage 1600 ) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM. You may.

An exemplary storage medium is coupled to the processor 1100 , the processor 1100 capable of reading information from, and writing information to, the storage medium. Alternatively, the storage medium may be integrated with the processor 1100 . The processor and storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and storage medium may reside as separate components within the user terminal.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (20)

a processor for learning a parking lot environment based on image data and generating a parking lot map, and controlling the vehicle to automatically search and park a parking space based on the learned parking lot map; and
a storage unit for storing the map;
A parking control device comprising a. The method according to claim 1,
The processor is
A parking control device, characterized in that for generating a parking lot map based on the parking lot driving history information. 3. The method according to claim 2,
The processor is
A parking control device, characterized in that for determining the time of map generation or learning through the parking lot determination condition. 3. The method according to claim 2,
The processor is
When the vehicle travels below a certain vehicle speed, changes direction every time it travels a certain distance, when a parking lot related phrase, parking line, or parking lot pictogram is recognized in the image data, or when the GPS signal is cut off, the map creation or learning starts Parking control device, characterized in that. 5. The method according to claim 4,
The processor is
Parking control device, characterized in that when the vehicle travels more than a certain vehicle speed, travels more than a certain distance without changing direction, a GPS signal is received again, or an exit related phrase is recognized in the image data, the parking control device, characterized in that it ends the recording of the parking lot driving history. The method according to claim 1,
The processor is
If the searched area is greater than or equal to a predetermined threshold by searching on both sides within the drivable width of the road, if the ratio of obstacles that do not exist in the parking space is less than or equal to the predetermined threshold, the left and right sides of the created map are fixed on the total driving path The parking control device, characterized in that it is determined that the map is in a learned state when the occupancy ratio of the obstacle is equal to or greater than a predetermined reference value. The method according to claim 1,
The processor is
Parking control device, characterized in that when there is a request by the user to perform manual map learning. The method according to claim 1,
The processor is
A parking control device, characterized in that it determines the inter-floor movement of the parking space according to the slope of the parking lot road, and performs learning on the inter-floor layer. The method according to claim 1,
The processor is
Based on the image data from the front camera, at least one or more of the number of floors displayed on the wall, parking space number, parking area number, entrance and exit information display, direction guidance, entry prohibition display, disabled parking area display, and electric vehicle parking area display is displayed. A parking control device, characterized in that it is recognized as a feature point and reflected on the map. The method according to claim 1,
The processor is
Based on the image data from the SVM camera, at least one or more of parking space number, progress direction, entrance and exit, parking space type, parking space size, fixed obstacles, and drivable space information is recognized as a feature point and reflected on the map A parking control device, characterized in that. The method according to claim 1,
The processor is
A parking control device, characterized in that generating by reflecting the boarding and disembarking zone, the parking available zone, the no-parking zone, the parking preference zone, and the parking avoidance zone on the map. The method according to claim 1,
The processor is
After the user gets off, the parking control device, characterized in that the user's get-off position is stored in the storage unit, and automatic parking control of the vehicle is performed. 13. The method of claim 12,
The processor is
After parking is completed, when receiving a ride call from the user, the parking control device, characterized in that the vehicle is controlled to move to a previous disembarkation position. 14. The method of claim 13,
The processor is
Parking control device, characterized in that for transmitting the user's getting off position and the parking position of the vehicle to the user's mobile communication terminal. The method according to claim 1,
The processor is
A parking control device, characterized in that it receives a ride request through a telematics terminal, a home LOT, and a mobile communication terminal. The method according to claim 1,
The processor is
When there is an obstacle that does not exist in the existing learning result during the search for the parking space or the vehicle exiting, the vehicle stops and waits until the obstacle disappears on the path, or if there is a path that can avoid the obstacle, the avoidance path is selected A parking control device, characterized in that it controls the vehicle to avoid driving through the evasion route or to reverse reverse when the avoidance route does not exist. Sensing device for photographing the parking area and sensing obstacles; and
a parking control device that learns a parking lot environment based on the image data by the sensing device and generates a parking lot map, and controls the vehicle to automatically search and park a parking space based on the learned parking lot map;
A vehicle system comprising a. 18. The method of claim 17,
The sensing device is
A front camera for photographing a distance in front of the vehicle;
an SVM camera that shoots a short distance around the vehicle; and
an ultrasonic sensor that detects obstacles around the vehicle;
A vehicle system comprising a. 19. The method of claim 18,
a server that uploads or downloads the learned parking lot map;
Vehicle system, characterized in that it further comprises. Filming around the parking lot and sensing obstacles;
learning a parking lot environment based on the sensed data and generating a parking lot map; and
controlling the vehicle to automatically search for and park a parking space based on the learned parking lot map;
A parking control method comprising a.

Images