KR20220047196A - Image capturing apparatus of vehicle, server, user terminal apparatus and method for providing parking lot guidance service using them - Google Patents

Abstract

A parking lot guidance service providing method of a server is disclosed. The method comprises the following steps of: receiving parking lot data including information of a parking space in a parking lot from an image photographing device for a vehicle installed in a vehicle; generating a parking lot model representing a real-time parking situation of the parking lot by an image based on the received parking lot data; and providing a parking lot guidance service to a user terminal device by using the generated parking lot model.

Description

Vehicle image recording device, server, user terminal device, and method of providing parking information service using them

The present invention relates to an image capturing apparatus for a vehicle, a server, a user terminal device, and a method of providing a parking lot guidance service using the same.

As the vehicle becomes a common means of transportation, the vehicle is positioned as an essential existence of people living in modern society, and the number of owners who own a vehicle is gradually increasing.

Accordingly, the importance of a parking space for parking a vehicle is also increasing, and a parking lot having a large number of parking spaces is essential in large buildings such as department stores or government office apartments or parking-only buildings.

Accordingly, various management technologies are being developed to provide a more efficient parking service to drivers.

Conventionally, a method of counting the number of vehicles entering and leaving a parking lot and a method of installing an infrared sensor in each parking area were used to determine a parking area of a parking lot.

However, according to the conventional method, since it is difficult for a driver to accurately grasp the number of parking floors or a parking available location, there is an inconvenience in that the driver finds an empty parking area for each floor in order to park.

In addition, the driver who parked remembered the parking location or recorded the approximate parking location of the vehicle by taking a picture. It is inconvenient to find the parking location because it is difficult to determine the parking location.

Accordingly, the need for a parking lot guidance service to improve user convenience before and after parking is increasing.

The present invention has been devised in accordance with the above-mentioned necessity, and an object of the present invention is to create a parking lot model representing a real-time situation for a parking lot as an image by using an image taken from an image capturing device for a vehicle, and based on the created parking lot model An object of the present invention is to provide a parking lot guidance service to a user's terminal device.

A method of providing a parking lot guidance service of a vehicle image photographing apparatus according to an embodiment of the present invention for achieving the above object includes acquiring a parking lot image according to the photographing, and using the parking lot image, information of a parking space in the parking lot and transmitting the generated parking lot data to a parking lot guide service providing server, wherein the parking lot data can be used to create a parking lot model for service provision in the parking lot guide service providing server. there is.

And, generating the parking lot data is a step of recognizing a parking vehicle parked in the parking space and a location identifier that enables identification of the location of the parking space in the parking lot from the parking lot image, based on the recognized location identifier It may include generating location information of the parking space and generating information on whether or not a parking space included in the parking space is parked according to the recognized location of the parking vehicle.

In addition, the generating of the parking lot data may further include generating parking vehicle information including at least one of vehicle type information and vehicle number information for the recognized parking vehicle, wherein the parking vehicle information is A plurality of parking spaces constituting the space may be classified and generated.

And, determining whether or not to update the parking lot data by detecting a change in the parking lot image as a parked vehicle leaves the parking lot included in the parking space or another vehicle enters the parking space included in the parking space may further include.

In addition, the steps of determining whether an impact event has occurred in a parked vehicle parked in the vicinity of the own vehicle, when it is determined that an impact event has occurred in the parked vehicle, updating the parking lot data and using the updated parking lot data to the parking lot guidance service Further comprising the step of transmitting to the providing server, the updated parking lot data may include data before and after a predetermined time of the impact event occurrence time.

On the other hand, an image photographing apparatus for a vehicle according to an embodiment of the present invention for achieving the above object includes a communication unit, a photographing unit for acquiring a parking lot image according to shooting, and information of a parking space in a parking lot using the parking lot image A parking lot data generation unit for generating parking lot data, and a control unit for controlling the communication unit to transmit the generated parking lot data to a parking lot information service providing server, wherein the parking lot data is for providing a service from the parking lot information service providing server It can be used to create a parking lot model.

And, the parking lot data generation unit, based on the recognized location identifier and a location identifier that enables identification of the location of the parking space in the parking lot from the parking lot image, and an image processing unit for recognizing a parked vehicle parked in the parking space and a parking space information generating unit generating location information of a parking space and generating information about whether a parking space included in the parking space is parked according to the recognized location of the parking vehicle.

In addition, the parking data generation unit further comprises a parking vehicle information generation unit for generating parking vehicle information including at least one of type information and vehicle number information for the recognized parking vehicle, the parking vehicle information A plurality of parking spaces constituting the space may be classified and generated.

And, the control unit, whether the need to update the parking lot data by detecting a change in the parking lot image as a parked vehicle leaves the parking lot included in the parking space or another vehicle enters the parking space included in the parking space can be judged

In addition, the control unit determines whether an impact event of a parked vehicle parked in the vicinity of the own vehicle occurs, and controls the parking lot data generator to update the parking lot data when it is determined that an impact event has occurred in the parked vehicle, and the update Controls the communication unit to transmit the parking lot data to the parking information service providing server, the updated parking lot data may include data before and after a predetermined time at the time of the impact event occurrence.

On the other hand, the method of providing a parking lot guidance service of a server according to an embodiment of the present invention for achieving the above object comprises the steps of receiving parking lot data including information of a parking space in a parking lot from an image photographing device for a vehicle provided in a vehicle; Creating a parking lot model representing the real-time parking situation of the parking lot as an image based on the received parking lot data and providing a parking lot guidance service to a user terminal device using the created parking lot model.

In addition, the information of the parking space includes location information of the parking space and information on whether or not parking spaces constituting the parking space are parked, and the step of generating the parking lot model includes: based on the location information of the parking space Determining the location of the parking space in the parking lot, determining whether to place a vehicle model in the parking space based on whether the parking space is parked or not, and placing the vehicle model in the parking space according to the determination result It may include generating the placed parking lot model.

In addition, the parking lot data includes parking vehicle information including at least one of type information of a parking vehicle and number information of a parked vehicle, and generating the parking lot model includes a license plate and a vehicle type based on the parking vehicle information. The method may further include generating a vehicle model reflecting at least one of them.

In addition, the generated parking lot model may be a 3D model.

In addition, the method further comprises the step of updating the generated parking lot model, wherein the updating step extracts only a difference part between the previously generated parking lot model and the subsequently generated parking lot model, and reflects only the extracted difference part to the parking lot You can update the model.

And, the step of providing the parking lot guide service includes the steps of detecting the parking lot model and the parking lot data corresponding to the parking lot in which the vehicle of the user terminal device connected to the server is parked, and the detected parking lot model and the parking lot It may include providing at least one of a parking available location guidance service, a vehicle parking location guidance service, and a parking lot route guidance service to the user terminal device by using the data.

In addition, the step of providing the parking guide service includes, when an impact event occurs in the first vehicle parked in the parking lot, a notification of the occurrence of the first vehicle impact event to the vehicle image photographing device of the second vehicle located around the first vehicle. Transmitting, receiving parking data from an image capturing device for a vehicle of a second vehicle according to the notification, and shock information on the impact situation of the first vehicle based on the parking data from the image capturing device for a vehicle of the second vehicle and providing a parking shock event guidance service based on the generated shock information.

On the other hand, the parking information service providing server according to an embodiment of the present invention for achieving the above object is a communication unit that receives parking lot data including information of a parking space in the parking lot from an image photographing device for a vehicle provided in the vehicle, the reception A parking lot model generator for generating a parking lot model representing the real-time parking situation of the parking lot as an image based on the obtained parking lot data, and a controller for providing a parking lot guidance service to a user terminal device using the created parking lot model.

And, the information of the parking space includes location information of the parking space and information on whether or not parking spaces constituting the parking space are parked, and the parking lot model generation unit, based on the location information of the parking space, in the parking lot Determining the location of the parking space, determining whether to place the vehicle model in the parking space based on the information on whether the parking space is parked, and placing the vehicle model in the parking space according to the determination result. can create

In addition, the parking lot data includes parking vehicle information including at least one of type information of a parking vehicle and number information of a parked vehicle, and the parking lot model generation unit, based on the parking vehicle information, at least one of a license plate and a vehicle type It is possible to create a vehicle model reflecting

In addition, the generated parking lot model may be a 3D model.

In addition, the parking lot model generation unit may extract only a difference part between the previously generated parking lot model and a later generated parking lot model, and update the parking lot model by reflecting only the extracted difference part.

In addition, the control unit detects the parking lot model and the parking lot data corresponding to the parking lot in which the vehicle of the user terminal device connected to the server is parked, and a parking available location using the detected parking lot model and the parking lot data At least one of a guide service, a vehicle parking location guide service, and a parking lot route guide service may be provided to the user terminal device.

In addition, when an impact event occurs in the first vehicle parked in the parking lot, the communication unit transmits a first vehicle impact event occurrence notification to a vehicle image capturing apparatus of a second vehicle located around the first vehicle, and Accordingly, receiving the parking data from the vehicle image photographing device of the second vehicle, the control unit generates impact information on the impact situation of the first vehicle based on the parking lot data from the vehicle image photographing device of the second vehicle, A parking impact event guide service may be provided based on the generated impact information.

On the other hand, the method for providing a parking lot guidance service of a user terminal device according to an embodiment of the present invention for achieving the above object includes the steps of accessing a parking lot guidance service providing server that provides a parking lot guidance service based on a vehicle image photographing device; Receiving a parking lot model and parking lot data representing the real-time parking situation of a parking lot as an image from the parking lot guide service providing server, and generating a user interface based on the received parking lot model and parking lot data, and displaying the generated user interface and the user interface may include at least one of a parking available location guide user interface, a vehicle parking location guide user interface, a parking lot route guide user interface, and a parking impact event guide user interface.

And, the available parking location guide user interface, based on the parking lot data, the user terminal device may be an interface for displaying the available parking location information of the parking lot located in the parking lot model.

In addition, the parking lot route guide user interface, based on the user's parking location information and the location information in the parking lot of the user terminal device may be an interface for displaying a route from the user's current location to the parking location in the parking lot model.

In addition, the vehicle parking location guide user interface may be an interface for displaying the user's parking location information on the parking lot model based on the parking lot data.

In addition, the parking impact event guide user interface may be an interface for displaying on the parking lot model shock information about an impact situation generated based on parking lot data of an image photographing device for a vehicle provided in another vehicle.

On the other hand, the user terminal device according to an embodiment of the present invention for achieving the above object connects to a parking lot guide service providing server that provides a parking lot guide service based on a display unit and a vehicle image photographing device, and the parking lot guide service A communication unit for receiving a parking lot model and parking lot data representing a real-time parking situation of a parking lot as an image from a providing server, and generating a user interface based on the received parking lot model and parking lot data, and displaying the generated user interface and a control unit for controlling the unit, wherein the user interface may include at least one of a parking available location guide user interface, a vehicle parking location guide user interface, a parking lot route guide user interface, and a parking impact event guide user interface.

And, the available parking location guide user interface, based on the parking lot data, the user terminal device may be an interface for displaying the available parking location information of the parking lot located in the parking lot model.

In addition, the parking lot route guide user interface, based on the user's parking location information and the location information in the parking lot of the user terminal device may be an interface for displaying a route from the user's current location to the parking location in the parking lot model.

In addition, the vehicle parking location guide user interface may be an interface for displaying the user's parking location information on the parking lot model based on the parking lot data.

In addition, the parking impact event guide user interface may be an interface for displaying on the parking lot model shock information about an impact situation generated based on parking lot data of an image photographing device for a vehicle provided in another vehicle.

On the other hand, in the computer-readable recording medium according to an embodiment of the present invention for achieving the above object, a program for executing the above-described method for providing a parking lot guidance service may be recorded.

In addition, the program stored in the recording medium according to an embodiment of the present invention for achieving the above object may include a program code for executing the above-described parking guide service providing method.

According to the present invention, it is possible to generate a parking lot model representing a real-time situation of a parking lot by using an image captured by an image photographing device for a vehicle, and provide a parking lot guidance service to a user using the generated 3D model.

In particular, according to the present invention, even if the parking control system is not provided in the parking lot, it is possible to provide a real-time parking status service.

In addition, according to the present invention, by providing a parking location guidance service such as a parking available area of a parking lot, the number of parking floors, and a parking space to a user who wants to park, it is possible to shorten the user's parking time and increase the parking convenience. .

In addition, according to the present invention, by providing a vehicle parking location guide service to a user who wants to find a parked car, it is possible to help the user to find a parked vehicle more conveniently.

In addition, according to the present invention, by providing a parking lot route guidance service from the parking position of the vehicle to the destination (eg, the exit of the parking lot, etc.), it is possible to increase the user's convenience.

In addition, according to the present invention, when an impact event occurs in a parked vehicle, by providing information about a parking impact based on an image captured by another vehicle nearby, more detailed information about the parking impact situation can be provided to the user. .

In addition, according to the present invention, by securing real-time image data using an image capturing device of a vehicle parked in a specific area, it is possible to provide a vehicle location confirmation service using a third party's vehicle image capturing device even without physically fixed CCTV. .

1 is a block diagram illustrating a parking lot guidance service system according to an embodiment of the present invention.
2 is a block diagram illustrating an image capturing apparatus for a vehicle according to an embodiment of the present invention.
3 is a block diagram illustrating in more detail a parking lot data generator according to an embodiment of the present invention.
4 is a diagram showing the configuration of a neural network according to an embodiment of the present invention.
5 is a view showing an image of a parking lot according to an embodiment of the present invention.
6 is a block diagram illustrating a parking lot guidance service providing server according to an embodiment of the present invention.
7 is a view showing a parking lot model according to an embodiment of the present invention.
8 is an exemplary view illustrating a parking shock event occurrence situation according to an embodiment of the present invention.
9 is a block diagram illustrating a user terminal device according to an embodiment of the present invention.
10 is a timing diagram illustrating a method of providing a parking lot guidance service according to an embodiment of the present invention.
11 is a timing diagram illustrating a method of providing a parking lot guidance service according to another embodiment of the present invention.
12 and 13 are diagrams illustrating a user interface according to an embodiment of the present invention.
14 is a timing diagram illustrating a method of providing a parking lot payment service according to another embodiment of the present invention.
15 is a block diagram illustrating an autonomous driving system for a vehicle according to an embodiment of the present invention.
16 is a block diagram of an autonomous driving system according to another embodiment of the present invention.
17 is a block diagram of a user terminal device according to another embodiment of the present invention.
18 is a block diagram of a service providing server according to another embodiment of the present invention.
19 is a diagram for explaining an operation flow of an autonomous parking system according to another embodiment of the present invention.
20 is a diagram for explaining an autonomous parking operation flow of a user terminal device according to another embodiment of the present invention.
21 is a diagram for explaining an autonomous parking operation flow of a service providing server according to another embodiment of the present invention.
22 is a diagram for explaining an operation flow for providing a vehicle call or passenger pickup service of an autonomous driving system according to another embodiment of the present invention.
23 is a diagram for describing a process in which an autonomous driving system of a vehicle performs autonomous parking according to another embodiment of the present invention.
24 is a diagram illustrating a UX screen displayed on a user terminal device when the autonomous parking system of a vehicle performs autonomous parking according to another embodiment of the present invention.
25 is a diagram illustrating an example of a push notification or a push message displayed on a user terminal device of a user using a vehicle autonomous parking service/vehicle call service according to another embodiment of the present invention.
26 is a diagram illustrating an example of a push notification or a push message displayed on a user terminal device of a user using an autonomous parking service of a vehicle according to another embodiment of the present invention.
27 is a view for explaining an example in which the autonomous parking service providing server identifies a parking space through deep learning analysis when an autonomous parking service of a vehicle is requested according to another embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices that, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. In addition, all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the concept of the present invention, and it should be understood that they are not limited to the specifically enumerated embodiments and states as such. do.

Moreover, it is to be understood that all detailed description reciting the principles, aspects, and embodiments of the invention, as well as specific embodiments, are intended to cover structural and functional equivalents of such matters. It should also be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

The functions of the various elements shown in the drawings including a processor or functional blocks represented by similar concepts may be provided by the use of dedicated hardware as well as hardware having the ability to execute software in association with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the clear use of terms presented as processor, control, or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile memory. Other common hardware may also be included.

In the claims of the present specification, a component expressed as a means for performing the function described in the detailed description includes, for example, a combination of circuit elements that perform the function or software in any form including firmware/microcode, etc. It is intended to include all methods of performing the functions of the device, coupled with suitable circuitry for executing the software to perform the functions. Since the present invention defined by these claims is combined with the functions provided by the various enumerated means and in a manner required by the claims, any means capable of providing the functions are equivalent to those contemplated from the present specification. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in the description of the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a parking lot guidance service system according to an embodiment of the present invention. Referring to FIG. 1 , the parking lot guidance service system 1000 includes a vehicle image capturing device 100 , a vehicle communication device 200 , a parking lot guidance service providing server 300 , a user terminal device 400 , and a base station 500 . include

The parking lot guidance service system 1000 generates a parking lot model representing a real-time situation for a parking lot by using the image captured by the vehicle image photographing device 100, and based on the generated parking lot model, the user's terminal device 400 We can provide parking guidance service.

Here, the parking lot may be a concept including both an indoor parking lot and an outdoor parking lot.

In addition, the parking lot may be composed of at least one or more floors, each floor may be composed of a plurality of parking spaces, and the parking space may be composed of a plurality of parking spaces.

In the present invention, the vehicle is an example of a moving body, and the moving body of the present invention is not limited to the vehicle. A moving body according to the present invention may include various movable objects such as a vehicle, a person, a bicycle, a ship, a train, and the like. Hereinafter, for convenience of description, a case in which the moving object is a vehicle will be described as an example.

The base station 500 is a wireless communication facility that connects a network and various terminals for a wireless communication service. ), it is possible to enable communication between the parking information service providing server 300 and the user terminal device 400 . For example, the vehicle communication device 200 may wirelessly connect to a communication network through the base station 500 , and when the vehicle communication device 200 is connected to the communication network, other devices connected to the network (eg, parking lot guide) Data may be exchanged with the service providing server 300 and the user terminal device 400 .

The vehicle image photographing apparatus 100 may be provided in a vehicle to photograph an image in a situation such as driving, stopping, or parking of the vehicle, and may store the photographed image.

Also, the vehicle image photographing apparatus 100 may be controlled by a user control input through the user terminal device 400 . For example, when the user selects an executable object installed in the user terminal device 400 , the vehicle image photographing apparatus 100 may perform operations corresponding to an event generated by a user input to the executable object. Here, the executable object may be a kind of application installed in the user terminal device 400 to remotely control the vehicle image capturing apparatus 100 .

In addition, in the present specification, an action that triggers the operation of the vehicle image capturing apparatus 100 will be defined as an event. For example, the type of the event may be shock detection, noise detection, motion detection, user gesture detection, user touch detection, and reception of a control command from a remote control. Here, the vehicle image photographing apparatus 100 includes a front photographing device for photographing the front of the vehicle, a rear photographing device for photographing the rear, a side photographing device for photographing left and right sides, a photographing device for photographing the face of the vehicle driver, and photographing the interior of the vehicle It may include all or part of the indoor photographing device.

In this specification, an infrared (Infra-Red) camera for a vehicle, a vehicle black box, a car dash cam, or a car video recorder are other expressions of the vehicle image photographing apparatus 100 and may all mean the same thing.

The vehicle communication device 200 is a device that is connected to the vehicle image photographing apparatus 100 to enable communication of the vehicle image photographing apparatus 100 , and the vehicle image photographing apparatus 100 is external through the vehicle communication device 200 . It can communicate with the server. Here, the vehicle communication device 200 may use various wireless communication connection methods, such as a cellular type mobile communication such as Long Term Evolution (LTE), and a Wireless Local Area Network (LAN) method such as WiFi (Wireless Fidelity).

Also, according to an embodiment of the present invention, the vehicle communication device 200 for performing wireless communication with a server may be implemented as a communication module using a low-power wide-area (LPWA) technology. Here, as an example of a low-power wide-area communication technology, a low-power wide-band wireless communication module such as Long Range (LoRa), Narrow Band-Internet of Things (NB-IoT), Cat M1, etc. may be used.

Meanwhile, the vehicle communication device 200 according to an embodiment of the present invention may also perform a location tracking function like a global positioning system (GPS) tracker.

Also, in FIG. 1 , the vehicle communication device 200 is an external type provided separately from the vehicle image photographing apparatus 100 as an example, but the present invention is not limited thereto, and the vehicle communication apparatus 200 is the vehicle image photographing apparatus 100 It may be implemented with a built-in communication module provided inside of

In the present specification, a dongle is another expression of the vehicle communication device 200 and may mean the same thing.

The parking lot guidance service providing server 300 may relay various data between the vehicle communication device 200 and the user terminal device 400 to enable a parking lot guidance service to be described later.

Specifically, the parking lot guidance service providing server 300 receives data including an image captured by the vehicle image photographing apparatus 100 and various information generated by the vehicle image photographing apparatus 100 from the vehicle communication device 200 . can

In addition, the parking lot guidance service providing server 300 may match the received data to the parking lot identification information and store it. Here, the parking lot identification information may refer to information that makes it possible to distinguish each of a plurality of parking lots, such as a parking lot ID, a parking lot name, a parking lot phone number, and a parking lot location.

In addition, the parking lot guidance service providing server 300 may generate a model of the parking lot representing the real-time situation of the parking lot as an image based on the received data, and based on the generated parking lot model, the user's terminal subscribed to the service Various data for providing a parking guide service to the device 400 may be transmitted.

Here, the parking lot guide service may include a parking space location guide service, a parking available location guide service, a vehicle parking location guide service, a parking lot route guide service, and a parking shock event guide service.

The available parking location guidance service may be a service for guiding a user who wants to park a parking space, such as a parking space of a parking lot, the number of parking available floors, and a parking space available.

In addition, the vehicle parking location guidance service may be a service for guiding a vehicle parking location to a user who wants to find a parked car.

In addition, the parking lot route guidance service may be a service for guiding a route from a parking location of the vehicle to a destination (eg, an exit of a parking lot, etc.).

In addition, the parking shock event guidance service may be a service that provides information about a parking shock based on an image captured by an adjacent vehicle when an impact event occurs in a parked vehicle.

The user terminal device 400 may display on the screen a user interface that provides various meaningful information based on the data received from the parking lot guidance service providing server 300 .

Specifically, the application according to the present invention (hereinafter, "parking lot guide service application") may be installed in the user terminal device 400, and the user executes the parking lot guide service application installed in the user terminal device 400, According to the execution of the application, a user interface may be configured and displayed on the screen based on various data received from the parking lot guidance service providing server 300 .

Here, the user interface may include a user interface corresponding to a parking available location guidance service, a user interface corresponding to a vehicle parking location guidance service, a user interface corresponding to a parking lot route guidance service, and a user interface corresponding to a parking shock event guidance service. can

Here, the user terminal device 400 is implemented as a smart phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), or the like, or smart glasses that can be worn on the user's body, and a head worn display (Head). It may be implemented as a wearable device such as a mounted display (HMD).

Here, the user is a vehicle owner, a vehicle driver, an owner of the vehicle image photographing apparatus 100 , a supervisor of the vehicle image photographing apparatus 100 , and/or a person having management authority for the vehicle and/or vehicle image photographing apparatus 100 . can

Hereinafter, with reference to the accompanying drawings, the image capturing apparatus 100 for a vehicle, the parking lot guidance service providing server 300 and the user terminal device 400 according to an embodiment of the present invention will be described in more detail.

The above-described parking information service providing server 300 according to the embodiment of the present invention has been described as determining a parking available location through analysis of an image obtained through the vehicle image photographing device 100 mounted on the vehicle, but the present invention In another embodiment of , the parking space is identified through deep learning analysis of the parking lot image acquired through a fixed image acquisition device such as CCTV (Closed Circuit Television) installed in the parking lot, and autonomous parking using the identified parking space The service may be provided to a user terminal device and/or an autonomous driving system.

2 is a block diagram illustrating an image capturing apparatus for a vehicle according to an embodiment of the present invention. Referring to FIG. 2 , the vehicle image photographing apparatus 100 includes a photographing unit 110 , a user input unit 120 , a microphone unit 130 , a display unit 140 , an audio unit 150 , a storage unit 160 , It may include an impact detection unit 170 , a parking lot data generation unit 175 , a vehicle driving support function unit 180 , a surrounding vehicle event determination unit 185 , a communication unit 190 , and a control unit 195 .

The photographing unit 110 may photograph an image in at least one of parking, stopping, and driving of the vehicle.

Here, the captured image may include a parking lot image that is a captured image about the parking lot. The parking lot image may include an image captured during a period from the time the vehicle enters the parking lot to the vehicle exits the parking lot. That is, the parking lot image is an image taken from the time the vehicle enters the parking lot to the time the vehicle is parked (ex. the time when the vehicle is turned off to park), the image taken during the parking period of the vehicle, and the time when the vehicle is parked (ex. It may include a captured image from the vehicle starting on (on) to the time of exiting the parking lot of the vehicle to get out.

And, the captured image may include at least one image of the front, rear, side, and interior of the vehicle.

In addition, the photographing unit 110 may include an infrared (Infra Red) camera capable of monitoring the driver's face or pupil, and the controller 195 monitors the driver's face or pupil through the infrared camera, thereby drowsiness of the driver. It is possible to determine the driver's state including whether the driver is driving or not.

The photographing unit 110 may include a lens unit and an imaging device. The lens unit may perform a function of condensing an optical signal, and the optical signal passing through the lens unit reaches the imaging area of the imaging device to form an optical image. Here, as the imaging device, a charge coupled device (CCD), a complementary metal oxide semiconductor image sensor (CIS), or a high-speed image sensor that converts an optical signal into an electrical signal may be used. In addition, the photographing unit 110 may further include all or a part of a lens unit driving unit, an iris, an iris driving unit, an imaging device control unit, and an image processor.

The user input unit 120 is configured to receive various user inputs for operating the vehicle image photographing apparatus 100, for example, a user input for setting an operation mode of the vehicle image photographing apparatus 100, and a recorded image to the display unit ( 140) may receive various user inputs, such as a user input for displaying in FIG.

Here, the operation mode of the vehicle image photographing apparatus 100 may include a regular recording mode, an event recording mode, a manual recording mode, and a parking recording mode.

The regular recording mode is a mode that is executed when the vehicle is started and driving is started, and the continuous recording mode may be maintained while the vehicle continues to be driven. In the regular recording mode, the vehicle image photographing apparatus 100 may record in a predetermined time unit (eg, 1 to 5 minutes). In the present invention, the regular recording mode and the regular mode may be used with the same meaning.

The parking recording mode may refer to a mode operated in a parked state because the engine of the vehicle is turned off or the supply of a battery for driving the vehicle is stopped. In the parking recording mode, the vehicle image photographing apparatus 100 may operate in a parking regular recording mode for performing regular recording while parking. Also, in the parking recording mode, the vehicle image photographing apparatus 100 may operate in a parking event recording mode for recording when an impact event is detected while parking. In this case, recording (eg, recording 10 seconds before to 10 seconds after the occurrence of the event) may be performed for a predetermined period from a predetermined time before the occurrence of the event to after a predetermined time. In the present invention, the parking recording mode and the parking mode may be used as the same meaning.

The event recording mode may mean a mode that is operated when various events occur while the vehicle is driving.

For example, when an impact event is detected by the impact detection unit 170 or an Advanced Driving Assistance System (ADAS) event is detected by the vehicle driving support function unit 180 , the event recording mode may operate.

In the event recording mode, the vehicle image photographing apparatus 100 may record time from before a predetermined time to after a predetermined time (eg, recording 10 seconds before to 10 seconds after the occurrence of the event).

The manual recording mode may mean a mode in which a user manually operates recording. In the manual recording mode, the vehicle image photographing apparatus 100 may perform recording (eg, recording 10 seconds before to 10 seconds after an event occurrence) from a predetermined time before a user's manual recording request to a predetermined time after.

Here, the user input unit 120 may be configured in various ways to receive a user input, such as a keypad, a dome switch, a touch pad, a jog wheel, and a jog switch.

The microphone unit 130 may receive a sound generated outside or inside the vehicle. Here, the received sound is a sound caused by an external shock or a human voice related to a situation inside/outside the vehicle, and may help to recognize the situation at the time together with the image captured by the photographing unit 110 . The sound received through the microphone unit 130 may be stored in the storage unit 160 .

The display unit 140 may display various types of information processed by the vehicle image capturing apparatus 100 . For example, the display unit may display a “live view image” that is an image captured in real time by the photographing unit 110 , and may display a setting screen for setting an operation mode of the vehicle image photographing apparatus 100 .

The audio unit 150 may output audio data received from an external device or stored in the storage unit 140 . Here, the audio unit 150 may be implemented as a speaker outputting audio data. For example, the audio unit 150 may output audio data indicating that a parking event has occurred.

The storage unit 160 stores various data and programs necessary for the operation of the vehicle image capturing apparatus 100 . In particular, the storage unit 160 may store the image captured by the photographing unit 110 , voice data input through the microphone unit 130 , and parking data generated by the parking lot data generation unit 175 .

In addition, the storage unit 160 may classify and store data obtained according to the operation mode of the vehicle image photographing apparatus 100 in different storage areas.

The storage unit 160 may be configured inside the vehicle image photographing apparatus 100 , or may be detachably configured through a port provided in the vehicle image photographing apparatus 100 , or may exist outside the vehicle image photographing apparatus 100 . When the storage unit 160 is configured inside the vehicle image capturing apparatus 100, it may exist in the form of a hard disk drive or a flash memory. When the storage unit 160 is detachably configured in the vehicle image photographing apparatus 100, it may exist in the form of an SD card, a micro SD card, a USB memory, or the like. When the storage unit 160 is configured outside the vehicle image capturing apparatus 100 , it may exist in another device or a storage space in a database server through the communication unit 190 .

The impact sensor 170 may detect an impact applied to the vehicle or detect a case in which an amount of change in acceleration is greater than or equal to a certain level. Here, the impact sensing unit 170 may include an acceleration sensor, a geomagnetic sensor, and the like to detect an impact or acceleration.

The vehicle driving support function unit 180 may determine whether the driver of the vehicle needs the driving assistance function based on the driving image captured by the photographing unit 110 .

For example, the vehicle driving support function unit 180 detects the departure of a vehicle located in front of the vehicle based on the driving image captured by the photographing unit 110 , and whether a forward vehicle start alarm (FVSA) is required to the driver. can be judged If a predetermined time elapses after the vehicle in front departs, the vehicle driving support function unit 180 may determine that the vehicle in front departure notification is necessary.

In addition, the vehicle driving support function unit 180 may detect whether a signal is changed based on the driving image captured by the photographing unit 110 and determine whether a traffic light change alarm (TLCA) is required to the driver. there is. For example, when the stop state (0 km/h) is maintained for 4 seconds in a state in which the signal is changed from a stop signal to a straight signal, the vehicle driving support function unit 180 may determine that a traffic light change notification is necessary.

In addition, the vehicle driving support function unit 180 detects whether the vehicle is deviating from a lane based on the driving image captured by the photographing unit 110 and determines whether a lane departure warning system (LDWS) is required for the driver. can do. For example, when the vehicle deviates from a lane, the vehicle driving support function unit 180 may determine that a lane departure warning is necessary.

In addition, the vehicle driving support function unit 180 detects a risk of collision with a vehicle in front of the vehicle based on the driving image captured by the photographing unit 110 and determines whether a Forward Collision Warning System (FCWS) is required for the driver. can determine whether For example, when detecting the initial risk of forward collision, the vehicle driving support function unit 180 determines that the first forward collision warning is necessary, and after detecting the initial risk of forward collision detection, the second time when the distance with the vehicle in front is further reduced It may be determined that forward collision warning is necessary.

Here, the forward collision warning may further include an urban forward collision warning (uFCWS: urban FCWS) that provides a forward collision warning at a lower driving speed suitable for an environment with a low driving speed.

On the other hand, the parking lot data generation unit 175 generates parking lot data during the period from the time the vehicle enters the parking lot (in other words, the entry point) to the time the vehicle leaves the parking lot (in other words, the entry point). can

Here, the parking lot data may include at least one of parking lot location information, parking space information, parked vehicle information, own vehicle location information, time information, and a parking lot image.

Specifically, referring to FIG. 3 , the parking lot data generation unit 175 includes a parking lot location information generation unit 175-1, a parking space information generation unit 175-2, a parking vehicle information generation unit 175-3, It may include an own vehicle location information generating unit 175-4 and an artificial intelligence (AI) processing unit 175-5.

The parking lot location information generating unit 175-1 may determine the location of the parking lot and generate parking lot location information. For example, when the vehicle is located in an outdoor parking lot, the parking lot location information generator 175-1 may generate location information of the outdoor parking lot by using satellite positioning data. As another example, when the vehicle is located in the indoor parking lot, the parking lot location information generating unit 175-1 generates location information of the indoor parking lot based on the last reception point of the satellite positioning data or the base stations of the cellular network located in the indoor parking lot. It is possible to generate location information of an indoor parking lot based on the used positioning information or generate location information of an indoor parking lot based on positioning information using access points of a WiFi network located in the indoor parking lot.

The parking space information generating unit 175 - 2 may include location information of a parking space for a parking space included in the parking lot image, and information about a parking space of the parking space. Here, the parking space information may identify and extract a parking space existing in a parking space from the parking lot image, and may include information on the number of parking spaces, parking space identification information, and whether or not the parking space is parked. The parking space information generating unit 175-2 according to an embodiment of the present invention detects an edge in an image included in the parking lot image or detects a feature point or a deep learning result for the marked lines of the parking space, deep learning for the added vehicle The result can be used to identify parking lot information.

Specifically, the parking space information generator 175 - 2 may generate location information of the parking space included in the parking lot image based on the location identifier included in the parking lot image.

Here, the location identifier is information that is included in the parking lot image to make it possible to identify the location of the parking space in the parking lot, text (eg, text such as "parking lot entrance", "3rd floor", "3B-2", etc.), It may include at least one of a structure (eg, a parking blocker, a parking toll booth, etc.) and a unique identification symbol with a defined location (eg, a specific QR code, a specific sticker, and a specific text).

That is, the parking space information generator 175 - 2 may generate location information of the parking space included in the parking lot image based on the location identifier recognized through the analysis of the captured image. For example, when the location identifiers of "3B-1" and "3B-2" are marked on both side pillars of the parking space, the parking space information generating unit 175-2 uses the " Can be created as a parking space between "3B-1" and "3B-2".

In this case, the parking space information generating unit 175-2 may recognize the location identifier using the learned neural network of the AI processing unit 175-5 to calculate a prediction result of the presence or absence of the location identifier in the captured image. . An example of such an artificial neural network will be described in more detail with reference to FIG. 4 .

4 is an exemplary diagram illustrating the configuration of a neural network according to an embodiment of the present invention.

Referring to FIG. 4 , the neural network 30 according to the present embodiment may be configured as a Convolution Neural Network (CNN) model composed of layers that perform a plurality of convolution operations.

When the parking lot image 12 is input to the neural network 30, the characteristic values according to the unique shape or color indicating the location identifier included in the parking lot image while passing through the layers inside the neural network 30 can be emphasized through convolution. .

As a location identifier, various feature values included in the parking lot image are output in the form of a new feature map through operation with a filter determined for each convolution layer, and the final feature map generated through iterative operation for each layer is It can be input as a fully-connected layer and flattened. A difference between the flattened feature information and the reference feature information defined for each location identifier may be calculated, and the existence probability of the location identifier may be output as the prediction result 32 according to the calculated difference.

In this case, in order to increase the accuracy, the parking lot image may be divided and input to the neural network 30 . As an example, since the location identifier is generally displayed in a non-parking space (eg, a pillar, etc.) rather than a parking space in which the vehicle is parked, according to the present invention, only the non-parking space image is divided from the parking lot image to the neural network 30 can also be entered in

The training of the neural network 30 may be performed using a training data set classified for each location identifier as labeling data including parking lot image data and a result of determining whether a location identifier exists. For example, a neural network may be trained using a plurality of driving image data labeled as having a specific QR code, a specific sticker, etc. as a location identifier in the parking lot image, respectively.

The learned neural network 30 may determine whether a location identifier exists with respect to the input parking lot image, and may provide a prediction probability value for each location identifier as the prediction result 32 .

That is, FIG. 5 is a view showing a parking lot image according to an embodiment of the present invention. Referring to FIG. 5 , the parking lot image captured by the vehicle image photographing apparatus 100 of a parked vehicle is a plurality of location identifiers 501 . and a plurality of vehicles 502 .

In the present invention, the AI processing unit 175-5 may recognize the location identifier 501 from the parking lot image using an artificial neural network.

Meanwhile, the parking space information generating unit 175 - 2 may analyze the parking lot image to generate information on the number of parking spaces of the parking spaces included in the parking lot image. Specifically, the parking space information generating unit 175-2 detects line marking of the parking space, generates information on the number of parking spaces in the parking space based on the detected line, and distinguishes each of the plurality of parking spaces. It is possible to generate parking lot identification information that makes it possible.

Also, the parking space information generating unit 175 - 2 may generate information on whether or not a parking space included in the parking space is parked. Specifically, the parking space information generating unit 175-2 analyzes the parking lot image to detect a vehicle, determines where the detected vehicle is located among a plurality of parking spaces constituting the parking space, and determines a parking space included in the parking space. of parking information can be generated. Here, the information on whether the parking lot is parked may be generated for each of a plurality of parking spaces constituting the parking space, and may be generated separately for each parking lot floor.

In this case, the parking space information generating unit 175-2 may recognize the vehicle from the main magnetic field image using the learned neural network of the AI processing unit 175-5. In this regard, since the principle is the same as that of the neural network 30 trained to recognize the location identifier in FIG. 4 , a detailed description thereof will be omitted.

That is, FIG. 5 is a view showing a parking lot image according to an embodiment of the present invention. In the present invention, the AI processing unit 175-5 may recognize a plurality of vehicles 502 from the parking lot image using an artificial neural network. .

Meanwhile, the parking vehicle information generating unit 175-3 may analyze the parking lot image to generate information on a plurality of parked vehicles parked around the own vehicle. Here, the parking vehicle information may include vehicle type information and vehicle number information. In addition, the parking vehicle information may be generated by being classified for each of a plurality of parking spaces constituting the parking space.

The vehicle type information may include classification information according to the purpose of using the vehicle, such as a sedan, hatchback, wagon, SUV, and the like, and classification information for each vehicle brand. And, the vehicle number information may be number information written on the vehicle number plate.

In this case, the parking vehicle information generating unit 175-3 may use the learned neural network of the AI processing unit 175-4 to recognize surrounding vehicles from the captured image. Accordingly, the learned neural network 30 may determine whether a vehicle exists with respect to the input parking lot image, and provide a type for each vehicle, a number for each vehicle, and the like as a prediction result.

Meanwhile, the own vehicle location information generator 175 - 4 may generate location information of a vehicle on which the vehicle image photographing apparatus 100 is mounted.

Specifically, when the vehicle is located in the outdoor parking lot, the own vehicle location information generator 175-4 uses satellite positioning data received from a Global Navigation Satellite System (GNSS) to locate the own vehicle in the outdoor parking lot. information can be created.

Also, when the vehicle is located in the indoor parking lot, the own vehicle location information generating unit 175-4 may generate location information of the own vehicle in the indoor parking lot by using the above-described location identifier.

However, the present invention is not limited thereto, and according to another embodiment of the present invention, even when the vehicle is located in the outdoor parking lot, location information of the own vehicle in the outdoor parking lot may be generated using the location identifier.

Also, the own vehicle location information generating unit 175 - 4 may generate location information of the own vehicle based on whether the own vehicle is parked. Here, the own vehicle location information generating unit 175-4 determines whether the own vehicle is parked, the ignition of the own vehicle is turned off, the power of the battery is turned off, the parking (P stage) gear shift, whether the passenger gets out of the vehicle, and the The determination may be made based on at least one of a location (when the vehicle key is located outside the vehicle), whether the side mirror is folded, and whether the user terminal device 400 is connected to the vehicle via Bluetooth.

For example, when the vehicle's parking gear shift and the user terminal device 400 and the Bluetooth connection of the vehicle are disconnected, the own vehicle location information generating unit 175-4 determines that the vehicle is parked at the location of the corresponding own vehicle It is possible to generate vehicle location information.

On the other hand, when parking lot location information, parking space information, surrounding parking vehicle information, and own vehicle location information are generated according to the above-described process, the parking lot data generation unit 175 provides time information matching the generated information and the generated information. By combining matching parking lot images, parking lot data can be generated.

Meanwhile, the surrounding vehicle event determining unit 185 may determine whether an event of another vehicle parked in the vicinity of the own vehicle has occurred. Here, the surrounding vehicle event may mean an event situation in which an impact is applied to another vehicle parked in the vicinity of the own vehicle by a vehicle, a person, or an object.

The surrounding vehicle event determination unit 185 may determine whether an event of another vehicle parked in the vicinity of the own vehicle has occurred based on a sound, a motion of a front object, and the like.

For example, when a scream, an impact sound, a tire sound, a conversation sound including a specific word, etc. is input from the microphone unit 130 , the surrounding vehicle event determination unit 185 generates an event of another vehicle parked in the vicinity of the own vehicle. It can be judged that

Alternatively, the nearby vehicle event determining unit 185 may determine whether a nearby vehicle event has occurred according to a remote request. For example, when an impact event is detected in another vehicle parked around the own vehicle and the vehicle image photographing device 100 mounted on the other vehicle transmits an impact notification of the parking lot guidance service providing server 300 or the user terminal device 400 , , the parking lot guidance service providing server 300 or the user terminal device 400 may notify the occurrence of an event to the vehicle image capturing apparatus 100 of the own vehicle located near the vehicle in which the impact has occurred. And, when the notification is received, the surrounding vehicle event determining unit 185 may recognize that an event has occurred in the surrounding vehicle.

On the other hand, the communication unit 190 may enable the vehicle image capturing apparatus 100 to communicate with other devices, where the communication unit 190 is a cellular type mobile communication such as LTE (Long Term Evolution), WiFi (Wireless Fidelity) ) can be implemented with various known communication modules, such as various wireless communication connection methods such as a Wireless LAN (Local Area Network) method, and a communication module using Low-Power Wide-Area (LPWA) technology. In addition, the communication unit 190 may also perform a location tracking function like a global positioning system (GPS) tracker.

Accordingly, the vehicle image capturing apparatus 100 may communicate with the parking lot guidance service providing server 300 and/or the user terminal device 400 through the communication unit 190 .

Here, the communication unit 190 may refer to the same thing as the vehicle communication device 200 of FIG. 1 .

The controller 195 controls the overall operation of the vehicle image capturing apparatus 100 . Specifically, the control unit 195 includes a photographing unit 110 , a user input unit 120 , a microphone unit 130 , a display unit 140 , an audio unit 150 , a storage unit 160 , and an impact sensing unit 170 . , the parking lot data generating unit 175 , the vehicle driving support function unit 180 , the surrounding vehicle event determining unit 185 , and the communication unit 190 may be all or partly controlled.

In particular, the control unit 195 based on at least one of whether the vehicle is started, a vehicle battery voltage measurement result, a detection result of the shock sensor 180 , a determination result of the vehicle driving support function unit 180 , and an operation mode setting value The operation mode of the vehicle image processing apparatus 100 may be set to one of a regular recording mode, an event recording mode, a parking recording mode, and a manual recording mode. Also, when the battery voltage of the vehicle falls below a threshold value, the controller 195 may control to stop the operation of the vehicle image capturing apparatus 100 .

In addition, the control unit 195 determines whether an update of the parking lot data is necessary, and when an update is required, controls the parking lot data generator 175 to update the parking lot data, and transmits the updated parking lot data to the parking lot information service providing server 300 ) can control the communication unit 190 to transmit.

Here, the update condition of the parking lot data includes a case in which a change occurs in the parking lot image as a parking vehicle located near the own vehicle exits from a parking slot of the parking space or another vehicle enters the parking space of the parking space. can

In addition, the update condition of the parking lot data may include a case where a preset period has arrived.

In addition, the update condition of the parking lot data may include a case where the completeness of the parking data is lower than a preset reference value. Here, the case where the degree of completeness is lower than the preset reference value may include a case where the resolution of the parking lot image is low or there is incomplete data.

In addition, the update condition of the parking lot data may include a case in which an update request from a remote (eg, the parking lot guide service providing server 300 or the user terminal device 400) is received. Here, the remote update request may be performed by determining the need for update in the parking lot guide service providing server 300 or the user terminal device 400 based on the above-described update condition of the parking lot data.

In addition, when it is determined that a surrounding vehicle impact event has occurred in the surrounding vehicle impact event determination unit 185, the control unit 195 controls the parking lot data generation unit 175 to update the parking lot data, and guides the updated parking lot data to the parking lot The communication unit 190 may be controlled to transmit to the service providing server 300 . Here, the updated parking lot data may include data before and after a predetermined time of the occurrence of the surrounding vehicle impact event.

6 is a block diagram illustrating a parking lot guidance service providing server according to an embodiment of the present invention. Referring to FIG. 6 , the parking lot guidance service providing server 300 may include a communication unit 310 , a parking lot model generation unit 320 , a storage unit 330 , and a control unit 340 .

Before explaining FIG. 6 , according to the above-described vehicle image photographing apparatus 100 , the operation of the parking lot data generating unit 175 and the surrounding vehicle event determining unit 185 is performed in the vehicle image photographing apparatus 100 . Although described as an example, all or part of these operations may be performed in the parking lot guidance service providing server 300 .

The communication unit 310 may be provided for the parking lot guidance service providing server 300 to communicate with other devices. Specifically, the communication unit 310 may transmit/receive data to/from at least one of the vehicle image capturing apparatus 100 and the user terminal device 400 . Here, the communication unit 310 may be implemented with various known communication modules.

The parking lot model generator 320 may generate a parking lot model representing a real-time situation of the parking lot as an image by using the parking lot data received from the vehicle image capturing apparatus 100 .

Specifically, the parking lot model generation unit 320 may perform modeling for the parking lot by using the parking space information and the surrounding parking vehicle information of the parking data received from the vehicle image photographing apparatus 100, and for each floor of the parking lot. Each can be modeled.

That is, the parking lot model generation unit 320 determines the location of the corresponding parking space in the parking lot based on the location information of the parking space, and performs modeling of the parking space for the parking space based on the information on the number of parking spaces in the parking space. can In addition, the parking lot model generation unit 320 may determine whether to arrange the vehicle model in the corresponding parking lot based on the information on whether the parking lot is parked.

In addition, the parking lot model generation unit 320 may generate a vehicle model reflecting the license plate and vehicle type based on the type information of the parking vehicle and the number information of the parking vehicle, and arrange the generated vehicle model in the corresponding parking space. .

In addition, the parking lot model generation unit 320 analyzes the parking lot image received from the vehicle image photographing device 100 to generate at least one of spatial shape information and road surface information, and to generate a parking lot model based on the generated information. can

Here, the space shape information may mean information about the shape of a structure in the parking lot, such as a wall, a pillar, a parking space, a parking barrier, and the like. In addition, the spatial shape information may further include color information of the corresponding structure.

In addition, the road surface mark is an indicator for guiding the movement of the vehicle in the parking lot, and may include the direction of the vehicle's passage. Here, the road surface mark may determine the direction of the route by referring to the road surface mark when guiding a vehicle route in the parking lot.

Such spatial shape information and road surface information may be generated by the vehicle image capturing apparatus 100 and transmitted to the server 300 or may be generated by the server 300 through image processing of the parking lot image.

In addition, the parking lot model generation unit 320 analyzes the parking lot image of the parking lot data received from the vehicle image photographing apparatus 100, and the parking lot data and the parking lot model generation unit 320 received from the vehicle image photographing apparatus 100. If there is a difference by comparing parking lot data generated through image analysis of

Meanwhile, the parking lot model generation unit 320 may hold a basic parking lot model for each of the plurality of parking lots. Here, the basic parking lot model is a model in which the real-time parking situation of the corresponding parking lot is not reflected, and may be a model in which a wall, a pillar, a parking space, etc. indicating the spatial shape of the corresponding parking lot are reflected. In this case, the parking lot model generator 320 may generate the parking lot model by updating the basic parking lot model using the parking lot data received from the vehicle image photographing apparatus 100 .

The parking lot model generated by the parking lot model generator 320 may be a 3D-shaped model. This will be described in more detail with reference to FIG. 7 .

7 is a view showing a parking lot model according to an embodiment of the present invention. Referring to FIG. 7 , the parking lot model generation unit 320 models a parking space for a parking space based on information on the number of parking spaces in the parking space, and creates a vehicle model in the corresponding parking space based on whether or not the parking space is parked. It is possible to determine whether to arrange the parking lot model in which the vehicle model is placed.

In addition, the parking lot model generation unit 320 may generate a parking lot model by reflecting the entrance and exit management equipment and road markings disposed in the entrance and exit passages of the parking lot.

This parking lot model may be transmitted in a format that can be expressed to the user terminal device 400 and displayed on the screen of the user terminal device 400 .

Meanwhile, the parking lot model generator 320 may continuously receive parking lot data from the vehicle image capturing apparatus 100 to update the parking lot model.

In this case, the parking lot model generator 320 may update the parking lot model based on the parking lot location information and the parking space location information included in the parking lot data.

For example, when the parking lot model generation unit 320 receives parking lot data for a parking space between “3B-1” and “3B-2” of the first parking lot from the first vehicle image photographing device 100-1, , by using the received parking data to perform modeling in the corresponding parking space, and to generate a parking lot model. After that, when the parking lot data for the parking space between "3B-1" and "3B-2" of the same first parking lot is received from the second vehicle image photographing apparatus 100-2, the second vehicle image photographing apparatus ( Using the parking lot data received from 100-2), modeling of the corresponding parking space may be performed, and the previously generated parking lot model may be updated.

At this time, the parking lot model generator 320 may update the parking lot model by reflecting the latest parking lot data in the chronological order of the received parking lot data.

Also, when the parking lot model is updated, the parking lot model generation unit 320 may extract only a difference part between the previously generated parking lot model and a later generated parking lot model, and then update the parking lot model by reflecting only the corresponding difference part.

Through this, a parking lot model representing the entire interior of the parking lot can be created, and changes in the parking lot can be quickly reflected in the parking lot model.

The storage unit 330 may store various data and programs for the operation of the parking lot guide service providing server 300 . Here, the storage unit 330 may include a service subscription information storage unit 331 , a parking lot model storage unit 332 , and a parking lot data storage unit 333 .

Specifically, when a user who wants to be provided with a parking lot guide service subscribes to the service using his or her terminal device 400, the service subscription information storage unit 331 is generated based on information input through membership registration. You can store service subscription information. Here, the service subscription information storage unit 331 may store information about a subscriber who has signed up for a parking lot guide service, and device information of the corresponding subscriber. The subscriber information may include subscriber identification information and subscription service information.

The subscription service information is information that details the service subscribed by the corresponding subscriber, and may include service application details, rate plan, service validity period, data rate, service type, and the like.

The subscriber identification information is information that enables identification of each of a plurality of subscribers, and may include a subscriber's ID, a subscriber's password, a subscriber's resident registration number, a subscriber's name, a subscriber's nickname, a subscriber's personal identification number (PIN), and the like.

In addition, the subscriber device information may include at least one of identification information of the vehicle image photographing apparatus 100 purchased by the corresponding subscriber and identification information of the vehicle communication apparatus 200 . Here, the identification information of the vehicle image photographing apparatus 100 is information that enables identification of each of a plurality of vehicle image photographing apparatuses, and may include a model name of the vehicle image photographing apparatus, a unique serial number of the vehicle image photographing apparatus, and the like. In addition, the identification information of the vehicle communication device 200 is information that enables identification of each of a plurality of vehicle communication devices, including a dongle model name, a dongle phone number, a dongle serial number, a USIM (Universal Subscriber Identity Module) serial number, etc. can

In addition, the subscriber device information may further include identification information of the terminal device 400 of the subscriber, and the identification information of the user terminal device 400 is IMSI, which is unique information given by the network to identify the user terminal device 400 . (International Mobile Subscriber Identity), ICCID (Integrated Circuit Card ID), and IMEI (International Mobile Equipment Identity) may be included.

In this case, the service subscription information storage unit 331 may match and store subscriber information and subscriber device information for each subscriber who has subscribed to the service.

Meanwhile, the parking lot model storage unit 332 may store the parking lot model generated by the parking lot model generation unit 320 .

In addition, the parking lot data storage unit 333 may store the parking lot data received from the vehicle image photographing apparatus 100 .

In this case, the parking lot model storage unit 332 and the parking lot data storage unit 333 may match and store the parking lot model and the corresponding parking lot data.

Specifically, the parking lot model storage unit 332 may match and store the parking lot model and the corresponding parking lot location information, parking space information, surrounding parking vehicle information, own vehicle location information, time information, and parking lot image.

Here, the storage unit 330 may be implemented as a built-in module of the server 300 or may be implemented as a separate database (Data Base: DB) server.

Meanwhile, the controller 340 may control the overall operation of the parking lot guide service providing server 300 so that the parking lot guide service according to the present invention is provided.

The operation of the parking information service providing server 300 is "new subscription process", "black box registration process", "user registration process" and "parking information service providing" that provides parking information service to subscribers who have subscribed to the service process" can be distinguished.

In the "new subscription process", when a service member subscription is requested from a subscriber, the control unit 340 initiates a service subscription procedure, acquires information about a subscriber who has signed up for a parking lot information service and device information of the corresponding subscriber, and uses the acquired information It can be controlled to classify and store in the storage unit 330 . Accordingly, the storage unit 330 may build a service subscriber information database.

When performing the "registration process of the vehicle image capturing device", the control unit 340 communicates with the vehicle communication device 200 to communicate with a communication device such as a Universal Subscriber Identity Module (USIM) chip embedded in the vehicle communication device 200 . It is possible to receive unique information for identifying , and check the validity of the communication device 200 that has requested registration by comparing it with information previously stored in the storage unit 330 .

Similarly, in the "user registration process", when the user terminal device 400 accesses the server 300, the control unit 340 obtains user identification information such as USIM embedded in the user terminal device 400, By comparing the information pre-stored in the storage 330 with the acquired user identification information, whether the user terminal device 400 has subscribed to a service, a service type, etc. may be checked. If the authentication for the user is successfully completed, the controller 340 may provide various types of information about the vehicle imaging apparatus 100 in various UX forms based on the authority assigned to the user.

In the "parking guide service providing process", when the user terminal device 400 accesses the parking lot guide service providing server 300, the control unit 340 in the parking lot where the user's vehicle of the connected user terminal device 400 is parked. After detecting the parking lot model and parking lot data for the user terminal device 400 , a parking lot guidance service may be provided. Here, the parking lot guide service may include a parking available location guide service, a vehicle parking location guide service, a parking lot route guide service, and a parking lot payment service.

For example, when providing a parking available location guidance service, the control unit 340 detects information on the parking lot entered by the user upon entering the parking lot based on the location information of the user terminal device 400, and the parking lot data storage unit 333 Based on the parking lot data stored in the parking lot, it is possible to detect the number of parking floors of the corresponding parking lot, the position of the parking space on each floor, the position of the parking space in the available parking space, the number of parking spaces in the available parking space, and the like. And, the control unit 340 displays the available parking space of the parking lot, the number of parking floors, and the parking available positions such as a parking space in the parking lot model to the terminal device 400 of the user who wants to park based on the detected information. Parking available location guidance service may be provided.

As another example, when providing a vehicle parking location guidance service, the control unit 340 detects the user's parking location information on the basis of the parking lot data stored in the parking lot data storage unit 333, and detects the parking location information of the user in the parking lot model. It is possible to provide a vehicle parking location guidance service displaying the detected parking location information.

In addition, the parking information service providing server 300 may determine the location information in the parking lot of the user terminal device 400, in this case, the control unit 340 of the user's parking location information and the user terminal device (400) Based on the location information in the parking lot, it is possible to provide a vehicle parking location guidance service that displays the optimal moving route and distance from the user's current location to the parking location on the parking lot model. Here, the route may be displayed in the shape of an arrow in consideration of the direction of passage in the parking lot. As an example, the user terminal device 400 may display a user interface for a vehicle parking location guidance service as shown in FIG. 13B .

As another example, when providing a parking lot route guidance service, the control unit 340 detects the parking location information of the user terminal device 400 based on the parking lot data stored in the parking lot model storage unit 332, and It is possible to provide a parking lot route guidance service that detects exit information and displays a route and distance from the parking location of the user terminal device 400 to the parking lot exit on the parking lot model based on the detected information. Here, the route may be displayed in the shape of an arrow in consideration of the direction of passage in the parking lot. As an example, the user terminal device 400 may display a user interface for a parking lot route guidance service as shown in FIG. 13( a ).

In addition, the parking lot guide service may further include a parking shock event guide service. Here, the parking impact event guidance service will be described in more detail with reference to FIG. 8 .

8 is an exemplary view showing the occurrence of a parking shock event according to an embodiment of the present invention. Referring to FIG. 8 , a first vehicle (a) parked in a parking lot may have an impact due to a collision with another vehicle (c) parked next to it.

In this case, the surrounding vehicle event determining unit 185 of the second vehicle (b) may determine that a parking shock event has occurred in the first vehicle (a) based on a sound, a motion of a front object, and the like.

Alternatively, the impact detection unit 170 of the first vehicle (a) detects an impact from a collision of another vehicle (c), and the vehicle image photographing device 100 of the first vehicle (a) provides a parking lot guidance service server ( 300) or the user terminal device 400 of the first vehicle (a) may be notified of the occurrence of the impact event. In this case, the parking lot guidance service providing server 300 or the user terminal device 400 is a vehicle located around the first vehicle (a), for example, the second vehicle (b) (ie, the vehicle photographing the first vehicle). may notify the occurrence of the event to the vehicle image capturing apparatus 100 of , and the surrounding vehicle event determination unit 185 of the second vehicle (b) may recognize that the event has occurred in the first vehicle (a).

On the other hand, when it is recognized that a parking shock event has occurred in the first vehicle (a), the vehicle image photographing apparatus 100 of the second vehicle (b) uses the parking data generated by the parking lot data generator 175 to provide a parking guidance service. It can be transmitted to the providing server 300 . In this case, the parking lot guidance service providing server 300 may provide a parking shock event guidance service. Specifically, the control unit 340 may detect the license plate of the vehicle (c) that caused the impact to the first vehicle (a) from the parking lot image of the parking data. In addition, the control unit 340 may detect the location information of the parking lot where the impact occurred, the number of floors information, the location information of the parking space, and the location information of the parking space from the parking data. In addition, the control unit 340 may provide a parking shock event guidance service that guides the number of the vehicle generating the shock, the position of the shock, and the like to the parking lot model based on the detected information.

In addition, the parking lot guidance service may further include a guidance service before entering the parking lot. That is, when providing the guidance service before entering the parking lot, the control unit 340 uses the location information of the user terminal device 400 to the user terminal device ( 400), it is possible to detect the parking data of the parking lot located in the vicinity. And, the control unit 340 detects the parking space information of the corresponding parking lot from the detected parking data, and based on the detected information, a guide service before entering the parking lot to display the number of parking spaces of the corresponding parking lot and the fee for the corresponding parking lot. may be provided to the user terminal device 400 .

In this case, the user terminal device 400 may display a guide user interface before entering the parking lot as shown in FIG. 12 . That is, referring to FIG. 12 , the user terminal device 400 may display a user interface including the number of available parking spaces of the corresponding parking lot and the fee of the corresponding parking lot. Additionally, the user interface may display the parking lot entry direction with an arrow in consideration of the location of the corresponding parking lot.

Meanwhile, the controller 340 may provide various services to the user terminal device 400 by analyzing the parking lot model built by the parking lot model generator 320 .

As an example, the control unit 340 generates the total number of parking spaces in the parking lot, the degree of congestion, the main congestion time, real-time remaining parking space information, and my car parking location information based on the parking lot model built in the parking lot model generation unit 320, The generated information may be matched to the parking lot model and stored in the storage unit 330 .

At this time, the controller 340 may calculate the congestion level by comparing the value obtained by dividing the number of occupied parking spaces in the parking lot by the total number of parking spaces in the parking lot with a preset value. , it can be determined that the 30-60% range is medium congestion, and the 60-100% range is high congestion. Then, the controller 340 may calculate the main congestion time of the corresponding parking lot based on the calculated congestion level and time information at that time.

In addition, the control unit 340 may generate parking fee information, parking lot operating hours, electric vehicle charging station information, etc., match the generated information to the parking lot model, and store it in the storage unit 330 . Here, the electric vehicle charging station information may include whether the parking lot has an electric vehicle parking space, the number of electric vehicle parking spaces, electric vehicle charging rates, electric vehicle charging station operating hours, and the like.

In this case, the control unit 340 may provide the total number of parking spaces, congestion level, main congestion time, charge information, operating hours, electric vehicle charging station information, and the like to the user terminal device 400 connected to the server 300 .

In addition, when it is determined that the parking location is outdoors based on the parking location information in which the vehicle is parked, the controller 340 determines whether the parking location is a parking lot on the back road based on the analysis and/or location information of the captured image data and/or It may be determined whether it is an on-street parking space, and the determination result may be stored in the storage unit 330 . In this case, the controller 340 may provide to the user terminal device 400 connected to the server 300 whether the user's parking location is a back road parking lot and/or a street parking space.

Also, the controller 340 may analyze a commercial area located within a predetermined distance range based on the location of the parking lot in which the vehicle is parked. Specifically, the control unit 340 is based on the type of shopping malls located within a predetermined distance range based on the location of the parking lot in which the vehicle is parked (eg, restaurants, PC rooms, auto repair shops, etc.), the rent of the shopping malls, the maintenance period of the shops, etc. You can analyze the trends in the commercial area. In this case, the control unit 340 may provide the analysis result of the commercial district trend in the vicinity of the parking location where the user parked to the terminal device 400 of the user who wants to visit the corresponding parking lot.

In addition, the control unit 340 predicts the parking lot and the expected parking time in which the vehicle is expected to be parked in advance based on the destination of the vehicle, the location of the vehicle, the traffic situation, etc., and considers the situation of the expected parking lot in consideration of the connected and/or alternative parking lot may be guided to the user terminal device 400 . For example, when the expected parking lot of the vehicle is highly congested at the expected parking time or parking is impossible, the controller 340 may guide the user terminal device 400 to another parking lot associated with the expected parking lot. As another example, if there is a history of parking in the vicinity of the same parking lot as the expected parking lot of the vehicle after another vehicle visits it, the controller 340 may guide the nearby parking lot to the user terminal device 400 as an alternative parking lot.

In addition, the control unit 340 determines whether the parking lot is in a dangerous situation (ex. fire occurrence, an accident inside the parking lot, etc.) based on the images captured by the vehicle image photographing device 100, and stores the determination result in the storage unit 330 can also be stored in In this case, the control unit 340 may provide information on whether a dangerous situation exists to the terminal device 400 of the user who wants to visit the corresponding parking lot.

Meanwhile, the controller 340 may mediate data communication between the plurality of vehicle image photographing apparatuses 100 provided in different vehicles so that the plurality of vehicle image photographing apparatuses 100 are communicatively connected to each other. As an example, the server 300 may be implemented as a cloud server.

Specifically, the controller 340 may perform an event monitoring function between users. That is, the vehicle image photographing apparatus 100 may determine whether an event has occurred for another vehicle. For example, the vehicle image photographing apparatus 100 may determine whether a situation requiring notification to another vehicle, such as an impact event or an accident event, occurs in another vehicle through image analysis. If it is determined that the event has occurred, the vehicle image photographing apparatus 100 may upload the event image to the server 300 , and the controller 340 of the server 300 controls the user's terminal device 400 of the event occurrence. may be determined, and the image captured by the vehicle image capturing apparatus 100 of vehicles located in the vicinity of another vehicle may be determined to the user's terminal device 400 , and an intermediary service capable of transacting image data may be provided.

In addition, the control unit 340 may provide an intermediary service in the same way for a human accident or theft accident event in addition to the vehicle.

In addition, the control unit 340 may provide a brokerage service in the same manner as to whether or not a crackdown event for other vehicles occurs.

9 is a block diagram illustrating a user terminal device according to an embodiment of the present invention. Referring to FIG. 9 , the user terminal device 400 may include all or part of a communication unit 410 , a storage unit 420 , an input unit 430 , an output unit 440 , and a control unit 450 .

The communication unit 410 may be provided for the user terminal device 400 to communicate with other devices. Specifically, the user terminal device 400 may transmit/receive data to and from at least one of the vehicle image photographing device 100 , the vehicle communication device 200 , and the parking lot guidance service providing server 300 through the communication unit 410 .

For example, the communication unit 410 connects to the parking lot guide service providing server 300 that stores data generated by the vehicle image photographing device 100 , and various types of parking guide services from the parking lot guide service providing server 300 . data can be received.

Here, the communication unit 410 is a type connected through a wireless or wired method through a local area network (LAN) and an Internet network, a type connected through a USB (Universal Serial Bus) port, a mobile communication network such as 3G, 4G It can be implemented using various communication methods, such as a form connected through a short-range wireless communication method such as near field communication (NFC), radio frequency identification (RFID), and Wi-Fi.

The storage unit 420 functions to store various data and applications required for the operation of the user terminal device 400 . In particular, the storage unit 420 may store a "parking lot guide service providing application" according to an embodiment of the present invention.

Here, the storage unit 420 includes a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a register, a hard disk, a removable disk, It may be implemented as a storage device in a built-in type such as a memory card or a Universal Subscriber Identity Module (USIM), as well as a storage device in a removable type such as a USB memory.

The input unit 430 functions to convert a physical input from the outside of the user terminal device 400 into a specific electrical signal. Here, the input unit 430 may include all or part of the user input unit and the microphone unit.

The user input unit may receive a user input such as a touch, a gesture, and a push operation. Here, the user input unit may be implemented in the form of various buttons, a touch sensor for receiving a touch input, a proximity sensor for receiving an approaching motion, and the like. In addition, the microphone unit may receive a user's voice and a sound generated inside or outside the vehicle.

The output unit 440 is configured to output data of the user terminal device 400 , and the output unit 440 may include a display unit 441 and an audio output unit 443 .

The display unit 441 may output data that can be visually recognized by the user of the user terminal device 400 . In particular, the display unit 441 may display a user interface corresponding to the parking lot guide service according to the execution of the "parking lot guide service providing application" according to an embodiment of the present invention.

Here, the parking lot guide service user interface may include a parking available location guide user interface, a vehicle parking location guide user interface, a parking lot route guide user interface, and a parking impact event guide user interface.

Meanwhile, the audio output unit 443 may output data that the user of the user terminal device 400 can audibly recognize. Here, the audio output unit 443 may be implemented as a speaker expressing sound of data to be notified to the user of the user terminal device 400 .

The controller 450 controls the overall operation of the user terminal device 400 . Specifically, the control unit 450 may control all or part of the communication unit 410 , the storage unit 420 , the input unit 430 , and the output unit 440 . In particular, when various data is received from the vehicle image capturing device 100 , the vehicle communication device 200 , and/or the parking lot guidance service providing server 300 , the control unit 450 processes the received data through the communication unit 410 . Accordingly, the user interface may be generated and the display unit 441 may be controlled to display the generated user interface.

The controller 450 may execute applications that provide advertisements, the Internet, games, videos, and the like. In various embodiments, the controller 450 may include one processor core or a plurality of processor cores. For example, the controller 450 may include, for example, a multi-core such as a dual-core, a quad-core, or a hexa-core. . In some embodiments, the controller 450 may further include a cache memory located inside or outside.

The controller 450 may receive commands from other components of the user terminal device 400 , interpret the received commands, and perform calculations or process data according to the interpreted commands.

The controller 450 may process data or signals generated or generated in an application. For example, the controller 450 may request a command, data, or signal from the storage 420 to execute or control an application. The controller 450 may record (or store) or update a command, data, or signal to the storage 420 in order to execute or control an application.

The control unit 450 may interpret and process a message, data, command, or signal received from the communication unit 410 , the storage unit 420 , the input unit 430 , and the output unit 440 . Also, the controller 450 may generate a new message, data, command, or signal based on the received message, data, command, or signal. The control unit 450 may provide the processed or generated message, data, command, or signal to the communication unit 410 , the storage unit 420 , the input unit 430 , the output unit 440 , and the like.

All or part of the control unit 450 electrically (eg, the communication unit 410, the storage unit 420, the input unit 430, the output unit 440) and other components in the user terminal device 400 ( electrically or operably coupled with or connected to.

According to embodiments, the controller 450 may be configured with one or more processors. For example, the controller 450 may include an application processor (AP) that controls an upper layer program, such as an application program, or a communication processor (CP) that controls communication.

Meanwhile, the aforementioned input unit 430 may receive an instruction, an interaction, or data from a user. The input unit 430 may sense a touch or hovering input of a finger and a pen. The input unit 430 may sense an input caused through a rotatable structure or a physical button. The input unit 430 may include sensors for detecting various types of inputs. The input received by the input unit 430 may have various types. For example, an input received by the input unit 430 may include a touch and release, a drag and drop, a long touch, a force touch, and a physical depression. and the like. The input unit 430 may provide the received input and data related to the received input to the controller 450 . In various embodiments, although not shown in FIG. 4 , the input unit 430 may include a microphone (or transducer) capable of receiving a user's voice command. In various embodiments, although not shown in FIG. 4 , the input unit 430 may include an image sensor or a camera capable of receiving a user's motion.

Meanwhile, the above-described display unit 441 may output content, data, or a signal. In various embodiments, the display 441 may display an image signal processed by the controller 450 . For example, the display unit 441 may display a captured or still image. As another example, the display 441 may display a video or a camera preview image. As another example, the display unit 441 may display a graphical user interface (GUI) so that the user can interact with the terminal device 400 .

The display unit 441 may be configured with a liquid crystal display (LCD) or an organic light emitting diode (OLED).

According to embodiments, the display unit 441 may be configured with an integrated touch screen by being coupled with a sensor capable of receiving a touch input or the like.

In various embodiments, the control unit 450 sets the at least one function to the input unit so that the input unit 430 has at least one function among a plurality of functions that the user terminal device 400 can provide to the user. 430) can be mapped. For example, the at least one function is an application execution function, a parking location guidance function of the vehicle, a live view view function that is a function to view a real-time photographed image of the vehicle image photographing apparatus 100, and a vehicle image photographing apparatus 100 power on / Off control function, vehicle power on/off function, vehicle parking/driving mode guidance function, event occurrence guidance function, current vehicle location inquiry function, vehicle parking location and parking time guidance function, parking history guidance function, driving history guidance function, It may include at least one of an image sharing function, an event history function, a remote playback function, and an image viewing function.

In various embodiments, the input unit 430 may receive the configuration information from the control unit 450 . The input unit 430 may display an indication for indicating the function based on the configuration information.

In various embodiments, the control unit 450 may transmit configuration information to the input unit 430 to indicate what the at least one function is mapped to the input unit 430 . The configuration information may include data for displaying through the display unit 411 an indication for indicating which function among the plurality of functions is provided through the input unit 430 . The configuration information may include data for indicating a function selected by the controller 450 among the plurality of functions.

Also, the controller 450 may generate a user interface based on data received from the parking lot guidance service providing server 300 and control the display 441 to display the generated user interface.

Meanwhile, when parking of the own vehicle is completed, the controller 150 automatically generates parking location information of the own vehicle, generates a user interface based on the automatically generated parking location information, and displays the generated user interface. It is also possible to control the unit 441 . In this case, the controller 450 may generate the parking location information of the own vehicle by using a satellite navigation device such as GPS provided in the user terminal device 400 .

Specifically, the control unit 150 determines whether the user terminal device 400 and the own vehicle are connected to Bluetooth or a vehicle application (eg, Apple ^TM Carplay ^TM , Android ^TM Auto ^TM , navigation, parking guidance service providing application, etc.) It is possible to generate parking location information of the own vehicle based on whether the connection is made.

For example, the controller 150 may generate the location of the user terminal device 400 at the point in time when the Bluetooth connection between the user terminal device 400 and the own vehicle is disconnected as the parking location information of the own vehicle.

Through this, the user terminal device 400 may provide a vehicle parking location guide service to the user even when the parking location information of the own vehicle is not generated or is erroneously generated in the parking lot guide service providing server 300 .

10 is a timing diagram illustrating a method of providing a parking lot guidance service according to an embodiment of the present invention.

Referring to FIG. 10 , each of the plurality of vehicle image photographing apparatuses 100 may acquire a parking lot image by performing photographing ( S1010 ). Here, the parking lot image may include an image captured during the period from the time of entering the parking lot of the vehicle to the time of leaving the parking lot of the vehicle.

In addition, each of the plurality of vehicle image photographing apparatuses 100 may generate at least one of parking lot location information, parking space information, surrounding parking vehicle information, and own vehicle location information ( S1020 ). Here, step S1020 is the above-described parking lot location information generating unit 175-1, parking space information generating unit 175-2, parking vehicle information generating unit 175-3, and own vehicle location information generating unit 175-4. ), the AI (Artificial Intelligence) processing unit 175 - 5 may be performed.

And, each of the plurality of vehicle image photographing apparatus 100 generates parking lot data by combining the time information and the parking lot image with the generated information (S1025), and transmits the generated parking lot data to the parking lot guide service providing server 300 It can be transmitted (S1030).

In this case, the parking lot guidance service providing server 300 may generate a parking lot model representing the real-time situation of the parking lot by using the received parking lot data (S1040). Here, the parking lot model generated by the parking lot model generator 320 may be a 3D model.

In addition, the parking lot guidance service providing server 300 may match and store the generated parking lot model and parking lot data (S1050). Specifically, the storing step (S1050) may be stored by matching the parking lot model with the corresponding parking lot location information, parking space information, surrounding parking vehicle information, own vehicle location information, time information, and parking lot image.

On the other hand, each of the plurality of vehicle image photographing apparatus 100 determines whether the update of the parking lot data is necessary (S1055), and if the update is necessary (S1055:Y), updates the parking lot data (S1060), and the updated parking lot data may be transmitted to the parking lot guidance service providing server 300 (S1065).

Here, the update condition of the parking lot data may include a case in which a change occurs in the parking lot image due to the taking out of a nearby vehicle of the own vehicle of the vehicle, a case in which a preset period arrives, and the like.

Meanwhile, the parking lot guidance service providing server 300 may update the previously generated parking lot model and parking lot data using the received parking lot data (S1070). Specifically, the parking lot guidance service providing server 300 extracts only the difference between the previously generated parking lot model and the subsequently generated parking lot model, and then reflects only the difference portion to update the parking lot model.

Meanwhile, the parking lot guidance service providing server 300 may receive a service provision request from the accessed user terminal device 400 (S1080).

In this case, the parking lot guidance service providing server 300 may provide a parking lot guidance service that meets the user's request based on the parking lot model and parking lot data (S1085).

In this case, the user terminal device 400 may display a parking lot guide service user interface corresponding to the service provided by the parking lot guide service providing server 300 ( S1090 ). Here, the user interface may include a parking available location guide user interface, a vehicle parking location guide user interface, and a parking lot route guide user interface.

Meanwhile, the parking lot guidance service providing server 300 may provide a service-related alarm to the user terminal device 400 according to a specific condition even without a user's service provision request. In this case, the service-related alarm is an alarm related to the parking location guidance service, and may be link data connected to a parking location of the own vehicle or a vehicle parking location guide user interface.

Specifically, in a specific condition, when the user's getting off the vehicle is detected, when the user's action to find a parked vehicle is detected (for example, when moving to a parking lot, when the vehicle is remotely started, when the navigation is executed, etc.) and a predetermined time interval after parking. Here, the specific condition may be received from the user terminal device 400 or the vehicle image capturing apparatus 100 .

11 is a timing diagram illustrating a method of providing a parking lot guidance service according to another embodiment of the present invention.

Referring to FIG. 11 , each of the plurality of vehicle image photographing apparatuses 100 may acquire a parking lot image by photographing ( S1110 ). In addition, each of the plurality of vehicle image photographing apparatuses 100 may generate at least one of parking lot location information, parking space information, surrounding parking vehicle information, and own vehicle location information ( S1120 ).

And, each of the plurality of vehicle image photographing apparatus 100 generates parking lot data by combining the time information and the parking lot image with the generated information (S1125), and transmits the generated parking lot data to the parking lot guide service providing server 300 It can be transmitted (S1130).

In this case, the parking lot guidance service providing server 300 may use the received parking lot data to generate a parking lot model representing the real-time situation of the parking lot (S1140). In addition, the parking lot guidance service providing server 300 may match and store the generated parking lot model and parking lot data (S1150).

Meanwhile, each of the plurality of vehicle image photographing apparatuses 100 may determine whether an event of another vehicle parked in the vicinity of the own vehicle has occurred ( S1155 ).

For example, each of the plurality of vehicle image photographing apparatuses 100 may determine whether the surrounding vehicle event has occurred based on a sound, a motion of a front object, and the like. Alternatively, each of the plurality of vehicle image photographing apparatuses 100 may determine whether a nearby vehicle event has occurred according to a remote request.

If it is determined that a surrounding vehicle impact event has occurred (S1155:Y), each of the plurality of vehicle image photographing apparatuses 100 updates parking lot data (S1160), and transmits the updated parking lot data to the parking lot information service providing server 300 ) can be transmitted (S1165). Here, the updated parking lot data may include data before and after a predetermined time of the occurrence of the surrounding vehicle impact event.

On the other hand, the parking lot guidance service providing server 300 may use the received parking lot data to update the previously created parking lot model and parking lot data (S1170).

Then, the parking lot guidance service providing server 300 may generate information about the vehicle that caused the shock from the updated parking lot data (S1180). Here, the vehicle information that generated the impact may include vehicle number information, location information of a parking lot where the impact occurred, number of floors information, location information of a parking space, and location information of a parking space.

In addition, the parking information service providing server 300 may provide a parking shock event guidance service to the terminal device 400 of the vehicle user who is shocked based on the generated vehicle information (S1185).

In this case, the user terminal device 400 may display a parking shock event guide user interface corresponding to the service provided by the parking lot guide service providing server 300 ( S1190 ). Here, the user interface may display the number of the vehicle generating the impact, the location of the impact, and the like.

14 is a timing diagram illustrating a method of providing a parking lot payment service according to another embodiment of the present invention.

14, the parking service providing server 300 acquires a parking lot image, generates parking lot location information, parking space information, surrounding parking vehicle information and own vehicle location information, and adds time information and parking lot image to the generated information. By combining, it is possible to generate parking lot data (S1210).

Then, the parking lot service providing server 300 transmits the generated parking lot data to the parking lot payment server 500 (S1220), and the parking lot payment server 500 may generate payment information based on the parking lot data (S1230) . Here, the payment information may include a parking fee of the corresponding vehicle, parking time, vehicle type, penalty and incentive information, and the like.

Specifically, the parking lot payment server 500 calculates penalty information or incentive information based on the location information of the own vehicle in the parking lot data, and calculates the parking fee of the corresponding vehicle based on the calculated penalty or incentive information and time information to calculate payment information After creating , you can create payment information. Here, the penalty information or incentive information is information related to the rate of addition or subtraction of the parking fee according to the parking position of the vehicle, and may be determined differently depending on the parking position of the vehicle and the parking time.

For example, when a non-disabled vehicle is parked in a disabled parking area, the parking payment server 500 calculates penalty information for a parking fee reduction rate proportional to the parked time, and penalty information on the parking fee according to the parked time can be applied to calculate the parking fee.

As another example, when parking in an area other than the parking area, when a medium-sized vehicle is parked in a light vehicle area, when parking to obstruct the parking of other vehicles (parking outside the parking area), etc., the parking lot payment server 500 may calculate penalty information and generate payment information.

In addition, the parking lot payment server 500 may generate the payment information by calculating incentive information based on the discount information, and calculating the parking fee of the corresponding vehicle based on the calculated incentive information and time information. Here, the discount information may include various information related to parking rate discounts such as card payment details in the building where the parking lot is located, parking discount coupons, Dadungi discount, electric vehicle discount, discount for the disabled, etc., and such discount information is stored on the parking service providing server ( 300 ) or may be received from the user terminal device 400 .

Then, the parking payment server 500 transmits the generated payment information to the user terminal device 400 (S1240), and the user terminal device 400 may display a parking payment guide user interface based on the payment information ( S1250). Here, the parking payment guide user interface may include parking status (parking time, parking area, etc.), parking fee inquiry, and parking fee payment.

In addition, the user terminal device 400 may receive a payment request from the user based on the parking payment guide user interface (S1260), and transmit the payment request to the parking lot payment server 500 (S1270). In this case, the payment request may include card information for paying the parking fee.

Then, the parking lot payment server 500 may pay the parking fee of the corresponding vehicle based on the payment request, and control the breaker of the corresponding parking lot (S1280).

15 is a block diagram illustrating an autonomous driving system 1500 for a vehicle according to an embodiment of the present invention.

The autonomous driving system 1500 of the vehicle according to FIG. 15 includes sensors 1503 , an image preprocessor 1505 , a deep learning network 1507 , an artificial intelligence (AI) processor 1509 , a vehicle control module 1511 , It is a deep learning network including a network interface 1513 and a communication unit 1515 . In various embodiments, each element may be connected through various interfaces. For example, sensor data sensed and output by the sensors 1503 is fed to the image preprocessor 1505 . The sensor data processed by the image preprocessor 1505 is fed to a deep learning network 1507 that runs on the AI processor 1509 . The output of the deep learning network 1507 running in the AI processor 1509 is fed to the vehicle control module 1511 . Intermediate results of the deep learning network 1507 running on the AI processor 1507 are fed to and loaded from the AI processor 1509 . In various embodiments, the network interface 1513 communicates with remote servers based on the autonomous driving operation of the vehicle, and transmits information transmitted and received through communication with the remote servers to internal block configurations. Also, the network interface 1513 is used to transmit sensor data acquired from the sensor(s) 1503 to a remote server or internal block configurations, and the like. In some embodiments, autonomous driving system 1500 may include additional or fewer components as appropriate. For example, in some embodiments, image preprocessor 1505 is an optional component. According to another example, in some embodiments, a post-processing component (not shown) is used to perform post-processing on the output of the deep learning network 1507 before the output is provided to the vehicle control module 1511 .

In some embodiments, sensors 1503 include one or more sensors. In various embodiments, the sensors 1503 may be attached at different locations on the vehicle and/or facing one or more different directions. For example, the sensors 1503 may be directed to the front, sides, rear of the vehicle in forward-facing, rear-facing, side-facing, etc. directions. , and/or may be attached to a roof. In some embodiments, the sensors 1503 may be image sensors, such as high dynamic range cameras. In some embodiments, sensors 1503 include non-visual sensors. In some embodiments, the sensors 1503 include radar (RADAR), Light Detection And Ranging (LiDAR), and/or ultrasonic sensors in addition to an image sensor. In some embodiments, the sensors 1503 are not mounted on a vehicle having the vehicle control module 1511 . For example, sensors 1503 may be included as part of a deep learning system for capturing sensor data and attached to an environment or roadway and/or mounted to surrounding vehicles.

In some embodiments, an image pre-processor 1505 is used to preprocess the sensor data of the sensors 1503 . For example, image preprocessor 1505 may be used to preprocess sensor data, split sensor data into one or more components, and/or post-process one or more components. In some embodiments, image preprocessor 1505 may be a graphics processing unit (GPU), central processing unit (CPU), image signal processor, or specialized image processor. there is. In various embodiments, the image preprocessor 1505 may be a tone-mapper processor for processing high dynamic range data. In some embodiments, the image preprocessor 1505 may be a component of the AI processor 1509 .

In some embodiments, deep learning network 1507 is a deep learning network for implementing control instructions for controlling an autonomous vehicle. For example, the deep learning network 1507 may be an artificial neural network, such as a convolutional neural network (CNN) trained using sensor data, the output of which is provided to the vehicle control module 1511 .

In some embodiments, artificial intelligence (AI) processor 1509 is a hardware processor for running deep learning network 1507 . In some embodiments, AI processor 1509 is a specialized AI processor for performing inference using convolutional neural networks (CNNs) on sensor data. In some embodiments, AI processor 1509 is optimized for bit depth of sensor data. In some embodiments, AI processor 1509 is optimized for deep learning operations, such as operations of a neural network, including convolution, dot product, vector and/or matrix operations, among others. In some embodiments, the AI processor 1509 may be implemented using a plurality of graphic processing units (GPUs) that can effectively perform parallel processing.

In various embodiments, the AI processor 1509, when executed, is configured to perform deep learning analysis on sensor data received from the sensor(s) 1503 and determine machine learning results used to operate the vehicle at least partially autonomously. coupled via an input/output interface to a memory configured to provide an AI processor having instructions that cause it. In some embodiments, the vehicle control module (Vehicle Control Module) 1511 processes commands for vehicle control output from the artificial intelligence (AI) processor 1509, in order to control various modules of the vehicle, the AI processor ( 1509) is used to translate the output of each vehicle into instructions for controlling the module. In some embodiments, the vehicle control module 1511 is used to control a vehicle for autonomous driving. In some embodiments, the vehicle control module 1511 may adjust the steering and/or speed of the vehicle. For example, the vehicle control module 1511 may be used to control driving of the vehicle, such as deceleration, acceleration, steering, lane change, and lane maintenance. In some embodiments, the vehicle control module 1511 may generate control signals for controlling vehicle lighting, such as brake lights, turns signals, headlights, etc. can In some embodiments, the vehicle control module 1511 is a vehicle's sound system, a vehicle's audio warnings, a vehicle's microphone system, and a vehicle's horn system. system), etc., are used to control vehicle audio related systems.

In some embodiments, vehicle control module 1511 is configured with notification systems, including warning systems, to notify passengers and/or driver of driving events, such as a potential collision or approach of an intended destination. ) is used to control In some embodiments, vehicle control module 1511 is used to calibrate sensors, such as sensors 1503 of a vehicle. For example, the vehicle control module 1511 may modify the orientation of the sensors 1503 , change the output resolution and/or format type of the sensors 1503 , and set the capture rate You can increase or decrease it, adjust the dynamic range, and adjust the focus of the camera. Also, the vehicle control module 1511 may individually or collectively turn on/off the operations of the sensors.

In some embodiments, the vehicle control module 1511 modifies the frequency range of filters, adjusts features and/or edge detection parameters for object detection, or sets bit depth and channels It can be used to change parameters of the image preprocessor 1505 in a manner such as adjusting channels and bit depth. In various embodiments, the vehicle control module 1511 is used to control autonomous driving of a vehicle and/or a driver assistance function of the vehicle.

In some embodiments, the network interface 1513 is responsible for an internal interface between the block components of the autonomous driving system 1500 and the communication unit 1515 . Specifically, the network interface 1513 is a communication interface for receiving and/or sending data including voice data. In various embodiments, the network interface 1513 connects voice calls or receives and/or sends text messages via the communication unit 1515 , transmits sensor data, updates the vehicle's software with an autonomous driving system, or the vehicle It interfaces with external servers to update the software of the autonomous driving system.

In various embodiments, the communication unit 1515 includes various wireless interfaces of a cellular or WiFi method. For example, the network interface 1513 includes the sensors 1503 , the image preprocessor 1505 , the deep learning network 1507 , the AI processor 1509 , and the vehicle control module from the servers connected through the communication unit 1515 . may be used to receive updates to operating parameters and/or instructions for 1511 . For example, the machine learning model of the deep learning network 1507 may be updated using the communication unit 1515 . According to another example, the communication unit 1515 may be used to update operating parameters of the image preprocessor 1505 such as image processing parameters and/or firmware of the sensors 1503 .

In another embodiment, the communication unit 1515 is used to activate communication for emergency services and emergency contact in an accident or near-accident event. For example, in a crash event, the communication unit 1515 may be used to call emergency services for assistance, and may inform emergency services of the vehicle's location and crash details. In various embodiments, the communication unit 1515 may update or obtain the expected arrival time and/or destination location.

16 is a block diagram of an autonomous driving system 1600 according to another embodiment of the present invention.

Referring to FIG. 16 , sensors 1602 include one or more sensors. In various embodiments, the sensors 1602 may be attached at different locations on the vehicle and/or facing one or more different directions. For example, the sensors 1602 may be directed to the front, sides, rear of the vehicle in forward-facing, rear-facing, side-facing, etc. directions. , and/or may be attached to a roof. In some embodiments, sensors 1503 may include image sensors, such as high dynamic range cameras, and/or non-visual sensors. In some embodiments, sensors 1602 may include radar, LiDAR, and/or ultrasonic sensors in addition to an image sensor.

The AI processor 1604 may include a high-performance processor capable of accelerating learning of AI algorithms such as deep learning by efficiently processing a large amount of data required to perform autonomous driving and autonomous parking of a vehicle.

The deep learning network 1606 is a deep learning network for implementing control instructions for controlling autonomous driving and/or autonomous parking of a vehicle. For example, the deep learning network 1606 may be an artificial neural network, such as a convolutional neural network (CNN) trained using sensor data, the output of which is provided to the vehicle control module 1614 .

The processor 1608 controls the overall operation of the autonomous driving system 1600, and according to the output result of the deep learning network 1606, the sensor(s) to acquire sensor information necessary for autonomous driving and/or autonomous parking of the vehicle 1602 can be controlled. In addition, the processor 1608 may generate control information of the vehicle for performing autonomous driving and/or autonomous parking of the vehicle using the acquired sensor information and the deep learning result and output it to the vehicle control module 1614 . .

In addition, when an autonomous parking request is input by the user, the processor 1608 transmits the autonomous parking service request to the service providing server 1800 through the communication unit 1612 (a parking lot empty space request message), and the service providing server 1800 ) may control the vehicle control module 1614 to perform autonomous driving and autonomous parking to a parking space according to the autonomous parking service response (parking empty space response message) received from. In this case, the autonomous parking request by the user may be performed through a user's touch gesture input through a display unit (not shown) or a voice command input through a voice input unit.

In addition, the processor 1608 controls to download the application and/or map data for the serviceable area from the service providing server through the communication unit 1612 when the vehicle enters the parking lot guidance service and/or autonomous parking service available area. can

In addition, the processor 1608 transmits a parking completion message to the service providing server 1800 through the communication unit 1612 when the autonomous parking is completed by the arrival of the vehicle claimable zone, and turns off the ignition of the vehicle, or turn off the power of In this case, the parking completion message may include information about a parking completion time and location of the vehicle, and wake-up time information of the autonomous driving system 1600 .

In addition, when the autonomous vehicle enters the parking space, the processor 1608 generates a control command for performing autonomous parking using various sensor information obtained from the sensors 1602 to the vehicle control module 1614 . print out For example, the processor 1608 may identify the parking lot located in the parking lot from the parking lot image obtained through the image acquisition sensor, and also identify whether the vehicle is parked in the parking lot. For example, when a parking line marked in a parking lot is detected through analysis of an image acquired through an image acquisition sensor, the processor 1608 identifies the detected area as a parking space, and self-loads in the identified parking space. It can be determined whether parking is possible according to the existence of Then, the processor 1608 uses the direction and position of the obstacle obtained from the sensors 1602 (ultrasonic sensor, RADAR, LiDAR, etc.) of the vehicle to autonomously park the vehicle in a parking space that can be parked. A control command for parking the vehicle while preventing the vehicle is output to the vehicle control module 1614 .

In another embodiment, when the autonomous vehicle enters the parking space, the processor 1608 moves the vehicle to the location information of the available parking space received from the service providing server 1800 to park the sensor data of the sensors 1602. use it Specifically, the processor 1608 controls the vehicle by using the sensor data of the sensors 1602 to provide a control command for performing autonomous parking while avoiding collision with other vehicles parked in walls, pillars, and other parking spaces of the parking lot. output to module 1614 .

The storage unit 1610 provides training data for a deep learning network for performing autonomous driving and/or autonomous parking of a vehicle and/or software for performing autonomous driving and/or autonomous parking of a vehicle, route guidance, and autonomous driving. Electronic map data for the purpose may be stored.

The communication unit 1612 transmits and receives data through a wireless communication network between the autonomous driving system 1600 and the user terminal device 1700 and/or the service providing server 1800 .

The vehicle control module 1614 may output control commands for controlling acceleration, deceleration, steering, shifting, etc. of a vehicle for performing autonomous driving of the vehicle and/or autonomous parking of the vehicle to respective components. For example, if acceleration of the vehicle is required, an acceleration command is output to the engine and/or electric motor of the vehicle, and if deceleration of the vehicle is required, a brake command is output to the engine and/or electric motor or braking device of the vehicle ), and if it is necessary to change the vehicle traveling direction, a control command for moving the vehicle in the vehicle traveling direction determined by the vehicle steering wheel or vehicle wheel is generated and output.

17 is a block diagram of a user terminal device 1700 according to another embodiment of the present invention.

The user terminal device 1700 according to another embodiment of the present invention includes a communication unit 1702 , a processor 1704 , a display unit 1706 , and a storage unit 1708 . The communication unit 1702 is connected to the autonomous driving system 1600 and/or the service providing server 1800 through a wireless network to transmit and receive data.

The processor 1704 controls the overall functions of the user terminal device 1700, and according to another embodiment of the present invention, the autonomous driving system ( 1600) is sent. When a push notification message related to autonomous driving and/or autonomous parking is received from the service providing server 1800 , the processor 1704 controls to be displayed to the user through the display unit 1706 . In this case, the push notification message may include autonomous driving information, autonomous parking completion, parking location information, fee information, and the like. In addition, when a parking fee payment request input is received by the user, the processor 1704 runs an application for parking fee payment, checks the user's payment information (credit card information, account number, etc.), and provides a service server ( 1800) may request the user's payment service providing server (not shown) to pay the parking fee charged by the user.

In addition, when a vehicle hailing service provision is requested by the user, the processor 1704 according to another embodiment of the present invention drives the vehicle hailing application and outputs it through the display unit 1706 , and the location of the vehicle hailing from the user After is input, when a vehicle call command is input, a vehicle hailing service request message is transmitted to the service providing server 1800 through the communication unit 1702 . Then, the processor 1704 according to another embodiment of the present invention receives a vehicle hailing request success message from the service providing server 1800 through the communication unit 1702, the vehicle call request is successful A notification indicating that has been made is provided to the user through the vehicle call application.

In addition, the processor 1704 according to another embodiment of the present invention receives various types of information (vehicle departure notification, expected vehicle arrival time and current location, arrival notification information) according to the vehicle call service from the service providing server 1800 . If it is, it is provided to the user through a push notification message or the like.

In addition, when the processor 1704 according to another embodiment of the present invention determines that the vehicle's current location information has departed from the serviceable area, the communication unit 1702 informs the service providing server 1800 that the vehicle has left the serviceable area. It can be controlled to transmit through , and control to delete the vehicle call application and/or autonomous parking application downloaded from the service providing server 1800 and stored in the storage unit 1708 .

In addition, the storage unit 1708 of the user terminal device 1700 may store at least one of an application for an autonomous parking service and/or a vehicle call service, a route guidance application, map data, and user payment information.

When a user gesture for the autonomous parking service application displayed through the display unit 1706 is input, the processor 1704 may perform a corresponding operation. For example, when a selection gesture for selecting an autonomous parking service providing parking lot and a parking slot is input from the user through the UX (User experience) of the display unit 1706, the processor 1704 displays the vehicle ID, parking lot An autonomous parking service request including a parking lot ID (ID) and a parking slot ID may be transmitted to the service providing server 1800 through the communication unit 1702 . In this case, the parking lot ID is information for identifying a parking lot supporting the autonomous parking service, and location information of the corresponding parking lot may also be mapped and stored in the storage unit 1806 .

Through this process, in another embodiment of the present invention, it is also possible for the user to reserve a space for autonomously parking the vehicle in the parking lot through the user terminal device 1700 . And, if parking is impossible for the parking lot ID included in the autonomous parking service request and the parking space ID of the parking lot ID, the service providing server 1800 transmits a parking impossible message to the user terminal device 1700, or The available parking lot ID and/or the available parking space ID may be transmitted to the user terminal device 1700 . The user terminal device 1700 may visually display a parking space corresponding to a parking space ID that can be parked, a parking space corresponding to a parking space ID that cannot be parked, and the like on the autonomous parking service providing application through the display unit 1706 .

In this specification, a parking lot ID (parking lot ID) is information given to identify a parking lot, and may be mapped to location information in which a parking lot is located and set, and a parking slot ID (parking slot ID) is a plurality of parking spaces included in the corresponding parking lot. As information for identifying parking slots, it may be mapped and set to relative location information of each parking slot.

18 is a block diagram of a service providing server 1800 according to another embodiment of the present invention.

The service providing server 1800 according to another embodiment of the present invention includes a communication unit 1802 , a processor 1804 , a display unit and a storage unit 1806 . The communication unit 1802 of the service providing server 1800 according to another embodiment of the present invention is connected to the autonomous driving system 1600 and/or the user terminal device 1700 through a wireless network to transmit and receive data.

When a parking lot empty space request message is received from the autonomous driving system 1600 through the communication unit 1802, the processor 1804 of the service providing server 1800 according to another embodiment of the present invention checks the available parking area, When the available area is confirmed, the location information of the available parking area and the digital map data of the parking lot are transmitted to the autonomous driving system through the communication unit 1802 . At this time, the processor 1804 of the service providing server 1800 shows the real-time situation of the parking lot image and the parking lot obtained from the CCTV (Closed Circuit Television) located in the parking lot, the video recording devices of the vehicle parked in the parking lot. The parking lot model created to bet and the available parking area are checked through sensor information obtained from sensors located in the parking slot. Specifically, it is possible to distinguish an empty parking space in the parking lot from a parking space in which the vehicle is parked through a parking lot image analysis obtained from a CCTV located in the parking lot and an image capturing device of a parked vehicle. In addition, sensors installed in each parking space within the parking lot sense whether a vehicle is parked in the corresponding parking space, and the processor 1804 of the service providing server 1800 uses the sensed information to determine whether the vehicle is parked in an empty parking space in the parking lot. The parked parking space can be identified. Then, the processor 1804 of the service providing server 1800 according to another embodiment of the present invention includes the identified parking available area in the parking lot empty space response message through the communication unit 1802 to the autonomous driving system 1600 or may be transmitted to the user terminal device 1700 . Then, when the parking completion message is received from the autonomous driving system 1600 , the processor 1802 transmits the parking completion message to the user terminal device 1700 through the communication unit 1802 .

When a vehicle call service request is received from the user terminal device 1700 through the communication unit 1802, the processor 1804 of the service providing server 1800 according to another embodiment of the present invention receives a vehicle identification (VID) in a parking lot. ), transmits a vehicle call service request to the autonomous driving system 1600 of a vehicle parked at the searched parking location, and information received in response to the vehicle call service from the autonomous driving system 1600 are transmitted to the user terminal device 1700 .

The processor 1804 of the service providing server 1800 according to another embodiment of the present invention includes a location for a parking lot that provides an autonomous parking service to the storage unit 1806, map data, a parking lot model indicating a real-time situation for the parking lot, It stores parking lot data, parking lot images, and parking space information. In addition, the processor 1804 of the service providing server 1800 according to another embodiment of the present invention is a vehicle stored in the storage unit 1806 when the vehicle of the user who has requested the vehicle call service and the autonomous parking service leaves the serviceable area. Delete ID, user ID and related information.

When an autonomous parking service request is received from a plurality of user terminal devices 1700 through the communication unit 1802, the processor 1804 of the service providing server 1800 schedules the order in which the autonomous parking service of each vehicle is performed, The autonomous parking service response may be delivered for each vehicle according to the scheduled order.

And, when the autonomous parking service request message is received from the user terminal device 1700, the processor 1804 of the service providing server 1800 according to another embodiment receives the parking lot ID and parking space ID information included in the autonomous parking service request message. is retrieved from the map data stored in the storage unit 1806, and the searched parking lot ID location information and parking space ID location information are transmitted to the autonomous driving system 1600, so that the autonomous driving system 1600 autonomously reaches the corresponding parking lot location. Driving and/or autonomous parking may be performed.

The processor 1804 of the service providing server 1800 according to another embodiment of the present invention may store parking lot-related information in the storage unit 1806 in the form shown in the table below in order to provide an autonomous parking service.

Field Parking lot ID Floor Parking Slot ID parking slot state Parking Time Parking Date Fare User ID Vehicle ID One 69 B3 74 Full 1 hour 09-30-2020 \4,000 tkhan 03H3394 ... ... ... ... ... ... ... ... ... ... 80 1st ^Floor 84 Empty - - - - - 42 B2 34 Reserved - - - kimkim 09A2413

The processor 1804 of the service providing server 1800 according to another embodiment of the present invention stores a database of the form shown in <Table 1> in the storage unit 1806 to provide an autonomous parking service, and the corresponding parking space Whenever the parking state of the vehicle for , it is possible to update the data in the database.

For example, when the autonomous parking service request message is received from the user terminal device 1700 , the processor 1804 of the service providing server 1800 searches the database for information on available parking lots and parking spaces, and then the user terminal device The searched parking lot ID, parking space ID, and corresponding location information are transmitted to the autonomous driving system 1600 of the vehicle connected to the 1700 . Then, the processor 1804 of the service providing server 1800 changes the parking slot state information to Full when the vehicle enters the parking lot for autonomous parking and confirms that parking in the parking space is completed, Update parking time, parking date, fee information, user ID, and vehicle ID information.

On the other hand, when a vehicle call request from the user terminal device 1800 for the autonomously parked vehicle is received, the processor 1804 of the service providing server 1800 updates the data field of <Table 1> stored in the database. For example, if the vehicle leaves the parking space after being changed to the autonomous driving state, the parking space state is changed to Empty, and when the user pays the parking fee, the parking time, parking date, parking fee information, etc. for the corresponding parking space are displayed. initialize again.

On the other hand, when a parking space of the autonomous vehicle is selected from the user terminal device 1700, the processor 1804 of the service providing server 1800 changes the selected parking space ID field in the database to a reserved state, and By updating the ID and vehicle ID fields, it is possible to prevent the provision of autonomous parking service for duplicate parking lot IDs to other users.

19 is a view for explaining an operation flow of an autonomous parking system according to another embodiment of the present invention.

First, the autonomous driving system 1600 transmits a parking lot empty space request message to the service providing server 1800 when the vehicle enters the serviceable area (step S1900) and an autonomous parking command is input from the user (step S1902). (Step S1904). In this case, the parking lot empty space request message may include a user ID and a vehicle ID requesting an autonomous parking service. In this case, the user ID may include information that can identify the user, such as an ID or social security number subscribed to the autonomous parking service, and the vehicle ID can identify the vehicle, such as a vehicle license plate or VIN (Vehicle Identification Number). information may be included.

Then, the service providing server 1800 that has received the parking lot empty space request message checks the available parking area in the parking lot where the vehicle is to be parked (S1906), and transmits the parking lot empty space response message to the autonomous driving system 1600 (S1908). In this case, the parking lot empty space response message may include location information of a parking available area and an electronic map of the parking lot. In the step S1906, the step of the service providing server 1800 confirming the available parking area includes an image obtained from a CCTV installed in the parking lot, an image obtained from an image capturing device installed in a vehicle parked in each parking space of the parking lot, and each parking lot. Through the sensing data sensed by the sensor installed in the car, it is possible to identify the parking available state for each parking space.

After receiving the parking lot empty space response message in step S1908, the autonomous driving system 1600 calculates a route from the current location of the vehicle to the confirmed parking area location information, and then performs autonomous driving to the parking available area. (S1910).

And, when the vehicle arrives in the parking available area (S1912), the autonomous driving system 1600 performs autonomous parking (S1914), and if the parking is completed (Yes in S1916), a parking completion message is sent to the service providing server ( 1800) (S1918), and turns off or turns off the power of the vehicle (S1922). In this case, the parking completion message may include information about a location where the vehicle is parked and information about a parking time.

The service providing server 1800 that has received the parking completion message in step S1918 transmits the parking completion message to the user terminal device 1700 (S1920).

20 is a diagram for explaining an autonomous parking operation flow of the user terminal device 1700 according to another embodiment of the present invention.

First, when the vehicle enters the serviceable area (step S2000), the user terminal device 1700 downloads an application for providing an autonomous parking service from the server (step S2002). At this time, when downloading the application, map data for the parking lot can also be downloaded. And, when receiving an autonomous parking command from the user (step S2004), the user terminal device 1700 obtains parking space location information (step S2006), and from the current location of the vehicle on the map data, the obtained parking space A route to the location is calculated (step S2008), and autonomous driving is performed to a parkingable location according to the calculated route (step S2010). If the autonomous vehicle arrives at a parking available location (Yes in S2012), the user terminal device 1700 performs autonomous parking (S2014), and if the parking is complete (Yes in S2016), a parking completion message is transmitted to the server (S2018). In this case, the parking completion message may include parking location information and parking completion time information. In this case, the parking location information may also include a parking lot ID and a parking lot location and a parked parking space (parking space) ID and location information of the parking space.

21 is a diagram for explaining an autonomous parking operation flow of the service providing server 1800 according to another embodiment of the present invention.

First, when the service providing server 1800 receives a parking lot empty space request message (S2100), it searches for an available parking space (S2102). If a parking space exists as a result of the search (“Yes” in S2104), the service providing server 1800 obtains parking space location information (S2108), and if there is no parking space (“No” in S2104), an alternative service is provided provided (S2106). In this case, the alternative service includes a function to search for and guide a nearby parking lot location and available parking space, or notify the user that there is no available parking space.

Then, the service providing server 1800 transmits the parking space location information to the autonomous driving system 1600 (step S2110), and when a parking completion message is received from the autonomous driving system 1600 (step S2112), the parking completion message is transmitted to the user terminal device 1700 (step S2114).

22 is a diagram for explaining an operation flow for providing a vehicle hailing service or a passenger pick up service of an autonomous driving system according to another embodiment of the present invention.

First, the service providing server 1800 stores location information for each vehicle ID parked in the serviceable area (S2200). Then, the user terminal device 1700 transmits a vehicle call service request message to the service providing server 1800 when a vehicle call location is input from the user (step S2202) and a vehicle call command is input (step S2204) (S2206). step). The service providing server 1800 that has received the vehicle call request message in step S2206 checks the vehicle ID (ID of the vehicle to be called) included in the vehicle call request message (step S2208), and the vehicle ID is the vehicle call service When it is identified that the vehicle is the user's vehicle to be provided, a parking location corresponding to the identified vehicle ID is searched for (step S2210), and a call request is transmitted to the autonomous driving system 1600 of the called vehicle (step S2218).

Then, the autonomous driving system 1600 transitions from an idle state (step S2212) to a wake-up state (step S2214). In this case, the transition from the idle state to the wake-up state may occur at a predetermined period or at a predetermined time. The reason why the autonomous driving system 1600 transitions from the idle state to the wake-up state whenever necessary is to save power of the vehicle battery. The processor 1608 of the autonomous driving system 1600 transitioned to the wake state in step S2214 may supply power to the communication unit 1612 to demodulate/decode the signals transmitted to the autonomous driving system 1600. . The idle state of the autonomous driving system 1600 is a state in which the processor 1608 stops supplying power to each block configuration and a process for the operation of the autonomous driving system 1600 is not being executed. It means that it is waiting for an operation command from

Alternatively, the processor 1608 of the autonomous driving system 1600 supplies operating power to the communication unit 1612 in the wake-up state to connect to the service providing server 1800 through the communication unit 1612 and call the autonomous parking service and vehicle. The existence of downlink traffic to be transmitted to the autonomous driving system 1600 for service provision may be checked. In addition, the autonomous driving system 1600 checks whether a vehicle call service/passenger pickup service request is received (step S2216), and when a call request message is received in step S2218, after turning on the system power of the vehicle (Step S2220), transition to an active state (Step S2222). When the autonomous driving system 1600 transitions to the active state in step S2222, the autonomous driving system 1600 supplies operating power for driving each part of the vehicle for autonomous driving of the vehicle, and a control command for vehicle control occurs

The autonomous driving system 1600 transitioned to the active state in step S2222 transmits a vehicle call response message to the service providing server 1800 (step S2224), and the service providing server 1800 requests the vehicle call service in step S2206. In response to the message, a vehicle call request success message is transmitted to the user terminal device 1700 (step S2226). Upon receiving the vehicle call request success message in step S2226, the user terminal device 1700 displays a push notification message informing the user that the vehicle call has been successful (step S2228).

In addition, the service providing server 1800 that has transmitted the vehicle call request success message to the user terminal device 1700 in step S2226 transmits a message including call place information to the autonomous driving system 1600 (step S2230) . The autonomous driving system 1600 calculates a route for autonomous driving to the calling place (step S2232), and when driving starts (step S2234), transmits a departure notification message to the service providing server 1800 (step S2236) .

The service providing server 1800 transmits a vehicle departure notification message to the user terminal device 1700 (step S2238), and the service providing server 1800 transmits the autonomous driving system 1600 while autonomously driving (S2240). When the estimated time of arrival (ETA) and the current location information are transmitted (step S2242), the ETA information and the current location information of the vehicle are transmitted to the user terminal device 1700 (step S2244).

Then, when the vehicle driving system 1600 arrives at the calling location (step S2246), it delivers an arrival notification to the service providing server 1800 (step S2248), and the service providing server 1800 to the user terminal device 1700 An arrival notification is delivered (step S2250).

In addition, when the vehicle leaves the serviceable area (step S2252), the user terminal device 1700 transmits a serviceable area departure message to the service providing server 1800 (step S2254), and the service providing server 1800 is The vehicle ID and related information included in the serviceable area departure message may be deleted (S2256), and the user terminal device 1700 may automatically delete the vehicle call service application (S2258).

On the other hand, although it has been described that the user terminal device 1700 performs the steps S2252, S2254, and S2258 in FIG. 22, the autonomous driving system 1600 of the vehicle may also perform the steps.

23 is a diagram for describing a process in which an autonomous driving system of a vehicle performs autonomous parking according to another embodiment of the present invention.

The autonomous driving system of the vehicle 2302 is based on the data 2304 sensed by the sensors 1602 attached to the vehicle 2302 and the result of deep learning by learning the deep learning network 1606 in the available parking space ( After recognizing the presence of 2306 and the parked vehicle 2308, autonomous parking is performed into the available parking space.

24 is a diagram illustrating a UX screen 2400 displayed on the user terminal device 1700 when the autonomous parking system of a vehicle performs autonomous parking according to another embodiment of the present invention.

In FIG. 24 , reference number 2402 indicates an area in which parking is impossible due to parking of other vehicles in a parking lot where a vehicle is to perform autonomous parking, reference number 2404 indicates a user-selectable autonomous parking parking area, and reference number 2406 indicates that parking is not possible. A possible empty space area is indicated, and reference numeral 2408 indicates an area in which the vehicle is autonomously parked.

Reference number 2450 is a screen visually showing a parking space in which the vehicle has completed autonomous parking in the parking lot. The reference number 2450 may display a parking space of a parking lot by moving it on the display unit 1706 of the user terminal device 1700 according to a user's touch gesture (drag, pinch-to-zoom, etc.).

In addition, the processor 1704 of the user terminal device 1700 according to another embodiment of the present invention provides a parking lot ID and a parking space ID corresponding to the parking area selected by the user through the display unit 1706 through the communication unit 1702. It may be transmitted to the providing server 1800 .

25 shows an example of a push notification or a push message displayed on the user terminal device 1700 of a user using the autonomous parking service/vehicle call service of the vehicle according to another embodiment of the present invention. it is one drawing

Reference numeral 2502 is a diagram showing a message for receiving an autonomous parking service request function. When a user selects a corresponding message through a touch gesture, the user terminal device transmits an autonomous parking service request message to the service providing server.

Reference numeral 2504 is a diagram for displaying a message informing that the autonomous parking of the vehicle has been completed in the parking lot and the location of the parked space in text form at the user's request. In addition, the push notification displayed in the text form of reference number 2504 may be connected to a hyperlink capable of displaying a location where the vehicle is parked on a map. That is, when the user selects a push notification message indicating the location where the vehicle of reference number 2504 is parked, the processor 1704 of the user terminal device 1700 runs the map data application and identifies the parked location on the map data. It can be displayed in the form of a symbol.

Reference numeral 2506 is a diagram indicating that selection of the call location is completed while the call location is displayed on the map when a vehicle call location is selected on the map by a user's request. The vehicle call location of reference number 2506 may be moved on the map by the user's touch gesture.

26 is a diagram illustrating an example of a push notification or a push message displayed on a user terminal device 1700 of a user using an autonomous parking service of a vehicle according to another embodiment of the present invention.

Reference number 2602 is a diagram for explaining a message displayed on the user terminal device 1700 when a user calls a parking vehicle. When the user selects a parking vehicle call message 2602a, the user terminal device 1700 sends a vehicle call request message to the service providing server. In addition, the user terminal device 1700 is a diagram for displaying a parking vehicle call completion message 2602b, a vehicle departure notification message 2602c, an ETA and movement information display message 2602d, and an arrival notification message 2602e.

Reference numeral 2604 is a diagram for displaying a message 2604a notifying that the user terminal device 1700 has left the serviceable area of the vehicle and a message 2604b for deleting a vehicle call application that can be used only in the designated serviceable area. When the application deletion message 2604a is selected by the user's request, the corresponding application is deleted.

Reference numeral 2606 is a diagram in which the user terminal device 1700 displays a message 2606a for displaying a parking time and a parking fee, a payment progress message 2606b, and a discount rate application notification message 2606c. In order for the user to receive a discount rate for the parking fee, a QR code may be input through the camera of the user terminal device 1700 or a discount code may be input through an input unit of the user terminal device 1700 .

27 is a diagram for explaining an example in which the service providing server 1800 identifies a parking space through deep learning analysis when an autonomous parking service of a vehicle is requested according to another embodiment of the present invention.

When a parking lot image is input through an image acquisition device such as a CCTV located in a parking lot, the service providing server 1800 performs a parking lot image analysis through deep learning according to another embodiment of the present invention. and determine whether the vehicle is parked for each identified parking space. In FIG. 27, numbers 1 to 43 are parking slot IDs assigned to each parking lot, and the service providing server 1800 recognizes parking lines marked on the road from the obtained parking lot image, and through the recognized results The parking lot can be identified. In addition, the service providing server 1800 may identify a parking space in which a vehicle that has requested autonomous parking can park by additionally analyzing whether a vehicle learned through deep learning is located in each parking lot. In the drawing, it can be determined that parking spaces 26, 31, and 36 are parking spaces.

In addition, the service providing server 1800 may transmit the parking lot ID determined to be able to park, its location information, and the parking space ID in the corresponding parking lot ID to the autonomous driving system 1600 . A parking lot ID determined to be available for parking and location information thereof and a parking space ID in the corresponding parking lot ID may be included in the parking available information. Then, after the service providing server 1800 transmits the parking available information to the autonomous driving system 1600, the parking space ID of the parking lot ID included in the transmitted parking available information is set to complete parking reservation, and another vehicle Even if a parking service provision request is received from the corresponding parking space ID, the service is not provided, thereby preventing overlapping services.

And, when the vehicle leaves the parking lot, the service providing server 1800 updates the vehicle parking state information of the parking space by resetting the ID of the parking space in which the vehicle is parked to an empty space.

On the other hand, terms such as "left", "right", "front", "back", "top", "bottom", "above", "below" in the specification and claims are used for description , is not necessarily intended to describe the relative position of invariants. It will be understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate otherwise than, for example, as shown or described herein. As used herein, the term “connected” is defined as being directly or indirectly connected in an electrical or non-electrical manner. Objects described herein as being "adjacent" to one another may be in physical contact with one another, in proximity to one another, or in the same general scope or area as appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein refers to the same embodiment, but not necessarily.

In addition, in the specification and claims, references to 'connected', 'connecting', 'fastened', 'fastening', 'coupled', 'coupled', etc., and various variations of these expressions, refer to other elements directly It is used in the sense of being connected or indirectly connected through other components.

On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

In addition, the suffixes "module" and "part" for components used in this specification are given or used in consideration of ease of writing the specification, and do not have distinct meanings or roles by themselves.

In addition, the terms used herein are for the purpose of describing the embodiments and are not intended to limit the present invention. As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps.

So far, the present invention has been looked at with respect to preferred embodiments thereof. All embodiments and conditional examples disclosed through this specification have been described with the intention of helping those of ordinary skill in the art to understand the principles and concepts of the present invention, and those skilled in the art It will be understood that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention.

Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Meanwhile, the method for providing a parking lot guidance service according to various embodiments of the present invention described above may be implemented as a program and provided to a server or devices. Accordingly, each device can download the program by accessing the server or device in which the program is stored.

In addition, the above-described method for providing a parking lot guidance service according to various embodiments of the present invention may be implemented as a program and stored in various non-transitory computer readable media to be provided. The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: vehicle image photographing device 200: vehicle communication device
300: parking information service providing server 400: user terminal device

Claims (36)

In the method of providing a parking guide service of a vehicle image photographing device,
acquiring a parking lot image according to the shooting;
generating parking lot data including information of a parking space in a parking lot by using the parking lot image; and
Including; transmitting the generated parking lot data to a parking lot information service providing server;
The parking lot data is a parking guide service providing method, characterized in that used to generate a parking lot model for providing the service in the parking lot guide service providing server. According to claim 1,
The step of generating the parking lot data includes:
Recognizing a location identifier that enables identification of a location of a parking space in the parking lot from the parking lot image and a parking vehicle parked in the parking space;
generating location information of the parking space based on the recognized location identifier; and
and generating information on whether or not a parking space included in the parking space is parked according to the recognized location of the parking vehicle. 3. The method of claim 2,
The step of generating the parking lot data includes:
Generating parking vehicle information including at least one of vehicle type information and vehicle number information for the recognized parking vehicle; further comprising,
The parking vehicle information is a parking guide service providing method, characterized in that generated by classifying for each of a plurality of parking spaces constituting the parking space. According to claim 1,
Detecting a change in the parking lot image as a parked vehicle exits from a parking space included in the parking space or another vehicle enters a parking space included in the parking space to determine whether it is necessary to update parking lot data; A method of providing a parking guide service, characterized in that it further comprises. According to claim 1,
determining whether an impact event of a parked vehicle parked in the vicinity of the own vehicle has occurred;
updating the parking lot data when it is determined that an impact event has occurred in the parked vehicle; and
Transmitting the updated parking lot data to the parking information service providing server; further comprising,
The updated parking lot data is a parking guide service providing method, characterized in that it includes data before and after a predetermined time of the point of occurrence of the impact event. In the vehicle image photographing apparatus,
communication department;
a photographing unit for obtaining an image of a parking lot according to the photographing;
a parking lot data generation unit for generating parking lot data including information on a parking space in a parking lot by using the parking lot image;
A control unit for controlling the communication unit to transmit the generated parking lot data to a parking lot information service providing server;
The parking lot data is a vehicle image photographing device, characterized in that used to generate a parking lot model for providing the service in the parking guide service providing server. 7. The method of claim 6,
The parking lot data generation unit,
an image processing unit for recognizing a parking vehicle parked in the parking space and a location identifier that enables identification of the location of the parking space in the parking lot from the parking lot image; and
Includes; a parking space information generator that generates the location information of the parking space based on the recognized location identifier, and generates information on whether or not a parking space included in the parking space is parked according to the recognized location of the parking vehicle. An image photographing device for a vehicle, characterized in that. 8. The method of claim 7,
The parking lot data generation unit,
Further comprising; a parking vehicle information generator for generating parking vehicle information including at least one of vehicle type information and vehicle number information for the recognized parking vehicle;
The parking vehicle information is an image capturing apparatus for a vehicle, characterized in that it is generated by being classified for each of a plurality of parking spaces constituting the parking space. 7. The method of claim 6,
The control unit is
It is characterized in that it is determined whether it is necessary to update the parking lot data by detecting a change in the parking lot image as a parked vehicle leaves the parking lot included in the parking space or another vehicle enters the parking space included in the parking space. A vehicle video recording device. 7. The method of claim 6,
The control unit is
It is determined whether an impact event has occurred in a parked vehicle parked in the vicinity of the own vehicle, and when it is determined that an impact event has occurred in the parked vehicle, the parking lot data generator is controlled to update the parking lot data, and the updated parking data is transmitted to the parking lot. Controls the communication unit to transmit to a guide service providing server,
The updated parking lot data is an image photographing apparatus for a vehicle, characterized in that it includes data before and after a predetermined time at the time of occurrence of the impact event. In the method of providing parking information service of the server,
Receiving the parking lot data including the information of the parking space in the parking lot from the vehicle image photographing device provided in the vehicle;
generating a parking lot model representing the real-time parking situation of the parking lot as an image based on the received parking lot data; and
Providing a parking guide service to a user terminal device by using the generated parking lot model; parking guide service providing method comprising a. 12. The method of claim 11,
The information of the parking space includes location information of the parking space and information on whether or not parking spaces constituting the parking space are parked,
The step of creating the parking lot model includes:
determining the location of the parking space in the parking lot based on the location information of the parking space;
determining whether to place a vehicle model in the parking lot based on the parking information of the parking lot; and
generating a parking lot model in which the vehicle model is disposed in the parking space according to the determination result; 13. The method of claim 12,
The parking lot data includes parking vehicle information including at least one of type information of a parking vehicle and number information of a parking vehicle,
The step of creating the parking lot model includes:
Based on the parking vehicle information, generating a vehicle model reflecting at least one of a license plate and a vehicle type; Parking guide service providing method further comprising a. 12. The method of claim 11,
The generated parking lot model is a parking lot guidance service providing method, characterized in that the model of the 3D shape. 12. The method of claim 11,
Updating the generated parking lot model; further comprising,
The updating step extracts only the difference part between the previously generated parking lot model and the subsequently generated parking lot model, and reflects only the extracted difference part to update the parking lot model. 12. The method of claim 11,
The step of providing the parking guide service is,
detecting the parking lot model and the parking lot data corresponding to the parking lot in which the vehicle of the user terminal device connected to the server is parked; and
Using the detected parking lot model and the parking lot data to provide at least one of a parking available location guidance service, a vehicle parking location guidance service, and a parking lot route guidance service to the user terminal device; HOW TO PROVIDE SERVICES. 12. The method of claim 11,
The step of providing the parking guide service is,
When an impact event occurs in the first vehicle parked in the parking lot, transmitting a notification of the occurrence of the first vehicle impact event to a vehicle image photographing apparatus of a second vehicle located around the first vehicle;
receiving parking data from a vehicle image capturing apparatus of a second vehicle according to the notification;
generating impact information on the impact situation of the first vehicle based on parking data from the vehicle image capturing apparatus of the second vehicle; and
Providing a parking impact event guide service based on the generated impact information; Parking guide service providing method comprising the. In the parking lot information service providing server,
a communication unit for receiving parking lot data including information on a parking space in a parking lot from an image photographing device for a vehicle provided in the vehicle;
a parking lot model generation unit for generating a parking lot model representing the real-time parking situation of the parking lot as an image based on the received parking lot data; and
A parking information service providing server comprising a; a control unit for providing a parking guide service to the user terminal device using the generated parking lot model. 19. The method of claim 18,
The information of the parking space includes location information of the parking space and information on whether or not parking spaces constituting the parking space are parked,
The parking lot model generation unit,
Determine the location of the parking space in the parking lot based on the location information of the parking space,
It is determined whether or not to place a vehicle model in the parking lot based on the parking information of the parking lot,
Parking guide service providing server, characterized in that for generating a parking lot model in which the vehicle model is arranged in the parking space according to the determination result. 20. The method of claim 19,
The parking lot data includes parking vehicle information including at least one of type information of a parking vehicle and number information of a parking vehicle,
The parking lot model generation unit,
Parking information service providing server, characterized in that generating a vehicle model reflecting at least one of a license plate and a vehicle type based on the parking vehicle information. 19. The method of claim 18,
The generated parking lot model is a parking information service providing server, characterized in that the model of the 3D shape. 19. The method of claim 18,
The parking lot model generation unit,
A parking guide service providing server, characterized in that extracting only the difference part between the previously generated parking lot model and the subsequently generated parking lot model, and performing the update of the parking lot model by reflecting only the extracted difference part. 19. The method of claim 18,
The control unit is
Detecting the parking lot model and the parking lot data corresponding to the parking lot in which the vehicle of the user terminal device connected to the server is parked,
Parking information service providing server, characterized in that by using the detected parking lot model and the parking lot data to provide at least one of a parking available location guidance service, a vehicle parking location guidance service, and a parking lot route guidance service to the user terminal device. 19. The method of claim 18,
The communication unit,
When an impact event occurs in the first vehicle parked in the parking lot, a notification of the occurrence of the first vehicle impact event is transmitted to a vehicle image photographing apparatus of a second vehicle located around the first vehicle, and according to the notification, the vehicle of the second vehicle Receive parking data from the video recording device,
The control unit is
Generating impact information on the impact situation of the first vehicle based on parking lot data from the vehicle image capturing device of the second vehicle, and providing a parking impact event guidance service based on the generated impact information A server that provides parking information service. In the method of providing a parking guide service of a user terminal device,
accessing a parking lot guide service providing server that provides a parking lot guide service based on a vehicle image capturing device;
Receiving a parking lot model and parking lot data representing the real-time parking situation of the parking lot as an image from the parking information service providing server; and
Including; generating a user interface based on the received parking lot model and parking lot data, and displaying the generated user interface;
The user interface includes at least one of a parking available location guide user interface, a vehicle parking location guide user interface, a parking lot route guide user interface, and a parking shock event guide user interface. 26. The method of claim 25,
The parking available location guidance user interface,
Parking guide service providing method, characterized in that the interface for displaying information on the available parking location of the parking lot where the user terminal device is located on the parking lot model based on the parking lot data. 26. The method of claim 25,
The parking lot route guide user interface,
Based on the user's parking location information and the location information in the parking lot of the user terminal device, the parking guide service providing method, characterized in that the interface for displaying the path from the user's current location to the parking location in the parking lot model. 26. The method of claim 25,
The vehicle parking location guide user interface,
Parking guide service providing method, characterized in that the interface for displaying the user's parking location information on the parking lot model based on the parking lot data. 26. The method of claim 25,
The parking impact event guide user interface,
A method of providing a parking guide service, characterized in that it is an interface for displaying on the parking lot model shock information about an impact situation generated based on parking lot data of an image capturing device for a vehicle provided in another vehicle. In the user terminal device,
display unit;
a communication unit for accessing a parking lot guide service providing server that provides a parking lot guide service based on a vehicle image capturing device, and receiving a parking lot model and parking lot data representing a real-time parking situation of a parking lot as an image from the parking lot guide service providing server;
A control unit for generating a user interface based on the received parking lot model and parking lot data, and controlling the display unit to display the generated user interface; including,
The user interface comprises at least one of a parking available location guide user interface, a vehicle parking location guide user interface, a parking lot route guide user interface, and a parking impact event guide user interface. 31. The method of claim 30,
The parking available location guidance user interface,
Based on the parking lot data, the user terminal device, characterized in that the user terminal device is an interface for displaying the parking available location information of the parking lot in which the user terminal device is located in the parking lot model. 31. The method of claim 30,
The parking lot route guide user interface,
Based on the user's parking location information and the location information in the parking lot of the user terminal device, the user terminal device, characterized in that the interface for displaying the path from the user's current location to the parking location in the parking lot model. 31. The method of claim 30,
The vehicle parking location guide user interface,
User terminal device, characterized in that the interface for displaying the user's parking location information on the parking lot model based on the parking lot data. 31. The method of claim 30,
The parking impact event guide user interface,
The user terminal device, characterized in that it is an interface for displaying, on the parking lot model, impact information on an impact situation generated based on parking lot data of an image photographing device for a vehicle provided in another vehicle. A computer-readable recording medium in which a program for executing the method of providing a parking lot guidance service according to any one of claims 1 to 5, 11 to 17 and 25 to 29 is recorded. A program stored in a recording medium comprising a program code for executing the method of providing a parking lot guidance service according to any one of claims 1 to 5, 11 to 17, and 25 to 29.

Images