WO2023283612A1

WO2023283612A1 - System and method for image-based parking determination using machine learning

Info

Publication number: WO2023283612A1
Application number: PCT/US2022/073524
Authority: WO
Inventors: Drake BURCIAGA
Original assignee: Drake Alexander Technologies, Inc.
Priority date: 2021-07-08
Filing date: 2022-07-07
Publication date: 2023-01-12

Abstract

An electronic device comprises at least one transceiver, at least one memory, and at least one processor coupled to the at least one memory and the at least one transceiver. The at least one processor is configured to obtain, using the at least one transceiver, from one or more remote cameras, at least one image of a location including one or more parking spots, detect the one or more parking spots in the at least one image using a machine learning model, assign a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier, and store, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

Description

SYSTEM AND METHOD FOR IMAGE-BASED PARKING DETERMINATION USING

MACHINE LEARNING

TECHNICAL FIELD

[0001] This disclosure relates generally to machine learning systems. More specifically, this disclosure relates to a system and method for image-based parking determination using machine learning.

BACKGROUND

[0002] Drivers attempting to park, especially in busy urban areas, often have difficulty finding available parking spaces, and resort to driving around the same area until the driver finds an open spot, which may require driving until an occupied spot is finally vacated. This leads to heavier traffic in such areas, as a large portion of the traffic can simply be the drivers looking for parking. Some previous technical solutions have been proposed that allow drivers to reserve select parking spots in parking garages or other establishments, but such solutions do not assist drivers with finding vacant spots in general parking areas such as on the street or at public parking meters.

SUMMARY

[0003] This disclosure provides a system and method for image-based parking determination using machine learning.

[0004] In at least one embodiment, an electronic device comprises at least one transceiver, at least one memory, and at least one processor coupled to the memory and the transceiver. The at least one memory includes instructions that, when executed by the at least one processor, cause the at least one processor to obtain, from one or more remote cameras using the transceiver, at least one image of a location including one or more parking spots, create bounding boxes around the one or more parking spots in the at least one image, assign a unique identifier to each one of the bounding boxes and associate global positioning system (GPS) coordinates with each unique identifier, receive, using the transceiver, a request for a parking spot from a remote electronic device, obtain and load, in response to the request, a current image of the location, resize each portion of the current image associated with one of the bounding boxes, grayscale each resized portion of the current image, convert each grayscaled and resized portion of the current image into an array of pixel values, input each array of pixel values into a logistic regression model, output, using the logistic regression model, a binary classifier for each array of pixels values, determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant, select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot, transmit, to the remote electronic device using the transceiver, a message including the GPS coordinates associated with the selected vacant parking spot.

[0005] In at least one embodiment, an electronic device includes at least one transceiver, at least one memory, and at least one processor coupled to the at least one memory and the at least one transceiver. The at least one processor is configured to obtain, using the at least one transceiver, from one or more remote cameras, at least one image of a location including one or more parking spots. The at least one processor is also configured to detect the one or more parking spots in the at least one image using a machine learning model. The at least one processor is also configured to assign a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier. The at least one processor is also configured to store, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

[0006] In at least one embodiment, a method includes obtaining, by at least one processor of an electronic device and using a transceiver, at least one image of a location including one or more parking spots from one or more remote cameras. The method further includes detecting, by the at least one processor, the one or more parking spots in the at least one image using a machine learning model. The method further includes assigning, by the at least one processor, a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier. The method further includes storing, by the at least one processor, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

[0007] In at least one embodiment, an electronic device includes at least one transceiver, at least one memory, and at least one processor coupled to the at least one memory and the at least one transceiver. The at least one memory includes instructions that, when executed by the at least one processor, cause the electronic device to establish, using the at least one transceiver, a connection with one or more remote cameras using a camera address, obtain, using the at least one transceiver, from the one or more remote cameras, at least one image of a location including one or more parking spots, create bounding boxes around the one or more parking spots in the at least one image, assign a unique identifier to each one of the bounding boxes and associate global positioning system (GPS) coordinates with each unique identifier, store the unique identifier for each one of the bounding boxes and the associated GPS coordinates in a database, wherein the database stores information on a plurality of locations each including a plurality of associated unique identifiers for parking spots and a status of each, receive, using the at least one transceiver, a request for a parking spot from a remote electronic device, obtain and load, in response to the request, a current image of the location, determine portions of the current image each corresponding to one of the bounding boxes, resize each portion of the current image corresponding to one of the bounding boxes, convert each resized portion of the current image to grayscale, convert each grayscaled and resized portion of the current image into an array of pixel values, input each array of pixel values into a logistic regression model, output, using the logistic regression model, a binary classifier for each array of pixels values, determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant, select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot from the GPS coordinates stored in the database in association with the unique identifier of the selected vacant parking spot, transmit, to the remote electronic device using the at least one transceiver, a message including a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, an occupation status, and the GPS coordinates associated with the selected vacant parking spot, periodically check whether the remote electronic device has traveled to the selected vacant parking spot, periodically check the associated occupation status of the selected vacant parking spot, determine that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot, select another vacant parking spot, transmit, to the remote electronic device using the at least one transceiver, an update including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.

[0008] Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims.

[0009] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document. The terms “transmit,” “receive,” and “communicate,” as well as derivatives thereof, encompass both direct and indirect communication. The terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation. The term “or” is inclusive, meaning and/or. The phrase “associated with,” as well as derivatives thereof, means to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, have a relationship to or with, or the like.

[0010] Moreover, various functions described below can be implemented or supported by one or more computer programs, each of which is formed from computer readable program code and embodied in a computer readable medium. The terms “application” and “program” refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted for implementation in a suitable computer readable program code. The phrase “computer readable program code” includes any type of computer code, including source code, object code, and executable code. The phrase “computer readable medium” includes any type of medium capable of being accessed by a computer, such as read only memory (ROM), random access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of memory. A “non-transitory” computer readable medium excludes wired, wireless, optical, or other communication links that transport transitory electrical or other signals. A non-transitory computer readable medium includes media where data can be permanently stored and media where data can be stored and later overwritten, such as a rewritable optical disc or an erasable memory device.

[0011] As used here, terms and phrases such as “have,” “may have,” “include,” or “may include” a feature (like a number, function, operation, or component such as a part) indicate the existence of the feature and do not exclude the existence of other features. Also, as used here, the phrases “A or B,” “at least one of A and/or B,” or “one or more of A and/or B” may include all possible combinations of A and B. For example, “A or B,” “at least one of A and B,” and “at least one of A or B” may indicate all of (1) including at least one A, (2) including at least one B, or (3) including at least one A and at least one B. Further, as used here, the terms “first” and “second” may modify various components regardless of importance and do not limit the components. These terms are only used to distinguish one component from another. For example, a first user device and a second user device may indicate different user devices from each other, regardless of the order or importance of the devices. A first component may be denoted a second component and vice versa without departing from the scope of this disclosure.

[0012] It will be understood that, when an element (such as a first element) is referred to as being (operatively or communicatively) “coupled with/to” or “connected with/to” another element (such as a second element), it can be coupled or connected with/to the other element directly or via a third element. In contrast, it will be understood that, when an element (such as a first element) is referred to as being “directly coupled with/to” or “directly connected with/to” another element (such as a second element), no other element (such as a third element) intervenes between the element and the other element.

[0013] As used here, the phrase “configured (or set) to” may be interchangeably used with the phrases “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of’ depending on the circumstances. The phrase “configured (or set) to” does not essentially mean “specifically designed in hardware to.” Rather, the phrase “configured to” may mean that a device can perform an operation together with another device or parts. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a generic-purpose processor (such as a CPU or application processor) that may perform the operations by executing one or more software programs stored in a memory device or a dedicated processor (such as an embedded processor) for performing the operations.

[0014] The terms and phrases as used here are provided merely to describe some embodiments of this disclosure but not to limit the scope of other embodiments of this disclosure. It is to be understood that the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. All terms and phrases, including technical and scientific terms and phrases, used here have the same meanings as commonly understood by one of ordinary skill in the art to which the embodiments of this disclosure belong. It will be further understood that terms and phrases, such as those defined in commonly- used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined here. In some cases, the terms and phrases defined here may be interpreted to exclude embodiments of this disclosure.

[0015] Examples of an “electronic device” according to embodiments of this disclosure may include at least one of a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop computer, a netbook computer, a workstation, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device (such as smart glasses, a head-mounted device (HMD), electronic clothes, an electronic bracelet, an electronic necklace, an electronic accessory, an electronic tattoo, a smart mirror, or a smart watch). Other examples of an electronic device include a smart home appliance. Examples of the smart home appliance may include at least one of a television, a digital video disc (DVD) player, an audio player, a refrigerator, an air conditioner, a cleaner, an oven, a microwave oven, a washer, a drier, an air cleaner, a set-top box, a home automation control panel, a security control panel, a TV box (such as SAMSUNG HOMESYNC, APPLETV, or GOOGLE TV), a smart speaker or speaker with an integrated digital assistant (such as SAMSUNG GALAXY HOME, APPLE HOMEPOD, or AMAZON ECHO), a gaming console (such as an XBOX, PLAYSTATION, or NINTENDO), an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame. Still other examples of an electronic device include at least one of various medical devices (such as diverse portable medical measuring devices (like a blood sugar measuring device, a heartbeat measuring device, or a body temperature measuring device), a magnetic resource angiography (MRA) device, a magnetic resource imaging (MRI) device, a computed tomography (CT) device, an imaging device, or an ultrasonic device), a navigation device, a global positioning system (GPS) receiver, an event data recorder (EDR), a flight data recorder (FDR), an automotive infotainment device, a sailing electronic device (such as a sailing navigation device or a gyro compass), avionics, security devices, vehicular head units, industrial or home robots, automatic teller machines (ATMs), point of sales (POS) devices, or Internet of Things (IoT) devices (such as a bulb, various sensors, electric or gas meter, sprinkler, fire alarm, thermostat, street light, toaster, fitness equipment, hot water tank, heater, or boiler). Other examples of an electronic device include at least one part of a piece of furniture or building/structure, an electronic board, an electronic signature receiving device, a projector, or various measurement devices (such as devices for measuring water, electricity, gas, or electromagnetic waves). Note that, according to various embodiments of this disclosure, an electronic device may be one or a combination of the above- listed devices. According to some embodiments of this disclosure, the electronic device may be a flexible electronic device. The electronic device disclosed here is not limited to the above-listed devices and may include new electronic devices depending on the development of technology.

[0016] In the following description, electronic devices are described with reference to the accompanying drawings, according to various embodiments of this disclosure. As used here, the term “user” may denote a human or another device (such as an artificial intelligent electronic device) using the electronic device.

[0017] Definitions for other certain words and phrases may be provided throughout this patent document. Those of ordinary skill in the art should understand that in many if not most instances, such definitions apply to prior as well as future uses of such defined words and phrases.

[0018] None of the description in this application should be read as implying that any particular element, step, or function is an essential element that must be included in the claim scope. The scope of patented subject matter is defined only by the claims. Moreover, none of the claims is intended to invoke 35 U.S.C. § 112(f) unless the exact words “means for” are followed by a participle. Use of any other term, including without limitation “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller,” within a claim is understood by the Applicant to refer to structures known to those skilled in the relevant art and is not intended to invoke 35 U.S.C. § 112(f). BRIEF DESCRIPTION OF THE DRAWINGS

[0019] For a more complete understanding of this disclosure and its advantages, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:

[0020] FIGURE 1 illustrates an example network configuration in accordance with various embodiments of this disclosure;

[0021] FIGURE 2A illustrates an example image-based parking determination system in accordance with various embodiments of this disclosure;

[0022] FIGURE 2B illustrates an example parking spot availability map in accordance with various embodiments of this disclosure;

[0023] FIGURES 3A-3E illustrate graphical user interface (GUI) examples of a parking spot availability application in accordance with various embodiments of this disclosure;

[0024] FIGURE 4 illustrates an example classification process in accordance with various embodiments of this disclosure;

[0025] FIGURES 5A and 5B illustrate a parking spot selection and navigation process in accordance with various embodiments of this disclosure;

[0026] FIGURE 6 illustrates an example parking spot status identification system in accordance with various embodiments of this disclosure; and

[0027] FIGURE 7 illustrates an example image retrieval and spot identification process in accordance with various embodiments of this disclosure.

DETAILED DESCRIPTION

[0028] FIGURES 1 through 7, discussed below, and the various embodiments used to describe the principles of the disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of this disclosure may be implemented in any suitably arranged device or system.

[0029] FIGURE 1 illustrates an example system or network configuration 100 in accordance with various embodiments of this disclosure. The embodiment of the network configuration 100 shown in FIGURE 1 is for illustration only. Other embodiments of the network configuration 100 could be used without departing from the scope of this disclosure.

[0030] According to embodiments of this disclosure, an electronic device 101 is included in the network configuration 100. The electronic device 101 can include at least one of a bus 110, a processor 120, a memory 130, an input/output (I/O) interface 150, a display 160, a communication interface 170, and a sensor 180. In some embodiments, the electronic device 101 may exclude at least one of these components or may add at least one other component. The bus 110 includes a circuit for connecting the components 120-180 with one another and for transferring communications (such as control messages and/or data) between the components.

[0031] The processor 120 includes one or more of a central processing unit (CPU), a graphics processor unit (GPU), an application processor (AP), or a communication processor (CP). The processor 120 is able to perform control on at least one of the other components of the electronic device 101 and/or perform an operation or data processing relating to communication. In accordance with various embodiments of this disclosure, the processor 120 can operate a machine learning model 149, such as a logistic regression model, convolutional neural network model, or other machine learning models stored in the memory 130 to perform predictions based on input data, such as input image data.

[0032] The memory 130 can include a volatile and/or non-volatile memory. For example, the memory 130 can store commands or data related to at least one other component of the electronic device 101. According to embodiments of this disclosure, the memory 130 can store software, programs, and/or one or more operating systems. For example, the memory can store a kernel 141, middleware 143, an application programming interface (API) 145, and/or an application program (or “application”) 147. At least a portion of the kernel 141, middleware 143, or API 145 may be denoted an operating system (OS). As described in the various embodiments herein, the memory 130 can also store a machine learning model, such as the machine learning model 149. In some embodiments, the machine learning model can be stored on another device while still being used or accessed by the electronic device 101, such as an electronic device 102 or a server 106.

[0033] The kernel 141 can control or manage system resources (such as the bus 110, processor 120, or memory 130) used to perform operations or functions implemented in other programs (such as the middleware 143, API 145, or application 147). The kernel 141 provides an interface that allows the middleware 143, the API 145, or the application 147 to access the individual components of the electronic device 101 to control or manage the system resources.

[0034] The middleware 143 can function as a relay to allow the API 145 or the application 147 to communicate data with the kernel 141, for instance. A plurality of applications 147 can be provided. The middleware 143 is able to control work requests received from the applications 147, such as by allocating the priority of using the system resources of the electronic device 101 (like the bus 110, the processor 120, or the memory 130) to at least one of the plurality of applications 147. The API 145 is an interface allowing the application 147 to control functions provided from the kernel 141 or the middleware 143. For example, the API 145 includes at least one interface or function (such as a command) for filing control, window control, image processing, or text control.

[0035] The application 147 can include an application running on the electronic device 101 such as an application that utilizes the machine learning model 149 to input data into the machine learning model 149, obtain predictions using the machine learning model 149, and output and transmit the predictions and/or other information to another electronic device, such as a remote electronic device 103. The remote electronic device 103 can execute an application that communicates over a network 162 with the electronic device 101 via the communication interface 170 of the electronic device 101. For example, the electronic device 103 can transmit requests for predictions, such as a request for a vacant parking spot in a particular location or area, from the electronic device 101. The electronic device 101 can respond by using the machine learning model 149 stored in the memory 130 to predict and/or determine one or more vacant parking spots in the particular location or area, and transmit the results via the transceiver and over the network 162 to the electronic device 103. In some embodiments, the remote electronic device 103 can first communicate with the server 106, which in turn communicates with the electronic device 101 to provide the requests from the remote electronic device 103. In such embodiments, the server 106 can perform a certain amount of preprocessing regarding the request from the remote electronic device 103.

[0036] The I/O interface 150 serves as an interface that can, for example, transfer commands or data input from a user or other external devices to other component(s) of the electronic device 101. The I/O interface 150 can also output commands or data received from other component(s) of the electronic device 101 to the user or the other external device.

[0037] The display 160 includes, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a quantum -dot light emitting diode (QLED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display. The display 160 can also be a depth-aware display, such as a multi-focal display. The display 160 is able to display, for example, various contents (such as text, images, videos, icons, or symbols) to the user. The display 160 can include a touchscreen and may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a body portion of the user.

[0038] The communication interface 170, for example, is able to set up communication between the electronic device 101 and an external electronic device (such as a first electronic device 102, a second electronic device such as at least one camera 104 disposed over the network 162, or a server 106). For example, the communication interface 170 can be connected with a network 162 or 164 through wireless or wired communication to communicate with the external electronic device. The communication interface 170 can be a wired or wireless transceiver or any other component for transmitting and receiving signals, such as signals received by the communication interface 170 from the cameras 104 including images or image data, signals from/to the server 106 regarding prediction requests or providing other information, and signals regarding predictions or requests provided to the electronic device 101 from the electronic device 103.

[0039] The wireless communication is able to use at least one of, for example, long term evolution (LTE), long term evolution-advanced (LTE-A), 5th generation wireless system (5G), millimeter-wave or 60 GHz wireless communication, Wireless USB, code division multiple access (CDMA), wideband code division multiple access (WCDMA), universal mobile telecommunication system (UMTS), wireless broadband (WiBro), or global system for mobile communication (GSM), as a cellular communication protocol. The wired connection can include, for example, at least one of a universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). The network 162 or 164 includes at least one communication network, such as a computer network (like a local area network (LAN) or wide area network (WAN)), Internet, or a telephone network. [0040] The electronic device 101 further includes one or more sensors 180 that can meter a physical quantity or detect an activation state of the electronic device 101 and convert metered or detected information into an electrical signal. For example, one or more sensors 180 can include one or more cameras or other imaging sensors, which may be used to capture images of scenes. The sensor(s) 180 can also include one or more audio sensors such as one or more microphones. The sensor(s) 180 can also include one or more buttons for touch input, a gesture sensor, a gyroscope or gyro sensor, an air pressure sensor, a magnetic sensor or magnetometer, an acceleration sensor or accelerometer, a grip sensor, a proximity sensor, a color sensor (such as a red green blue (RGB) sensor), a bio-physical sensor, a temperature sensor, a humidity sensor, an illumination sensor, an ultraviolet (UV) sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an ultrasound sensor, an iris sensor, or a fingerprint sensor. The sensor(s) 180 can further include an inertial measurement unit, which can include one or more accelerometers, gyroscopes, and other components. In addition, the sensor(s) 180 can include a control circuit for controlling at least one of the sensors included here . Any of these sensor(s) 180 can be located within the electronic device 101.

[0041] The external electronic devices 102, 103, and the server 106 each can be a device of the same or a different type from the electronic device 101. According to certain embodiments of this disclosure, the server 106 includes a group of one or more servers. Also, according to certain embodiments of this disclosure, all or some of operations executed on the electronic device 101 can be executed on another or multiple other electronic devices (such as the electronic device 102 or server 106). Further, according to certain embodiments of this disclosure, when the electronic device 101 should perform some function or service automatically or at a request, the electronic device 101, instead of executing the function or service on its own or additionally, can request another device (such as electronic device 102 or server 106) to perform at least some functions associated therewith. The other electronic device (such as electronic device 102 or server 106) is able to execute the requested functions or additional functions and transfer a result of the execution to the electronic device 101. The electronic device 101 can provide a requested function or service by processing the received result as it is or additionally. To that end, a cloud computing, distributed computing, or client-server computing technique may be used, for example. While FIGURE 1 shows that the electronic device 101 includes the communication interface 170 to communicate with the cameras 104 or server 106 via the network 162, the electronic device 101 may be independently operated without a separate communication function, according to embodiments of this disclosure.

[0042] The server 106 can include the same or similar components 110-180 as the electronic device 101 (or a suitable subset thereof). The server 106 can support to drive the electronic device 101 by performing at least one of operations (or functions) implemented on the electronic device 101. For example, the server 106 can include a processing module or processor that may support the processor 120 implemented in the electronic device 101.

[0043] Although FIGURE 1 illustrates one example of a network configuration 100, various changes may be made to FIGURE 1. For example, the network configuration 100 could include any number of each component in any suitable arrangement. In general, computing and communication systems come in a wide variety of configurations, and FIGURE 1 does not limit the scope of this disclosure to any particular configuration. While FIGURE 1 illustrates one operational environment in which various features disclosed in this patent document can be used, these features could be used in any other suitable system.

[0044] FIGURE 2A illustrates an example image-based parking determination system 200 in accordance with various embodiments of this disclosure. The system 200 includes an electronic device 201, which, in various embodiments, can be the electronic device 101, 102 or the server 106 illustrated in FIGURE 1. The electronic device 201 includes a transceiver 270, a memory 230 including a machine learning model 249, and a processor 220 communicatively coupled to the transceiver 270 and the memory 230. In various embodiments, the transceiver 270, the memory 230, the machine learning model 249, and the processor 220 can be the communication interface 170, the memory 130, the machine learning model 149, and the processor 120 illustrated in FIGURE 1, respectively.

[0045] The electronic device 201 communicates over a network 262 with one or more electronic devices 203 and one or more cameras 204 located at one or more geographic locations 205. In various embodiments, the network 262, the one or more electronic devices 203, and the one or more cameras 204 can be the network 162, the electronic device 103, and the cameras 104 illustrated in FIGURE 1, respectively.

[0046] In various embodiments of this disclosure, the electronic device 201 monitors the geographic location 205, as well as other geographic locations, by receiving images of the location via the one or more cameras 204 to track vacant and fdled parking spots. In some embodiments, the images are captured periodically, such as in response to a request from the electronic device 203. In some embodiments, the electronic device 201 receives images from the one or more cameras 204 continuously, to constantly monitor the location. In some embodiments, the images are still images. In some embodiments, the images can be a video feed. The geographic location 205 monitored by the one or more cameras 204 can be a parking lot, a street, such as in a business or residential area, or other locations where parking spots are available.

[0047] In some embodiments, when a geographic location 205 for monitoring is first established, images received from the one or more cameras 204 can be used in an initial configuration process to establish the locations of parking spots in the image frame received. For example, as illustrated in FIGURE 2A, an image from a particular angle or vantage point as defined by the physical placement of the one or more cameras 204 at the geographic location 205 can be received by the electronic device 201 via the transceiver 270 and under the control ofthe processor 220. The received images can be stored in the memory 230. Once the images are received, the processor 220 analyzes at least one image 202 using the trained machine learning model 249 to identify and draw bounding boxes around each parking spot within the frame of the image 202.

[0048] The image 202 can also be associated with other information such as a location name associated with the geographic location 205, a camera identification, such as “camera 1,” and date or timestamp, and image information such as resolution, pixel information, or other information. The bounding boxes can be associated with a unique identifier representing the particular parking spot defined by the bounding box. Additionally, each unique identifier can be associated with global positioning system (GPS) coordinates to define a specific location for each parking spot. As illustrated in FIGURE 2B, which illustrates an example parking spot availability map 225 in accordance with various embodiments of this disclosure, in some embodiments, the parking spot availability map 225 is created that tracks the number of empty spots and maps relative physical locations of all spots, along with additional status identifiers for each spot. In the example illustrated in FIGURE 2B, the status identifiers are placed within the availability map at locations corresponding to the image frame 202 depicted in FIGURE 2A. In some embodiments, the electronic device 201 performs the above process of configuring the scene image after a request from the electronic device 203 is received.

[0049] The electronic device 201 is configured to receive requests from other electronic devices such as the electronic device 203 for parking spot availability in the geographic location 205 or other geographic locations. In various embodiments, requests sent from the electronic device 203 to the electronic device 201 can be formatted and transmitted using a proprietary application, such as a parking availability application, associated with the owner of the electronic device 201. When the electronic device 201 receives a request for a parking spot from the electronic device 203, the processor 220, using the machine learning model 249, uses a current image of the scene to determine if there are vacant spots available at the location.

[0050] In some embodiments as described in the various embodiments of this disclosure, after configuration of the image 202 is performed and the bounding boxes established within the frame, a current image can be received from the camera 204 by the electronic device 201, with the image being updated, including the portions of the image defined by the bounding boxes, to a current view of the location. As such, parking spots defined by the bounding boxes are updated with respect to the current scene, such as if a bounding box or spot now has, or does not have, a vehicle present.

[0051] If a vacant spot is available, the processor 220 causes a transmission to be sent via the transceiver 270 over the network 262 to the electronic device 203 to present the available parking spot in a user interface of the proprietary application on the electronic device 203. In some embodiments, the proprietary application issues the request for a spot and receives the response from the electronic device 201 , including the GPS coordinates for the vacant spot. The electronic device 203 can then launch or execute another application on the electronic device 203 used for GPS location searches and navigation. In some embodiments, navigation using the GPS location can be performed within the same application that requested the parking spot availability, such as by sending an API call to servers associated with the GPS navigation application and presenting the navigation and map within the same application that requested the parking spot availability. Example GPS location and navigation applications include GOOGLE MAPS, WAZE, APPLE MAPS, etc. Once the electronic device 203 receives the GPS coordinates, an assisted navigation feature, including voiced directions, can be used to navigate the user of the electronic device 203 to the parking spot associated with the GPS coordinates. In some embodiments, when multiple vacant spots are present, one can simply be selected at random or based on an established priority for the spots at the location. [0052] FIGURES 3A and 3B illustrate graphical user interface (GUI) examples of a parking spot availability application in accordance with various embodiments of this disclosure. FIGURE 3 A illustrates a first interface 302, FIGURE 3B illustrates a second interface 304, FIGURE 3C illustrates a third interface 306, FIGURE 3D illustrates a fourth interface 308, and FIGURE 3E illustrates a fifth interface 310. In various embodiments of this disclosure, the interfaces 302 and 304 of a parking spot availability application executed by a user’s electronic device, such as electronic device 203 in FIGURE 2A, are used to request parking spot availability at a particular location or geographical area, and to trigger navigation to a parking spot at the particular location or geographical area.

[0053] For example, as shown in FIGURE 3A, the interface 302 includes interface objects associated with a geographical area and each defining a sub area or specific location within the geographical area. In some embodiments, the interface 302 also specifies how many parking spots are currently available across each area defined by each interface object. In some embodiments, when a user highlights one of the interface objects such as by a touch input, a mouseover, etc., the number of available parking spaces for the location associated with the highlighted user interface object is displayed, and periodically updated while the interface object remains highlighted.

[0054] As shown in the example interface 304 in FIGURE 3B, upon selection of one of the interface objects illustrated in FIGURE 3A, an available parking spot having associated coordinates is selected and navigation window shown in interface 304 is displayed, allowing navigation to the parking spot location either manually by the user or using navigation assistance. In some embodiments, the interface 304 can also display navigational information such as navigation lines, routes, estimated travel time and/or distance. In some embodiments, the interface 304 can also display other information on the parking spot, such as spot type, e.g., handicap spot, free spot, paid spot, and/or EV spot and/or spot pricing information, e.g., $5.00 per hour.

[0055] As shown in the example interface 306 in FIGURE 3C, a current location of the electronic device is displayed on the map, with a current location bubble displayed above a graphical representation of a vehicle at the GPS location of the electronic device, in this example. As shown in the example interface 308 in FIGURE 3D, the interface 308 includes one or more pins indicating the location of available parking spots on the map at a location requested by the electronic device. In the example interface 308, the interface 308 displays a vehicle and a navigational line or route to one of the pins. In this example, a bubble is displayed for the pin associated with the parking spot, with the bubble showing the address of the location and a parking spot type, a handicap spot in this example. As shown in FIGURE 3E, the interface 310 shows a bubble over the other pin indicating the address and a price for parking at the parking spot, $5.00 per hour in this example.

[0056] FIGURE 4 illustrates an example classification process 400 in accordance with various embodiments of this disclosure. For ease of explanation, the process 400 may be described as being executed or otherwise used by the processor(s) 120 of any of the electronic devices 101, 102, or the server 106 in FIGURE 1, or the processor 220 of electronic device 201 in FIGURE 2. However, the process 400 may be used by any suitable device(s) and in any suitable system.

[0057] The process 400 begins at block 402, where the processor obtains, using atransceiver connected to a network, from one or more remote cameras, at least one image of a location including one or more parking spots. At block 404, the processor creates bounding boxes around the one or more parking spots in the at least one image, assigns a unique identifier to each one of the bounding boxes, and associates global positioning system (GPS) coordinates with each unique identifier. Bounding boxes can be created by opening an image into a plotting area and plotting each comer of each bounding box. At block 406, the processor receives, using the transceiver, a request for a parking spot from a remote electronic device. In various embodiments, the request can include a geographic area, a specific location name, GPS coordinates, or any other location information that allows the processor to determine an area to search for parking spots, as well as to determine which cameras to retrieve additional images.

[0058] The processor then obtains and loads, in response to the request, a current image of the location from the one or more cameras. At block 408, the processor resizes each portion or car patch of the current image associated with one of the bounding boxes, such as resizing the image to a 25 x 25 pixels image, and grayscales each resized portion of the current image. In some embodiments, as part of resizing the image, the processor can also crop the image to reduce the image size. Grayscaling the image reduces the amount of data associated with the image and associated with each pixel. Instead of multiple values being associated with each pixel, such as separate RGB values, grayscaling the image provides a single 8-bit value, such as from 0 to 255, representing each pixel. This allows for data input into the machine learning model for each car patch to be efficient for fast processing of the car patch image by the machine learning model, since the car patch image includes a smaller number of pixels, each with a reduced amount of information. However, even though the data is minimized, the machine learning model still performs accurate predictions based on the data, as information such as color values are not used in the predictions.

[0059] At block 410, the processor converts each grayscaled and resized portion or car patch of the current image into an array of pixel values, such as by converting the images to numpy arrays, and inputs each array of pixel values into a machine learning model. At block 412, the processor, using the machine learning model, analyzes each input array. At block 414, the processor outputs, using the machine learning model, a binary classifier for each array of pixels values.

[0060] In some embodiments, the machine learning model is a trained logistic regression model that utilizes logistic regression to provide fast classification results for the systems described in this disclosure. Using a logistic regression model over other types of models provides a lightweight model for outputting binary classifiers (occupied/vacant) relating to a spot, increasing the efficiency and processing speed of the disclosed systems while maintaining accurate classification results. Once the machine learning model is trained, to increase the efficient and lightweight nature of the model, the model can be pickled. Pickling the trained model refers to serializing the model such as by converting the model into a character stream and storing the pickled model in the system, such as on electronic device 101 or server 106. The pickled model or stream includes all information used to reconstruct or de-serialize the model by a script or other application function when the model is needed, such as upon receiving a parking spot request. Pickling also allows the model to be a self- referencing object, i.e., pickling stores the model once, and ensures that all other references point to the master copy.

[0061] In embodiments using a logistic regression model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function. That is, using logistic regression involves predicting how close to 0 or 1 (negative or positive - vacant or occupied) an input is. In some embodiments, the inputs provided to the logistic regression model are mapped to probabilities using a sigmoid function, such as f{x) =

such that the hypothesis for the model is s(Z) = s(b₀ + b C). In various embodiments, training the binary classifier logistic regression model minimizes the following cost function:

1 ⁷¹ min-w^Tw + C ^ log(exp (— y;(Y w + c) + 1)

^W,C i=i

[0062] In some embodiments, the machine learning model can be a convolutional neural network trained to identify vacant or occupied parking spots and to create a bounding box around each detected spot, such as based on recognizable or regionally common features (pixel intensity, etc.). The machine learning model then can classify the detected object based on the model’s training. For example, the convolutional neural network receives an input image, and convolutional layer fdters (feature detectors) extract image features from the input image, sliding through the elements of the image and multiplying each element by the fdter, to product a feature map (a matrix). In some embodiments, an activation function, such as a Rectified Linear Unit (ReLU) function, is performed on the feature map to replace negative pixels values to add non-linearity to the model. In some embodiments, the machine learning model includes pooling layers between convolutional layers to reduce the dimensionality of each feature map, reducing noisy and redundant convolutions. In some embodiments, the convolutional neural network also includes at least one fully connected layer that receive the outputs from the convolution layers and classifies each spot within the image as occupied or vacant.

[0063] The machine learning model can be of various types, such as a logistic regression model, a deep and/or a region based CN s, a single shot detector (SSD) model, a You Only Look Once (Y OLO) model, or other image recognition and object detection models, without departing from the scope of this disclosure. [0064] The trained machine learning model takes the inputs generated from the car patches and predicts the probability that the parking spot associated with the car patch is occupied, and outputs a binary classifier. At decision block 416, the processor determines if a parking spot is vacant and available based on the output binary classifiers. If not, the process 400 moves to block 418. At block 418, the processor issues a recommendation to the remote electronic device to check a nearby location for other parking spots. The process 400 then loops back to block 406, where the processor receives a request from the remote electronic device to find a spot in the recommended location. If, at decision block 416, the processor determines a parking spot is vacant and available, at block 420, the processor selects a vacant parking spot, determines GPS coordinates associated with the selected vacant parking spot, and transmits, to the remote electronic device using the transceiver, a message including the GPS coordinates associated with the selected vacant parking spot. In some embodiments, the message includes a JavaScript Object Notation (JSON) object that includes a spot ID and the latitude and longitude for the spot. In some embodiments, the JSON object can also include a timestamp, a spot size, a spot price (e.g., $5.00 per hour), location information such as an address, city, county, etc., and a spot type such as a handicap spot, free spot, paid spot, and/or EV spot, etc. The process 400 ends at block 422.

[0065] Although FIGURE 4 illustrates one example of a classification process 400, various changes may be made to FIGURE 4. For example, while shown as a series of steps, various steps in FIGURE 4 can overlap, occur in parallel, occur in a different order, or occur any number of times. For instance, in some embodiments, blocks 402 and 404 can occur after block 406, such that images are processed and bounding boxes within the image created in response to receiving a request from the remote electronic device. As another example, in some embodiments, at block 418, the issued recommendation for another parking location can include first determining by the processor whether the recommended location has a parking spot, and then notifying the electronic device, with the electronic device requesting the recommended parking spot at block 406. In some embodiments, at block 418, the issued recommendation includes a verification request that the user wishes to see other locations, and the request at block 406 includes such a verification, with the processor then determining if spots are available at the other location.

[0066] FIGURES 5A and 5B illustrate a parking spot selection and navigation process 500 in accordance with various embodiments of this disclosure. For ease of explanation, the process 500 may be described as being executed or otherwise used by the processor(s) 120 of any of the electronic devices 101, 102, or the server 106 in FIGURE 1, or the processor 220 of electronic device 201 in FIGURE 2. However, the process 500 may be used by any suitable device(s) and in any suitable system.

[0067] The process 500 begins at block 502, where the processor receives, using the transceiver, a request for a parking spot from a remote electronic device. In various embodiments, the request can include a geographic area, a specific location name, GPS coordinates, or any other location information that allows the processor to determine an area to search for parking spots, as well as to determine which cameras to retrieve images. At block 504, the processor outputs, using the machine learning model, one or more binary classifiers indicating occupied or vacant status of one or more parking spots in the specified geographic area, such as also described with respect to FIGURE 4.

[0068] At decision block 506, the processor determines if a parking spot is vacant and available based on the output binary classifiers. If not, the process 500 moves to block 508. At block 508, the processor issues a recommendation to the remote electronic device to check in a nearby location for other parking spots. The process 500 then loops back to block 502, where the processor receives a request from the remote electronic device to find a spot in the recommended location. If, at decision block 506, the processor determines a parking spot is vacant and available, at block 510, the processor selects a vacant parking spot, sets a status of the spot as assigned, determines GPS coordinates associated with the selected vacant parking spot, and transmits, to the remote electronic device using the transceiver, a message including the GPS coordinates associated with the selected vacant parking spot.

[0069] In various embodiments of this disclosure, setting a spot status as assigned, such as in association with the ID for the spot, prevents the processor from providing the same spot to two different remote devices requesting a parking spot. It will be understood that, in embodiments where the parking spots assigned are in general parking areas, the processor and/or system does not reserve a parking spot for the user, e.g., a third party driver could still take the spot. However, setting the assigned status will prevent the processor and/or system from allowing two users of the system to come into conflict.

[0070] At decision block 512, the processor determines if a timeout occurs. In various embodiments, if a user of the remote device does not send a verification that navigation has commenced after a period of time, if further communications are not received after a period of time such as in blocks 524 and 530 discussed below, or if the processor determines the remote device has not traveled towards the spot destination after a period of time, such as related to block 530, the processor determines the timeout occurs. If the timeout occurs, the process 500 moves to block 514 to remove the assigned status of for the spot selected at block 510. This allows that spot to be reassigned to another user or device. The process 500 then ends at block 516.

[0071] If, at decision block 512, the processor determines the timeout has not occurred, the process 500 moves to block 518. At block 518, the remote device begins navigation based on the transmitted GPS coordinates to the spot selected at block 510. At decision block 520, the processor determines whether to track the progress of the remote device. In some embodiments, the processor can track the progress of the remote device in order to prevent the user from arriving at a spot that has already been filled by another driver. Deciding whether to track the progress at decision block 520 can either be chosen at the system level or by the user as part of the services selected by, or based the preferences of, the user. If, at decision block 520, the processor does not track the progress, the processor removes the assigned status for the spot after a period of time. As such, the processor assumes the user of the remote device traveled to the selected spot and parked. Removing the assigned status frees up the spot to be reassigned upon a later request, but of course, based on the images being received of the parking spot, if the vehicle of the user of the remote device is still parked in the spot, the spot will be considered occupied and will not be reassigned until the processor determines using the machine learning model that the spot has been vacated, avoiding any overlapping assignment of spots. The process 500 then ends at block 538.

[0072] If, at decision block 520, the processor tracks progress, the process 500 can perform different tracking methods. For example, at block 524, the processor receives confirmation from the remote device that the user is close to the spot, such as in response to a user selecting a button to send the confirmation, in response to the user responding to a notification that pops up in response to the GPS coordinates of the remote device being within a predetermined distance from the spot, as determined by the application executed by the remote device, or in response to an automatic confirmation based on such GPS coordinates. At decision block 526, the processor determines, based on newly received image data of the spot, if the selected spot has been taken before the user could reach the spot, such as if another driver that does not use the systems described herein independently sees and takes the spot. If so, the process moves to block 528 where the processor finds a new vacant spot in the same area as the spot selected in block 510. In some embodiments, if no vacant spots are left, the processor can issue a recommendation for another nearby location and spot therein, such as similarly described with respect to block 508. The process 500 then loops back to block 518 to begin navigation to the new spot. If, at decision block 526, the processor determines the spot has not been taken, the process 500 moves to block 536. At block 536, the processor removes the assigned status from the spot so that it can be reassigned once the processor determines from later-acquired image data using the machine learning model that the spot has been vacated. The process 500 then ends at block 538.

[0073] As another example, the processor can track the progress of the user or the remote device at block 530 by periodically receiving GPS data from the remote device. The GPS data can be received rapidly, such as every few seconds, or spaced out to preserve bandwidth and processing, such as in intervals of multiple minutes. The intervals for transmitting GPS data can also be tailored to the travel distance. For example, the remote device may transmit GPS data more frequently for shorter distances, and transmit less frequently for longer distances. At decision block 532, the processor determines, based on newly received image data of the spot, if the selected spot has been taken before the user could reach the spot, such as if another driver that does not use the systems described herein independently sees and takes the spot. If so, the process moves to block 528 where the processor finds a new vacant spot in the same area as the spot selected in block 510. In some embodiments, if no vacant spots are left, the processor can issue a recommendation for another nearby location and spot therein, such as similarly described with respect to block 508. The process 500 then loops back to block 518 to begin navigation to the new spot.

[0074] If, at decision block 532, the processor determines the spot has not been taken, the process 500 moves to decision block 534. At decision block 534, the processor determines if the remote device has arrived at the spot based on the periodically received GPS data. If not, the process 500 loops back to block 530 to continue tracking the remote device. If, at decision block 534, the processor determines the remote device has arrived at the spot, the process 500 moves to block 536. At block 536, the processor removes the assigned status from the spot so that it can be reassigned once the processor determines from later- acquired image data using the machine learning model that the spot has been vacated. The process 500 then ends at block 538.

[0075] Although FIGURES 5A and 5B illustrate one example of a parking spot selection and navigation process 500, various changes may be made to FIGURES 5A and 5B. For example, while shown as a series of steps, various steps in FIGURES 5A and 5B can overlap, occur in parallel, occur in a different order, or occur any number of times. For instance, in some embodiments, at block 508, the issued recommendation for another parking location can include first determining by the processor whether the recommended location has a parking spot, and then notifying the electronic device, with the electronic device requesting the recommended parking spot at block 502. In some embodiments, at block 508, the issued recommendation includes a verification request that the user wishes to see other locations, and the request at block 502 includes such a verification, with the processor then determining if spots are available at the other location.

[0076] FIGURE 6 illustrates an example parking spot status identification system 600 in accordance with various embodiments of this disclosure. The system 600 includes an electronic device 601, which, in various embodiments, can be the electronic device 101, 102 or the server 106 illustrated in FIGURE 1. The electronic device 601 includes a transceiver 670, a memory 630 including a machine learning model 649, and a processor 620 communicatively coupled to the transceiver 670 and the memory 630. In various embodiments, the transceiver 670, the memory 630, the machine learning model 649, and the processor 620 can be the communication interface 170, the memory 130, the machine learning model 149, and the processor 120 illustrated in FIGURE 1, respectively.

[0077] The electronic device 601 communicates over a network 662 with one or more cameras 604 located at one or more geographic locations 605. In various embodiments, the network 662 and the one or more cameras 604 can be the network 162 and the cameras 104 illustrated in FIGURE 1, respectively. [0078] In various embodiments of this disclosure, the electronic device 601 receives images 602 of a location via the one or more cameras 604 to track vacant and filled parking spots. In some embodiments, the images are captured periodically, such as in response to a request from another electronic device 613. In some embodiments, the electronic device 601 receives images from the one or more cameras 604 continuously, to constantly monitor the location. In some embodiments, the images are still images. In some embodiments, the images can be a video feed. The geographic location monitored by the one or more cameras 604 can be a parking lot, a street, such as in a business or residential area, or other locations where parking spots are available.

[0079] An image from a particular angle or vantage point as defined by the physical placement of the one or more cameras 604 at the geographic location can be received by the electronic device 601 via the transceiver 670 and under the control of the processor 620. The received images can be stored in the memory 630, or in another storage location, such as a remotely accessible server system. Once the images are received, the processor 620 analyzes at least one image of the images 602 using the trained machine learning model 649 to identify, draw bounding boxes around each parking spot within the frame of the image, and classify each parking spot.

[0080] In various embodiments of this disclosure, the system 600 provides a database 607 providing the occupancy of every parking space within the view of each of the cameras 604. In some embodiments, existing camera systems or infrastructures can be used to receive the images 602, such as public camera systems (e.g., municipal camera systems) as well as private camera systems. For example, the municipalities may provide access to its municipal camera system to the electronic device 601, or such camera systems may be allowed to be publicly addressable. Based on the images 602 received by the electronic device 601, the database is continuously updated, storing in relation to each detected parking spot various data 609. In various embodiments of this disclosure, the data 609 includes an object ID for the spot and a status of the spot. For example, each spot can have a status of “vacant,” “occupied,” or, in some embodiments, “assigned,” such as described with respect to FIGURES 5 A and 5B. The database 607 can also store information in association with each spot such as a location (e.g., either a specific location name, address, or location ID), GPS coordinates, and a camera address for the camera or cameras that provides images of the spot, such as an IP address and/or a Uniform Resource Uocator (URL). Cameras or camera systems can be addressable using the camera address, and images retrieved using the addressable cameras. For example, an image can be retrieved from an addressable camera using a GET command that references the camera address and performs a snapshot of the camera feed, e.g., a video camera feed, stores the snapshot, and performs any conversion steps on the image (e.g., grayscale, resizing, etc.). In some embodiments, the database 607 can include other information such as a camera ID, such as “camera 1,” a date or timestamp of the last image for the spot, and image information such as resolution, pixel information, spot pricing, or other information. In this way, the system 600 maintains up-to-date, real-time, statuses for a large number of parking spots across the world.

[0081] As also described in various other embodiments of this disclosure, the electronic device 601 is configured to receive requests from other electronic devices 613 for parking spot availability. In various embodiments, requests sent from the electronic device 613 to the electronic device 601 can be formatted and transmitted using a proprietary application, such as a parking availability application, associated with the owner of the electronic device 601. When the electronic device 601 receives a request for a parking spot from another electronic device 613, the processor 620 can access the database 607 to retrieve the data 609, and transmit one or more messages containing various combinations of information from data 609. In some embodiments, the messages can include a JavaScript Object Notation (JSON) object that includes a spot ID and the latitude and longitude for the spot. In some embodiments, the JSON object can also include a timestamp, a spot size, a spot price (e.g., $5.00 per hour), location information such as an address, city, county, etc., and a spot type such as a handicap spot, free spot, paid spot, and/or EV spot, etc.

[0082] The one or more messages can be sent to the requesting electronic device 613, to provide information to an application executed by to the requesting electronic device 613. For example, if a vacant spot is available, the processor 620 causes a transmission to be sent via the transceiver 670 over the network 662 to the electronic device 603 to present the available parking spot in a user interface of the proprietary application on the electronic device 613. Using the associated GPS coordinates, the electronic device 613 can then launch or execute another application on the electronic device 613 used for GPS location searches and navigation. In some embodiments, navigation using the GPS location can be performed within the same application that requested the parking spot availability, such as by sending an API call to servers of a navigational system 611 associated with the GPS navigation application and presenting the navigation and map within the same application that requested the parking spot availability. Example GPS location and navigation applications include GOOGLE MAPS, WAZE, APPLE MAPS, etc. Once the electronic device 613 receives the GPS coordinates, an assisted navigation feature, including voiced directions, can be used to navigate the user of the electronic device 613 to the parking spot associated with the GPS coordinates. In some embodiments, when multiple vacant spots are present at a location, a spot can be selected at random or based on an established priority for the spots at the location.

[0083] FIGURE 7 illustrates an example image retrieval and spot identification process 700 in accordance with various embodiments of this disclosure. For ease of explanation, the process 700 may be described as being executed or otherwise used by the processor(s) 120 of any of the electronic devices 101, 102, or the server 106 in FIGURE 1, or the processor 220 of electronic device 201 in FIGURE 2. However, the process 700 may be used by any suitable device(s) and in any suitable system.

[0084] The process 700 begins at block 702. At block 702, the processor establishes a connection with an addressable camera, such as an addressable camera 604. At block 704, the processor retrieves an image of a location from the addressable camera and analyzes the image. In some embodiments, existing camera systems or infrastructures can be used to retrieve the image, such as public camera systems (e.g., municipal camera systems) as well as private camera systems. For example, the municipalities may provide access to its municipal camera system, or such camera systems may be allowed to be publicly addressable. For example, an image can be retrieved from an addressable camera using a GET command that references the camera address and performs a snapshot of the camera feed, e.g., a video camera feed, stores the snapshot, and performs any conversion steps on the image (e.g., grayscale, resizing, etc.). Once retrieved, the processor analyzes the image using a trained machine learning model such as those described in this disclosure to identify each parking spot in the image, draw bounding boxes around each parking spot within the image, and classify each parking spot, e.g., vacant or occupied.

[0085] At block 706, the processor stores, in one or more databases, the status of each identified parking spots, e.g., vacant or occupied, in association with a spot ID for the identified parking spot. In various embodiments, the database can also store information in association with each spot such as a location (e.g., either a specific location name, address, or location ID), GPS coordinates, and a camera address for the camera or cameras that provides images of the spot, such as an IP address and/or a URL. In some embodiments, the database can include other information such as a camera ID, such as “camera 1,” a date or timestamp of the last image for the spot, and image information such as resolution, pixel information, spot pricing, or other information. In this way, the system maintains up-to-date, real-time, statuses for a large number of parking spots across the world.

[0086] At block 708, the processor receives a request for parking spot availability for a particular geographic location or area. In various embodiments, requests sent from an and received by the processor can be formatted and transmitted using a proprietary application, such as a parking availability application. When the processor receives a request for a parking spot from the other electronic device, the processor can access the database to retrieve the data associated with parking spots in the requested geographic location or area. It will be understood that the processor can continuously or periodically retrieve images of the geographic location or area, and can thus be updating the database continuously or periodically during the process 700. At decision block 710, the processor determines if the geographic location or area includes a vacant parking spot, based on the information stored in the database, e.g., the stored statuses for the parking spots at the geographic location or area, and/or based on retrieved images. If not, at block 712, the processor issues a recommendation to the requesting electronic device of another nearby location that has an available parking spot, or to have the processor check in the nearby location for available parking spots. The process 700 then loops back to block 702, so that the processor retrieves updated location information for the recommended location, and receives a request from the remote electronic device to find a spot in the recommended location. In some embodiments, the process 700 can loop from block 712 to block 708, such as if the processor already recommended a parking spot in the nearby location. In some embodiments, the process 700 can move from block 712 to block 714, such as if the recommended location has an available parking spot and the processor receives confirmation to navigate to that spot.

[0087] If, at decision block 710, the processor determines a parking spot is vacant and available, at block 714, the processor selects a vacant parking spot, sets a status of the spot as assigned, and transmits a data object to the requesting electronic device. In various embodiments of this disclosure, setting a spot status as assigned, such as in association with the ID for the spot, prevents the processor from providing the same spot to two different remote devices requesting a parking spot. It will be understood that, in embodiments where the parking spots assigned are in general parking areas, the processor and/or system does not reserve a parking spot for the user, e.g., a third-party driver could still take the spot. However, setting the assigned status will prevent the processor and/or system from allowing two users of the system to come into conflict during navigation and prior to setting the status as “occupied” once the driver completes navigation to the spot.

[0088] In some embodiments, the data object can include a JSON object that includes a spot ID and the latitude and longitude for the spot. In some embodiments, the JSON object can also include atimestamp, a spot size, a spot price (e.g., $5.00 per hour), location information such as an address, city, county, etc., and a spot type such as a handicap spot, free spot, paid spot, and/or EV spot, etc. The data object can be sent to the requesting electronic device in one or more messages, to provide information to an application executed by to the requesting electronic device. For example, if a vacant spot is available, the processor causes a transmission to be sent to the electronic device to present the available parking spot in a user interface of the proprietary application on the electronic device. Using the associated GPS coordinates, the electronic device can then launch or execute another application on the electronic device used for GPS location searches and navigation. In some embodiments, navigation using the GPS location can be performed within the same application that requested the parking spot availability, such as by sending an API call to servers associated with the GPS navigation application and presenting the navigation and map within the same application that requested the parking spot availability. Example GPS location and navigation applications include GOOGLE MAPS, WAZE, APPLE MAPS, etc. Once the electronic device receives the GPS coordinates, an assisted navigation feature, including voiced directions, can be used to navigate the user of the electronic device to the parking spot associated with the GPS coordinates. In some embodiments, when multiple vacant spots are present at a location, a spot can be selected at random or based on an established priority for the spots at the location. [0089] At decision block 716, the processor determines if navigation to the selected spot is successful. This can be performed by tracking the GPS coordinates of the requesting device, via confirmation, etc., as described in the various embodiments of this disclosure. If, at decision block 716, the processor determines navigation is not successful, such as if the user of the requesting device abandons the parking attempt, the process 700 loops back to block 702. If, at decision block 716, the processor determines navigation is successful, at block 718 the processor sets the status for the spot as occupied. In some embodiments, setting the status as occupied at block 718 is performed in response to the determination of successful navigation, in response to detecting a vehicle in the parking spot from one or more retrieved images, or both. At block 720, the processor tracks the occupation status of the parking spot and processes payment for parking. For example, the user of the requesting electronic device can use the application on the requesting electronic device to submit payment, which can be performed automatically by the processor and the requesting electronic device, in association with other payment systems such as banks, Internet payment services, etc., at various points in the process, such as when navigation is successful at block 716, such as if the parking spot has a one-time parking charge, or at the end of the process 700, such as if the parking spot has a periodic charge such as a set amount per hour of parking. In some embodiments, the processor can track how long the vehicle remains parked using the GPS coordinates of the requesting device, subsequent retrieved images, e.g., tracking when the spot first becomes vacant after the user parks, or both. The process 700 loops back to block 702 to continue tracking parking spot availability for one or more geographic locations. [0090] Although FIGURE 7 illustrates one example of an image retrieval and spot identification process 700, various changes may be made to FIGURE 7. For example, while shown as a series of steps, various steps in FIGURE 7 can overlap, occur in parallel, occur in a different order, or occur any number of times. For instance, in some embodiments, at block 712, the issued recommendation for another parking location can include first determining by the processor whether the recommended location has a parking spot, and then notifying the electronic device, with the electronic device requesting the recommended parking spot at block 708. In some embodiments, at block 712, the issued recommendation includes a verification request that the user wishes to see other locations, and the request at block 708 includes such a verification, with the processor then determining if spots are available at the other location.

[0091] In one example embodiment, an electronic device comprises at least one transceiver, at least one memory, and at least one processor coupled to the at least one memory and the at least one transceiver, wherein the at least one processor is configured to obtain, using the at least one transceiver, from one or more remote cameras, at least one image of a location including one or more parking spots, detect the one or more parking spots in the at least one image using a machine learning mode, assign a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier, and store, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database. [0092] In one or more of the above examples, to obtain the at least one image of the location, the at least one processor is further configured to establish, using the at least one transceiver, a connection with the one or more remote cameras using a camera address.

[0093] In one or more of the above examples, the at least one processor is further configured to receive, using the at least one transceiver, a request for a parking spot from a remote electronic device, select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot, and transmit, to the remote electronic device using the at least one transceiver, a message including the associated occupation status and the associated GPS coordinates for the selected vacant parking spot. [0094] In one or more of the above examples, to detect the one or more parking spots in the at least one image using the machine learning model, the at least one processor is further configured to create bounding boxes around the one or more parking spots in the at least one image, and assign the unique identifier to each one of the bounding boxes.

[0095] In one or more of the above examples, to select the vacant parking spot and determine the associated GPS coordinates for the selected vacant parking spot, the at least one processor is further configured to obtain a current image of the location, determine portions of the current image each corresponding to one of the bounding boxes, convert each of the portions of the current image into an array of pixel values, input each array of pixel values into the machine learning model, output, using the machine learning model, a binary classifier for each array of pixels values, and determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant.

[0096] In one or more of the above examples, to convert each of the portions of the current image into the array of pixel values, the at least one processor is further configured to resize each of the portions of the current image, convert each resized portion of the current image to grayscale, and convert each grayscaled and resized portion of the current image into the array of pixel values.

[0097] In one or more of the above examples, the transmitted message includes a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, the associated occupation status, and the associated GPS coordinates for the selected vacant parking spot.

[0098] In one or more of the above examples, the transmitted message causes a navigational application to display directions to the selected vacant parking spot and information associated with the selected vacant parking spot.

[0099] In one or more of the above examples, the at least one processor is further configured to periodically check whether the remote electronic device has traveled to the selected vacant parking spot, periodically check the associated occupation status of the selected vacant parking spot, determine that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot, select another vacant parking spot, and transmit, to the remote electronic device using the at least one transceiver, an update including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot. [0100] In one or more of the above examples, to select the vacant parking spot and determine the GPS coordinates for the selected vacant parking spot, the at least one processor is further configured to retrieve the vacant parking spot and the associated GPS coordinates from the database using the unique identifier for the selected vacant parking spot.

[0101] In another example embodiment, a method comprises obtaining, by at least one processor of an electronic device and using a transceiver, at least one image of a location including one or more parking spots from one or more remote cameras, detecting, by the at least one processor, the one or more parking spots in the at least one image using a machine learning model, assigning, by the at least one processor, a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier, and storing, by the at least one processor, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

[0102] In one or more of the above examples, obtaining the at least one image of the location includes establishing, by the at least one processor and using the transceiver, a connection with the one or more remote cameras using a camera address.

[0103] In one or more of the above examples, the method further comprises receiving, by the at least one processor and using the transceiver, a request for a parking spot from a remote electronic device, selecting, by the at least one processor, a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot, and transmitting, by the at least one processor, to the remote electronic device using the transceiver, a message including the associated occupation status and the associated GPS coordinates for the selected vacant parking spot.

[0104] In one or more of the above examples, detecting the one or more parking spots in the at least one image using the machine learning model includes creating, by the at least one processor, bounding boxes around the one or more parking spots in the at least one image, and assigning, by the at least one processor, the unique identifier to each one of the bounding boxes.

[0105] In one or more of the above examples, selecting the vacant parking spot and determining the associated GPS coordinates for the selected vacant parking spot includes obtaining, by the at least one processor, a current image of the location, determining, by the at least one processor, portions of the current image each corresponding to one of the bounding boxes, converting, by the at least one processor, each of the portions of the current image into an array of pixel values, inputting, by the at least one processor, each array of pixel values into the machine learning model, outputting, by the at least one processor and using the machine learning model, a binary classifier for each array of pixels values, and determining, by the at least one processor, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant.

[0106] In one or more of the above examples, converting each of the portions of the current image into the array of pixel values includes resizing, by the at least one processor, each of the portions of the current image, converting, by the at least one processor, each resized portion of the current image to grayscale, and converting, by the at least one processor, each grayscaled and resized portion of the current image into the array of pixel values.

[0107] In one or more of the above examples, the transmitted message includes a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, the associated occupation status, and the associated GPS coordinates for the selected vacant parking spot.

[0108] In one or more of the above examples, the method further comprises periodically checking, by the at least one processor, whether the remote electronic device has traveled to the selected vacant parking spot, periodically checking, by the at least one processor, the associated occupation status of the selected vacant parking spot, determining, by the at least one processor, that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot, selecting, by the at least one processor, another vacant parking spot, and transmitting, by the at least one processor and using the transceiver, an update to the remote electronic device including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.

[0109] In one or more of the above examples, selecting the vacant parking spot and determining the GPS coordinates for the selected vacant parking spot includes retrieving, by the at least one processor, the vacant parking spot and the associated GPS coordinates from the database using the unique identifier for the selected vacant parking spot.

[0110] In another example embodiment, an electronic device comprises at least one transceiver, at least one memory, and at least one processor coupled to the at least one memory and the at least one transceiver, wherein the at least one memory includes instructions that, when executed by the at least one processor, cause the electronic device to establish, using the at least one transceiver, a connection with one or more remote cameras using a camera address, obtain, using the at least one transceiver, from the one or more remote cameras, at least one image of a location including one or more parking spots, create bounding boxes around the one or more parking spots in the at least one image, assign a unique identifier to each one of the bounding boxes and associate global positioning system (GPS) coordinates with each unique identifier, store the unique identifier for each one of the bounding boxes and the associated GPS coordinates in a database, wherein the database stores information on a plurality of locations each including a plurality of associated unique identifiers for parking spots and a status of each, receive, using the at least one transceiver, a request for a parking spot from a remote electronic device, obtain and load, in response to the request, a current image of the location, determine portions of the current image each corresponding to one of the bounding boxes, resize each portion of the current image corresponding to one of the bounding boxes, convert each resized portion of the current image to grayscale, convert each grayscaled and resized portion of the current image into an array of pixel values, input each array of pixel values into a logistic regression model, output, using the logistic regression model, a binary classifier for each array of pixels values, determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant, select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot from the GPS coordinates stored in the database in association with the unique identifier of the selected vacant parking spot, transmit, to the remote electronic device using the at least one transceiver, a message including a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, an occupation status, and the GPS coordinates associated with the selected vacant parking spot, periodically check whether the remote electronic device has traveled to the selected vacant parking spot, periodically check the associated occupation status of the selected vacant parking spot, determine that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot, select another vacant parking spot, and transmit, to the remote electronic device using the at least one transceiver, an update including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.

[0111] While this disclosure has described certain embodiments and generally associated methods, alterations and permutations of these embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of example embodiments does not define or constrain this disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this disclosure, as defined by the following claims.

Claims

WHAT IS CLAIMED IS:

1. An electronic device comprising: at least one transceiver; at least one memory; and at least one processor coupled to the at least one memory and the at least one transceiver, wherein the at least one processor is configured to: obtain, using the at least one transceiver, from one or more remote cameras, at least one image of a location including one or more parking spots; detect the one or more parking spots in the at least one image using a machine learning model; assign a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier; and store, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

2. The electronic device of Claim 1, wherein, to obtain the at least one image of the location, the at least one processor is further configured to establish, using the at least one transceiver, a connection with the one or more remote cameras using a camera address.

3. The electronic device of Claim 1, wherein the at least one processor is further configured to: receive, using the at least one transceiver, a request for a parking spot from a remote electronic device; select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot; and transmit, to the remote electronic device using the at least one transceiver, a message including the associated occupation status and the associated GPS coordinates for the selected vacant parking spot.

4. The electronic device of Claim 3, wherein, to detect the one or more parking spots in the at least one image using the machine learning model, the at least one processor is further configured to: create bounding boxes around the one or more parking spots in the at least one image; and assign the unique identifier to each one of the bounding boxes.

5. The electronic device of Claim 4, wherein, to select the vacant parking spot and determine the associated GPS coordinates for the selected vacant parking spot, the at least one processor is further configured to: obtain a current image of the location; determine portions of the current image each corresponding to one of the bounding boxes; convert each of the portions of the current image into an array of pixel values; input each array of pixel values into the machine learning model; output, using the machine learning model, a binary classifier for each array of pixels values; and determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant.

6. The electronic device of Claim 5, wherein, to convert each of the portions of the current image into the array of pixel values, the at least one processor is further configured to: resize each of the portions of the current image; convert each resized portion of the current image to grayscale; and convert each grayscaled and resized portion of the current image into the array of pixel values.

7. The electronic device of Claim 3, wherein the transmitted message includes a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, the associated occupation status, and the associated GPS coordinates for the selected vacant parking spot.

8. The electronic device of Claim 7, wherein the transmitted message causes a navigational application to display directions to the selected vacant parking spot and information associated with the selected vacant parking spot.

9. The electronic device of Claim 8, wherein the at least one processor is further configured to: periodically check whether the remote electronic device has traveled to the selected vacant parking spot; periodically check the associated occupation status of the selected vacant parking spot; determine that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot; select another vacant parking spot; and transmit, to the remote electronic device using the at least one transceiver, an update including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.

10. The electronic device of Claim 3, wherein, to select the vacant parking spot and determine the GPS coordinates for the selected vacant parking spot, the at least one processor is further configured to retrieve the vacant parking spot and the associated GPS coordinates from the database using the unique identifier for the selected vacant parking spot.

11. A method comprising: obtaining, by at least one processor of an electronic device and using a transceiver, at least one image of a location including one or more parking spots from one or more remote cameras; detecting, by the at least one processor, the one or more parking spots in the at least one image using a machine learning model; assigning, by the at least one processor, a unique identifier to each one of the one or more parking spots and associate global positioning system (GPS) coordinates and an occupation status with each unique identifier; and storing, by the at least one processor, for each one of the one or more parking spots, the unique identifier, the associated GPS coordinates, and the associated occupation status in a database.

12. The method of Claim 11, wherein obtaining the at least one image of the location includes establishing, by the at least one processor and using the transceiver, a connection with the one or more remote cameras using a camera address.

13. The method of Claim 11, further comprising: receiving, by the at least one processor and using the transceiver, a request for a parking spot from a remote electronic device; selecting, by the at least one processor, a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot; and transmitting, by the at least one processor, to the remote electronic device using the transceiver, a message including the associated occupation status and the associated GPS coordinates for the selected vacant parking spot.

14. The method of Claim 13, wherein detecting the one or more parking spots in the at least one image using the machine learning model includes: creating, by the at least one processor, bounding boxes around the one or more parking spots in the at least one image; and assigning, by the at least one processor, the unique identifier to each one of the bounding boxes.

15. The method of Claim 14, wherein selecting the vacant parking spot and determining the associated GPS coordinates for the selected vacant parking spot includes: obtaining, by the at least one processor, a current image of the location; determining, by the at least one processor, portions of the current image each corresponding to one of the bounding boxes; converting, by the at least one processor, each of the portions of the current image into an array of pixel values; inputting, by the at least one processor, each array of pixel values into the machine learning model; outputting, by the at least one processor and using the machine learning model, a binary classifier for each array of pixels values; and determining, by the at least one processor, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant.

16. The method of Claim 15, wherein converting each of the portions of the current image into the array of pixel values includes: resizing, by the at least one processor, each of the portions of the current image; converting, by the at least one processor, each resized portion of the current image to grayscale; and converting, by the at least one processor, each grayscaled and resized portion of the current image into the array of pixel values.

17. The method of Claim 13, wherein the transmitted message includes a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, the associated occupation status, and the associated GPS coordinates for the selected vacant parking spot.

18. The method of Claim 17, further comprising: periodically checking, by the at least one processor, whether the remote electronic device has traveled to the selected vacant parking spot; periodically checking, by the at least one processor, the associated occupation status of the selected vacant parking spot; determining, by the at least one processor, that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot; selecting, by the at least one processor, another vacant parking spot; and transmitting, by the at least one processor and using the transceiver, an update to the remote electronic device including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.

19. The method of Claim 13, wherein selecting the vacant parking spot and determining the GPS coordinates for the selected vacant parking spot includes retrieving, by the at least one processor, the vacant parking spot and the associated GPS coordinates from the database using the unique identifier for the selected vacant parking spot.

20. An electronic device comprising: at least one transceiver; at least one memory; and at least one processor coupled to the at least one memory and the at least one transceiver, wherein the at least one memory includes instructions that, when executed by the at least one processor, cause the electronic device to: establish, using the at least one transceiver, a connection with one or more remote cameras using a camera address; obtain, using the at least one transceiver, from the one or more remote cameras, at least one image of a location including one or more parking spots; create bounding boxes around the one or more parking spots in the at least one image; assign a unique identifier to each one of the bounding boxes and associate global positioning system (GPS) coordinates with each unique identifier; store the unique identifier for each one of the bounding boxes and the associated GPS coordinates in a database, wherein the database stores information on a plurality of locations each including a plurality of associated unique identifiers for parking spots and a status of each; receive, using the at least one transceiver, a request for a parking spot from a remote electronic device; obtain and load, in response to the request, a current image of the location; determine portions of the current image each corresponding to one of the bounding boxes; resize each portion of the current image corresponding to one of the bounding boxes; convert each resized portion of the current image to grayscale; convert each grayscaled and resized portion of the current image into an array of pixel values; input each array of pixel values into a logistic regression model; output, using the logistic regression model, a binary classifier for each array of pixels values; determine, based on the binary classifier, that at least one parking spot associated with one of the bounding boxes is vacant; select a vacant parking spot and determine GPS coordinates associated with the selected vacant parking spot from the GPS coordinates stored in the database in association with the unique identifier of the selected vacant parking spot; transmit, to the remote electronic device using the at least one transceiver, a message including a JavaScript Object Notation (JSON) object, wherein the JSON object includes the unique identifier, an occupation status, and the GPS coordinates associated with the selected vacant parking spot; periodically check whether the remote electronic device has traveled to the selected vacant parking spot; periodically check the associated occupation status of the selected vacant parking spot; determine that the associated occupation status of the selected vacant parking spot switches to an occupied status prior to the remote electronic device traveling to the selected vacant parking spot; select another vacant parking spot; and transmit, to the remote electronic device using the at least one transceiver, an update including the associated occupation status and the associated GPS coordinates for the selected another vacant parking spot.