WO2019132852A1

WO2019132852A1 - Smart camera, information system and method of identifying free parking places in parking facilities having such a camera

Info

Publication number: WO2019132852A1
Application number: PCT/UA2018/000008
Authority: WO
Inventors: Олэг Лэонидовыч ДЫВИНЭЦЬ; Олэг Володымыровыч МОИСЭЕНКО; Виталий Гэоргийовыч ЧЭНЦОВ; Егор Вадымовыч КОЛОДЯЖНЫЙ; Олэксандр Васыльовыч ДЭСЯТНЫЧЭНКО
Original assignee: Олэг Лэонидовыч ДЫВИНЭЦЬ
Priority date: 2017-12-29
Filing date: 2018-02-08
Publication date: 2019-07-04

Abstract

The invention relates as a whole to video surveillance systems, in particular to any open parking facilities such as small street parking lots, large parking lots and parking garages having a ceiling height from 10 metres. The invention also relates to smart cameras which are used in such systems. In particular, the invention relates to a smart camera, an information system and a method for identifying free parking places in parking facilities having such a camera, according to which, owing to a computer unit additionally installed in the smart camera and having a software system using a deep learning neural network, an information system for identifying free parking places in a parking facility has the capability of identifying free parking places in a parking facility using a pretrained neural network, which has the capability of additional learning for a specific geographical location, and transmitting information relating to occupancy/non-occupancy to a central server for storage and subsequent transmission to a user.

Description

CMAPT-KAMEPA, INFORMATION SYSTEM AND METHOD

IDENTIFICATIONS OF FREE PARKING AREAS IN PARKING AREAS

OBJECTS WITH SUCH CAMERA

The invention generally relates to video surveillance systems, in particular to any outdoor parking facilities, such as small street parking, large parking lots and parking garages with a ceiling height of 10 meters. The invention also relates to smart cameras that are used in such systems.

Finding an available parking space in crowded car parks can be very unpleasant for the driver, and this invention is a way to fix these problems.

The smart camera is a nodal object of the system, since it takes on the task of recognizing vehicles and transferring data to a central server.

A known smart camera, which includes a housing in which a camera is installed to receive an image of an object, an image processing functional unit connected to the camera, a rear panel connected to the functional unit, the camera housing having one or more sockets, each of which contains a connector for electrical connection to the rear panel (application US 20030193571, prototype).

The disadvantages of the well-known smart camera is that the camera in the patent is connected directly to a personal computer, it works in a room environment and is engaged in the detection of completely identical objects.

Known smart camera does not provide sufficient accuracy in determining the employment of parking spaces. The image analysis algorithm based on the Fourier transform, which is described in this US patent, does not ensure the accuracy of recognition of similar objects. Difficulty in finding affordable parking space leads to time and fuel costs, and also contributes to the pollution of the air near the parking lot.

A well-known information system and method for detecting free parking spaces in parking facilities, which has a lot of parking spaces, several of which cannot be visually detected by the user, a central server for posting information about the employment status of parking spaces, at least one smart camera that can make a conclusion about the availability of parking spaces relative to employment, has wired or wireless communication with a central server (US patent 6107942).

The said system analyzes the parking space according to the “busy / unoccupied” principle. The system periodically compares the image of the space of a particular place according to the standard, and the standard mentioned is the image of the place mentioned when the specified place is not occupied or occupied, and generates a vacancy signal for the said place.

A disadvantage of the known system and method is that it has a smart camera installed that can detect low-lying cameras in garage-type parking lots and is not designed to recognize a large number of parking spaces. One camera in these systems recognizes up to 3 cars.

The above system and method have the following disadvantages:

1. Expensive, judging by the drawings, one camera handles one parking space, that is, you need a lot of cameras.

2. The system uses a comparison algorithm for a non-empty parking place with a previously prepared image of an empty parking space, which may cause collisions when weather conditions change.

3. The system is most suitable for indoor parking spaces and ideal conditions, such as constant lighting, lack of wind and the presence of wired communication lines between its elements.

4. Cameras are engaged only in image acquisition and do not contain a processing and recognition module; this is done by a separate Image Processing module. It is necessary to provide a permanent connection to it. The well-known camera cannot work offline and does not provide synchronization of ready data when restoring a communication line, which is important for GSM communications.

The invention is based on the task of introducing new nodes and connections with the system into a smart camera to ensure the ability not only to detect objects, but also to search for objects in the video camera's scope and classify them by type with high accuracy. With additional training of the neural network, it is possible to provide a classification of objects by type, for example, freight, passenger, motorcycles.

The problem is solved in that in a smart camera, which includes a housing in which a camera is installed to receive an image of an object, a functional unit for image processing connected to the camera, a rear panel connected to the functional unit, the camera body having one or more sockets, each of which contains a connector for electrical connection with the rear panel, according to the invention, an additional computing unit is installed with a software complex using a deep neural network th training.

The task is also solved by the fact that in the information system of identifying free parking spaces in a parking facility, which has many parking spaces, several of which cannot be visually detected by the user, the central server for posting information regarding the busy status of parking spaces is at least one a smart camera that can make a conclusion about the state of availability of free parking spaces relative to employment, which has a wired or wireless connection to the central server According to the invention, a smart camera is installed in the system, which additionally contains a computing unit with a software package using a deep learning neural network.

The task is also solved by the fact that in the information system to identify free parking spaces in a parking facility, which has many parking spaces, several of which cannot be visually detected by the user, according to which information regarding the employment status of parking spaces is fed to the central server of the system, at least one smart camera via wired or wireless communication with the central server, according to the invention, the detection of free parking spaces in the parking th object is performed using a pre-trained neural network, with additional training under a certain geographic location, and sends information regarding the busy / idle on a central server for storage and onward transmission to users.

The invention is illustrated by drawings, which schematically depicts:

FIG. 1 shows a block diagram of a smart camera,

FIG. 2 is a parking plan containing one camera with direct visibility, FIG. 3 is a side view of FIG. 2

FIG. 4 is a parking plan containing several cameras with indirect visibility,

FIG. 5 is a side view of FIG. four

FIG. 6 is a diagram of a multi-compartment parking with cameras located on lighting poles.

FIG. 7 block diagram of the entire system (cameras, central server, light panel, mobile phones)

The smart camera 1, the block diagram of which is shown in FIG. 1, is a key object of the system. The block diagram conventionally shows lens 2, video camera 3, mini-computer 4, antennas 5, power supply 6, and network cable 7.

The smart camera takes on the task of recognizing vehicles and transferring data to a central server. To ensure this functionality, the smart camera is equipped with a high-performance minicomputer 4 and a communication controller (not shown in the drawing).

The central part of the smart camera 1 is the mini-computer 4. It receives the video from the video camera 3 and analyzes them for the presence of vehicles with a frequency of 3 frames per minute. The system uses a 6th nuclear ARM server-type processor that operates at a frequency of 2 Hz and has a low power consumption of 12V - 4 A.

Computer software uses image processing algorithms and uses a unique neural network for object recognition. The neural network is pre-trained on 10 thousand car images. It is possible to recognize not only cars, but also any other objects (people, animals, etc.) after prior training of the neural network. The software of the smart camera interacts with the central server 8 (Fig. 7), after which the neural network and its parameters are updated.

The system uses an HD camera with a varifocal lens for ease of setting it up for a specific parking space. The lens has a focal length of 5 to 50 mm. The matrix includes a photosensitive element with a minimum illumination of 00.1 lux. This is enough for her work in low light. The camera is connected to the microcomputer via a USB interface that allows you to transfer an image with a HD resolution (1920x1280) using H.264 compression.

Since smart camera 1 transmits small amounts of data, 3G GSM connections to the Internet, via antenna 5, or via available WiFi are used to transfer them. and Ethernet connections. To provide a 3G connection, an external 3G modem and MiniPci Express connector are used. This makes it easy to replace it with a modem that is most suitable for mobile networks in a specific geographic location.

The cooling of the smart camera 1 consists of a passive radiator (not shown in the drawing) located on its case, which is sufficient to cool this type of central processor.

Temperature range from -20 to +40 degrees Celsius.

To ensure the protection of personal data, in many countries it is prohibited to transmit or store video from busy public places. In this situation, you need a camera that analyzes the image and transmits only data on the status of parking spaces without transferring images.

Often, in the area of parking space there is no possibility of a wired internet connection, so the camera used the wireless connection. Unlike conventional video cameras used for video surveillance, the invention has the versatility of installation and works independently of other objects of the system, both online and offline.

FIG. 2 shows a plan of a street parking lot that was developed to implement this invention. The area shown in FIG. 2 is exemplary only in the sense that it is one of many mechanisms in which the basic elements of the invention described in this application may be applicable. This parking is equipped with one smart camera 1, which captures cars in line of sight (located so that the cars do not overlap each other). FIG. 3 position 9 shows a separate parking space, which corresponds to a pre-marked grid. As can be seen from the figure, a smart camera can capture a lot of parking spaces - it depends on how it is installed and pre-configured. FIG. 3 position 10 shows the carriageway. FIG. 2 and 3 position 1 shows a smart camera (IC), and position 1 1 shows the angle of view of a smart camera. The smart camera used in the system can be located at a distance of 4 to 50 meters from the intended parking and can capture from 1 to 60 cars 12. This feature allows you to place the cameras in the most convenient places, for example, on neighboring buildings and will work even with partial visibility of traffic means, this option will be discussed below. FIG. Figures 4 and 5 show a plan view and a side view of a parking area equipped with several cameras located so that some cars overlap each other. Positions 12 and 13 show how one car partially overlaps another car. If it is impossible to establish an IC with direct visibility to vehicles when they do not overlap, it is necessary to complete the system for a specific parking space, after which the system will begin to “see” the vehicles partially blocked. Each SC analyzes its part of the parking lot, information from several cameras is grouped and displayed either in aggregate or by zone.

FIG. 6 shows a plan view of a multi-sectional parking lot with IC 1 located on the lighting pillars. Cameras cover the entire parking space and transmit information on a predetermined grid. Positions 14 and 15 show entry and exit respectively. FIG. 6 parking spaces 16, 17, 18 and 19 are controlled by smart camera 1. The camera analyzes each image for empty and occupied places and does not use the previous images as described in US patent 6,107,942 A, which allows to get rid of collisions and loss of accuracy. There is no need to have a previously saved image of empty spaces, which changes when the time of day changes, or when precipitation falls. There is no comparison of two images to assess the employment of parking spaces. In the event of a collision, the system self-restores at the next iteration of the algorithm.

The method of detection using a deep neural network is that the network is pre-studied by tens of thousands of images, and the general properties of objects, for example, wheels, headlights, etc., are highlighted. Based on this information, an assumption is made whether the object is in the zone of interest (parking space). This avoids such collisions as temporary overlapping of the zone of interest (for example, a heavy vehicle), precipitation, camera vibration, day and night change. Thus, the system analyzes the features of the object and compares them with the already known features of the prototype, which increases the accuracy of the system and makes it universal for various geolocations.

To improve the accuracy of the system, there is also the possibility of retraining it in semi-automatic mode by the server operator, this will be discussed below. There are no additional controllers and devices in the system for communication between its elements. The system does not imply the installation of expensive sensors and special communication lines. The system requires only access to the Internet.

FIG. 7 shows a block diagram of the entire system.

Positions 1 show smart cameras located on specific parking lots and tied to specific GPS coordinates and physical parking addresses. Cameras are connected to the Internet in one of 3 ways: wired (Ethrenet) or wireless (WiFi, GSM). In the case of a wired connection, you need to lay a twisted pair cable to the nearest Internet access point 20. This type of connection will allow you to use the IC as a video surveillance system. The ability to transmit the video signal in its pure form is configured using the central server software 8. Also, the smart camera has the ability to save the video signal to external media when motion is detected in the area of interest. Any actions in the parking lot, including collisions between two cars, can be monitored 24 hours a day, which can help system owners in finding the cause of accidents, or in the event of car theft. To work the UK you need to apply power. All its settings, maintenance are performed by the central server operator, that is, there is no need for physical access to an already installed smart camera.

Unlike existing solutions, this system has the ease of installing components and does not require special knowledge to administer it.

The smart camera works offline and with the presence of the Internet in online modes. In offline mode, the camera stores statistics in its own database, followed by synchronization of data with the server in online mode. Thus, no important information is stored on the camera media. The advantage is that the main storage medium is located in a secure cloud storage and not in a smart camera. Important information is stored offline. In case of loss of information in the SC, it can be easily recovered from the central server. All data can be synchronized. The smart camera contains only current data that is transmitted and stored on a central server. In offline mode, it accumulates data and then transmits it at the time the link is restored.

FIG. 7, reference numeral 20 shows the global Internet, which is the main transport hub for communication between system devices. All data between the devices of the system are protected with a password and can be encrypted, which prevents unauthorized access to the system.

Position 7 shows the central server containing statistical information coming from SC 1 and transferring it to clients: light boards, mobile devices, WEB statistics website 21. The central server can be located both on a physical computer and on a virtual one in a cloud service. There is a need to ensure fast access and reliable Internet connection. In another, the server does not require high computing power, since it receives and sends the data generated by the smart camera already prepared. One of the main functions of the server is to manage the settings of the smart camera, which are synchronized with it on request.

After the physical installation of the smart camera, it registers on the central server and sends images to it. The server operator sets the world coordinates (GPS) for the SC, the physical address and makes grid markings based on the received image. After these presets, the camera is ready for use. Sending images is disabled. The normal operation of the system involves the exchange of statistical information about the employment of parking spaces, eliminates the high load on the system communication lines.

The central server 7 has an interface for transmitting data to clients and uses the REST (https://wikipedia.org/wiki/REST) protocol for interaction.

This type of data transfer allows third-party developers who have access to connect to the system. Web server interface reflects the digital map of the world based on Google Mar, displays on it parking spaces and statistical information about employment. The web interface and the central server can be located physically on different computers. The manager can generate reports on the time during which monitored spaces were occupied and when the state of objects changes. Such data can be compared with cash receipts for the same period.

The time of status change is saved and can be used to create reports, and can also be used to calculate the fee that needed to be collected. Information can be displayed and / or printed and used to generate data or reports on the time during which the monitored spaces were occupied, the number of times in a certain period when the state of the monitored objects changed from unoccupied to occupied and back to vacant. Such data can be compared with actual cash receipts over the same period and can be used, for example, to check the accuracy of such receipts to determine if a leak has been made.

The statistics also allows drivers to see the free and busy places directly in the mobile application and pave the way to the nearest free parking online.

There is a problem with the accuracy of determining the occupancy of parking spaces based on optical sensors. The use of a neural network allows to achieve almost 100 percent of the definition of objects, this distinguishes this solution from existing patents.

Smart cameras come with an already trained neural network. It is also possible to learn using the server software. To do this, the smart camera is transferred to the mode of sending images to the server at a certain interval (1 - 3 min), after the formation of the image database using them, the existing neural network is re-trained with augmented images. After synchronization of the camera with the server, the camera receives the file of the new neural network.

For the operation of the smart camera, OS Linux is used which is highly reliable, for the operation of the server OS Linux or Windows by the owner.

Position 22 shows information panels associated with specific GPS coordinates and physical address. Each information panel is configured with a unique ID key for a specific parking space, or for several spaces and reflects the number of occupied and free parking spaces. It is possible to supply information panels with the ability to display advertising, which can reduce the final cost of the system.

Position 23 reflects Android and IOS mobile devices, in which special software is installed that interacts with the central server. In the mobile application, the user can search for the nearest free parking and pave the way for the vehicle to move to it. Also obtain information on the number of free parking places and the number of the parking zone, if it is multi-zone.

Claims

CLAIM

1. A smart camera that includes a housing in which a camera is installed to acquire an image of an object, a functional unit for image processing connected to the camera, a rear panel connected to a functional unit, the camera body having one or more sockets, each of which contains a connector for electrical connection with the rear panel, which differs in that it additionally has a computing unit installed with a software package using a deep learning neural network.

2. The information system for detecting free parking spaces in a parking facility has a lot of parking spaces, several of which cannot be visually detected by the user, a central server for posting information about the employment status of parking spaces, at least one smart camera that can do the conclusion about the state of the availability of parking spaces relative to employment, has a wired or wireless connection with the central server, which is different in that it has a smart camera, in which A computational unit with a software complex using a deep learning neural network has been carefully installed.

3. The method of detecting free parking spaces in parking facilities has a lot of parking spaces, several of which cannot be visually detected by the user, according to which information on the employment status of parking spaces is provided to the central server of the system using at least one smart camera via wired or wireless communication with the central server, which is distinguished by the fact that the identification of free parking spaces in the parking facility is carried out using a pre-trained neural network and with the possibility of additional training under a certain geographic location, and send information about employment / unemployment a central server for storage and onward transmission to users.