AU2021103555A4 - Machine learning based energy efficient smart city management - Google Patents

Abstract

Design of a smart city is focused in this invention with advanced automated smart devices consuming less energy, comprising of smart homes with intelligent features such as smart energy management, smart waste management and smart surveillance with advanced security and automated street lights able to learn from previous experiences based on Machine learning. As security is the main objective of such smart cities, detection of objects and people creating threat to the inhabitants by utilizing neural networks along with image classifier and detection of object. Design of the system is merged with the concept of Internet of Things (IoT) for controlling various components involved in the smart cities including efficient energy management of smart homes to smart surveillance detecting objects such as gun from the footage of closed circuit television in order to detect unknown dangers on the streets of the smart cities especially by training of a neural network by the outputs generated from image classifier. There are no sensors available to detect objects creating threat to life hence image classifier is utilized for detection of such objects providing security to the residents of smart cities. 11 Convolution & Pooling Convolution &t Hyperspectral oln ehpn ima-ing01 Healthy el 10K Diseased specimen \ \Dense Output DoeLyroeOtpu DenseLayer Input: Data Cube (64x64x240) Figure 1. Architecture of Machine Learning based 3 D Convolution Neural Networks Divide the dataset PromDt r ntecasfiro Dataset created into training and gaerfort atetion trained casasife testing datasets.u Yes [mage captured by Image sent to the Gun detected 1Highi-alert sent to the CCTV camera classifier. authorities. No lmage discarded Figure 2. Proposed Method of Training and Object Detection using Image Classifier 12

Description

Convolution

& Pooling Convolution &t

Hyperspectral oln ehpn

ima-ing01 Healthy

el 10K Diseased specimen \ \Dense Output DoeLyroeOtpu DenseLayer

Input: Data Cube (64x64x240)

Figure 1. Architecture of Machine Learning based 3 D Convolution Neural Networks

Divide the dataset PromDt r ntecasfiro Dataset created into training and gaerfort atetion trained casasife testing datasets.u

Yes

[mage captured by Image sent to the Gun detected 1Highi-alert sent to the CCTV camera classifier. authorities.

No

lmage discarded

Figure 2. Proposed Method of Training and Object Detection using Image Classifier

THEPATENTSACT,1970 (39 of 1970) AND THE PATENTS RULES, 2003

COMPLETE SPECIFICATION

(See Section 10; rule 13)

TITLE OF THE INVENTION MACHINE LEARNING BASED ENERGY EFFICIENT SMART CITY MANAGEMENT APPLICANT

The following specification particularly describes the invention and the manner in which it is to be performed

MACHINE LEARNING BASED ENERGY EFFICIENT SMART CITY MANAGEMENT Field and background of the invention

Smart devices are able to communicate with other machines, in the current

era using the technology of Internet of Things but it does not have the ability of

detecting objects from the images obtained from Closed Circuit Television

(CCTV) fixed in smart cities involving various smart devices connected together to

provide convenient life style to its residents. This invention has the capability of

The proposed system has the ability of detecting objects by utilizing image

classifier along with neural networks. Processing of images obtained from CCTV

is done by implementation of Machine learning which involves trained neural

network. If any objects creating threat for the inhabitants of smart city are

identified by the proposed system then alert is created to its management as well as

residents.

This invention is able to fill the gap between detection of objects and IoT

technology thereby allowing new innovations and applications to be introduced in

the future. Smart city implementation requires three main components which work

together to meet the demands. First component is the smart home which required

automated lighting configured with light dependent resistor (LDR) which can also

be controlled from remote places through mobile phone. Along with this monitoring of temperature and humidity is involved with automatic control by using air conditioner in regulating the temperature.

Summary of Invention

• Machine learning has achieved rapid development resulting in remarkable

results in various fields accompanied by development of neural networks

with its architectures, computer vision and algorithms.

• Internet of Things has gained popularity as it has various advantages but in

this technology there is no way of detecting an object directly as there is no

sensor involved. Vital role is played by object detection hence in this case

where IoT is used, sensor replacement is required.

• In the field of computer vision, object detection is focused as it is one of the

significant applications where great progress is attained by convolution

neural network in detecting object which varies from recognition of single

object to recognition of multiple objects in an environment.

• Object detection is integrated with IoT technology; hence alerts are

generated and sent to remote location whenever necessary. Security features

are equipped in smart home where any intruder entering the home is

detected; photograph is taken and sent to the owner through email as well as

to the management authorities of the smart city.

• Overall communication delay is reduced by proper selection of protocol

such that it is configured specially to attain optimization in the process of

communication in an efficient way.

Brief description of the system

• Smart city have significant features which includes smart surveillance and

security system, smart waste management system, smart energy

management system, smart lightning system, smart traffic management

system which makes life in smart cities very much convenient.

• Municipal corporation is able to track waste in a smart way using the system

of smart waste management, by which waste bins are located from which

toxic gas release is continuously monitored to provide a better environment.

• Smart energy management system involves usage of energy in a smart way

by avoiding wastage of energy as street lights are integrated with

autonomous system which automatically lit up whenever needed and gets

switched off automatically based on motion sensors when motion is detected

in surrounding and when natural light reduces thereby saving energy.

• This autonomous system is also able to control flow of traffic as it

continuously senses the density of traffic at several points of the smart city.

" In the architecture of Convolutional Neural Network, pooling layer does the

function of reducing of spacial size of the representation in a progressive

way thereby reducing parameters involved and the amount of computation.

• Autonomously, operation is done by pooling layer on each of the feature

map where common approach is max pooling method.

• Neural network with pre-training is used to build the image classifier by

using custom classifier layer. Training is given to the classifier with a dataset

consisting of images with output already known by utilizing PyTorch.

• Working of classifier involves the following steps:

• Dataset is categorized into two types namely training dataset and testing

dataset. Better performance is obtained from trained classifier at

generalizing by implementing data augmentation while testing of the model

is done using the testing dataset.

• Training parameters of tweaking network such as learning rate is used to

attain sufficient accuracy. Then attributes of the network are saved.

• Image classifier is used to detect objects present in the environment of smart

cities especially objects creating threat such as gun are identified from the

images received from CCTV camera creating alert to the authorities.

• Image classifier is trained based on the approach of supervised learning

where the training dataset represents both positives of the target object as

well as negatives of the training object in the algorithm of object detection.

• Desired results are obtained from the test dataset once after training of

neural network successfully. If the output determined from the test data is

not found to be satisfactory then repetition of training process occurs by

which weights are adjusted in order to obtain the classifier.

• For instance, in this invention classifier is able to detect the presence of gun

in the environment, once detected alert of high risk is generated and sent to

remotely located owner and management authorities along with its location,

else image obtained is discarded once reaching preset time span.

• Several sensors are equipped in the smart city model, where the user is able

to access real time graph of temperature and humidity recorded in the city

which is updated at regular intervals including smart home location.

• Web functionalities are provided dynamically such as smart home

monitoring in real time, smart management of waste in live based on Python

where the users are allowed to define dedicated functions in order to handle

request of the web page. Training data and testing data for the image

classifier is obtained using PiCamera.

The invention is herein described, with the accompanying block diagrams.

Wherein:

Figure 1. Architecture of Machine Learning based 3 D Convolution Neural

Networks

Figure 2. Proposed Method of Training and Object Detection using Image

Classifier

Description of the system

" In this invention, Software program files cannot be loaded directly into

google Colab. We have to use code snippets for including files through local

machine or via Google Drive.

• File loading is done only temporarily by these code snippets, as at the end,

the files gets deleted at the end of the session. Hence to overcome this

problem, organization of dataset is done first only then it is uploaded to

GitHub once after which cloning of GitHub repository is done in the colab

project file. By this way cloning of dataset is done at every session every

time run command is executed in the system.

• Data organization must be of definite structure as per the requirement of pre

trained models where the training data set must be in root folder and the

testing data set must present in separate folder in the system.

• In turn these folders contain several other folders of different categories with

each of the folder with labels obtained from folders name. Everytime training and testing dataset is created, passing of every directory's pathname is done. Once the pre-trained model is loaded from checkpoint, model architecture gets displayed eventhough layers involved is beyond understanding of the end user of the system.

• Once the project is powered up, Raspbian operation system is booted up.

• Each of the smart components is programmed with Python script which is

executed at the stage of starting up. Sensors calibrated for measurement

becomes active and start sensing for any trigger action.

• Initiation of cloud based services such as Blynk and Flask occurs where the

resolution dependencies along with integration among various sensors

connected in the system as there should not be any conflict as they access

the same pin for GPIO.

• Sensors have ideal delays for achieving proper functioning with appropriate

calibration. Automatic control of devices such as ambient light is to have

efficient energy management in the smart city.

• Flame sensor checks constantly for any fire outbreak, such that detection of

any fire generated sounded alarm with just a push notification using a

mobile application.

* Monitoring of temperature, humidity, waste generated, suspicious objects

within the campus are the key features along with management of resources

along with replacement of substitutes of any failure component.

Claims (6)

CLAIMS We Claim:

1. Autonomous integrated efficient system is proposed for implementing in smart

cities.

2. Convenient environmental residential area with integrated smart devices to

provides all smart services.

3. Efficient energy management and safety are the main features of the design as

automated neural networks provide efficient performance.

4. Energy is utilized efficiently as smart streets are fixed with smart lightning

system and these streets are monitored continuously for regulation of traffic

avoiding heavy density of traffic.

5. Deep learning is merged with Internet of Things for controlling the components

of smart city.

6. No sensors are involved in this system; hence objects are detected by image

classified in integration with neural networks.

Figure 1. Architecture of Machine Learning based 3 D Convolution Neural Networks

Figure 2. Proposed Method of Training and Object Detection using Image Classifier