EP4294986A1

EP4294986A1 - Cyberphysical system for cleaning roads and/or outdoor facilities, and method therefor

Info

Publication number: EP4294986A1
Application number: EP22722745.1A
Authority: EP
Inventors: Martin Kramer
Original assignee: Siemens AG
Current assignee: Siemens AG
Priority date: 2021-04-13
Filing date: 2022-04-13
Publication date: 2023-12-27
Also published as: EP4074896A1; US20240044094A1; CN117280096A; WO2022219048A1

Abstract

The invention relates to a cyberphysical system -CPS- for the automated detection and removal of interfering objects in parks, outdoor facilities, roadside ditches, etc. The invention furthermore relates to a method for the automated cleaning of parks, roadsides, outdoor facilities and green spaces, in particular in city centres. In this case, modules of a first, second and third type communicate with one another via the IoT such that modules of the first type adopt the actual state, provide the data generated therefrom to the modules of the second type via the IoT and the modules of the second type then calculate a target state and activate modules of the third type, possibly via the IoT, which collect and remove the interfering object.

Description

description

Cyber-physical system for cleaning traffic routes and/or outdoor facilities and methods for this

The invention relates to a cyber-physical system -CPS- for the automated detection and removal of disturbing objects in parks, outdoor areas, ditches, etc. The invention also relates to a method for automated cleaning of parks, roadsides, outdoor areas and green areas, especially in inner cities.

So far, automated street cleaning has only taken place on paved roads, where cleaning vehicles, which are preferably controlled manually, can move easily.

Cyber-physical systems describe a network of sensory and/or mechanical modules with a data infrastructure that communicate with each other via wired or wireless communication networks, in the form of machine communication and/or with the network - Internet of Things "IoT". For example, IT traffic control and traffic logistics systems are implemented via cyber-physical systems.Such systems usually include several subsystems and are correspondingly complex.

With increasing urbanization and "to-go" nutrition, green areas and public parks are also becoming increasingly soiled with food leftovers, packaging waste and other rubbish. Dogs and other pets may also leave their mark.

There is therefore a need for automated detection and cleaning of roadsides, green areas and public parks.

The object of the invention is therefore to provide a cyber-physical system that automates or semi-automatically removes rubbish from roadsides and/or green spaces.

This object is achieved by the subject matter of the present invention, as defined in the description, the figure and the claims.

Accordingly, the subject of the present invention is a cyber-physical system for the automated cleaning of the edge areas of traffic routes and/or green areas, modules of the first and third type, at least one communication network and associated interfaces, with at least

- a module of the first type that automatically and/or user-driven generates data that reflects an actual state with a disruptive object,

- if necessary, a module of the second type receives this data via at least one suitable interface, and the at least one module of the second type comprises at least one processor in a management system, which processes the data and, if necessary, uses artificial intelligence to train neural networks with the data and

- the management system of the second module or direct communication of a first module with a suitable interface of a third-type module, this causes at least one manipulator which is connected to the third-type module and is suitable for picking up and/or transporting away the disturbing object , to start, where

- Each module includes at least one data store and one processor and can start an action independently, i.e. decentrally.

In addition, the subject of the invention is a

Method for the automated cleaning of green spaces, by means of a cyber-physical system as described above, in which

- an actual state of a part of the green area with a disturbing object is recorded, - whereby the collection of the data on a disruptive object is automated and/or user-driven, at least the reporting of the data generated by the user is carried out by means of transmission and provision of the data, if appropriate to a management system in a module of the second type,

- a module of the third type with a manipulator is activated by the management system directly or by means of at least one processor configured and set up for this purpose and/or by suitable communication without the intermediary of the management system and at least one module of the third type moves to the disturbing object, swims to it and/or or approaches, picks up and/or removes .

General knowledge of the invention is that a communicating combination of known modules, in particular

- Modules of the first type, which are devices for acquiring, mapping, locating and forwarding the relevant data,

- Modules of the second type, which are devices for processing, controlling, regulating, machine learning and/or comparing data, as well as

- Modules of a third type, which are devices with manipulators for picking up, sucking up and/or transporting objects, connects them, for example in such a way that

- a navigation system is connected and in particular with the help of

- artificial intelligence, a constantly improving and outdated cyber-physical system can be implemented, either via smartphone users moving in parks or outdoor facilities, from the cyber-physical system's own operating data and/or via stationary or mobile surveillance systems, the outdoor facilities and/or Scan parks, detect disturbing objects there, with the collected data from these sources that Cyber-physical system can be trained and automatically remove interfering objects. For this purpose, the cyber-physical system includes modules of the third type with manipulators, which, after a message has been received by one or more modules of the first type, about at least one disruptive object - possibly by a management system as part of a module of the second type - the modules of the third type possibly with feedback are selected with IoT and/or AI, these are set in motion and guided to the location of the disturbing object.

These modules of the third type can include several individual devices which—possibly in cooperation—automatically remotely approach, swim to and/or approach, pick up and/or transport the at least one interfering object. The notification of the disruptive object is either done manually by a user using a smartphone app or automatically via appropriately trained detection systems that are suitable and/or trained to detect disruptive objects from non-disruptive objects in the relevant outdoor area or in outdoor areas in general, such as lawns.

"Machine communication" is an automated exchange of information between devices, such as machines, drones, machines, vehicles, containers with each other and/or with a - preferably central - management system, for example via Bluetooth, Wifi, optically, using ultrasound, sound waves and/or Radio waves, referred to, for example, by means of low-power wireless communications such as LoRa, LoRaWAN, IrDA, ZigBee, XBee, and/or infrasound.

Machine communication can take place from machine to machine, but also via the Internet of Things, IoT, via mobile networks or other networks. The technology of machine communication combines information and communication technology. "ZigBee" and/or "XBee" are communication tools specifically for wireless networks and low data volumes, they also communicate, for example, in a mesh network and/or a so-called ad-hoc network.

"LoRaWAN" has a network architecture that is typically arranged in a star topology, in which the gateways establish the connection between the end devices and the central network server. The gateways are connected to the corresponding network server via a standard IP connection, while the end devices use a single-hop connection to one or more gateways. The communication is usually bidirectional. It also supports the operation of multicast address groups to ensure efficient use of the spectrum for e.g. over-the-air updates to ensure "OTA" updates.

LoRaWAN is a Low Power Wide Area Network (LPWAN) specification for wireless battery-powered systems in a regional, national or even global network. In particular, it also aims at secure bidirectional communication, localization and mobility of services, end-to-end encryption and the mobility of services. The LoRa WAN specification offers seamless cooperation between different sub-systems and technologies under Smart Things without the need for rigid, local complex installations. "LoRa" describes the physical layer that enables the "long-range" communication link.

In the present case, the term "edge area", "green area", "outdoor area", "park" refers to an area that can be soiled by all kinds of disturbing objects that have been left lying around and thrown off. It is irrelevant whether this area is otherwise maintained or cleaned by gardeners, foresters or other managers, but it is - at least as long as the cyber-physical system of the park is not has worked long enough for it to be able to make its own decisions - it depends on the user of the green area whether he or she finds an object left there "disturbing" or not , food leftovers, empty, full or partially filled glass packaging, other rubbish, plastic, cloth and/or paper bags, in short everything that the user would like to see in a rubbish bin, but here on the sandy path, the lawn, the forest and/or or lying around in hedges and bushes.

In the present case, "target condition" refers to the condition of the green area and the peripheral area in which the disturbing object is gone, in particular removed, and the user-unpolluted, desired and/or natural condition of the peripheral area and/or the park and /or the outdoor area and/or the green area is established.

Using the appropriate communication infrastructure - possibly including the IoT and artificial intelligence, the cyber-physical system is set up in such a way that it can be used automatically to clean green spaces.

A module of the second type comprises a processor and, for example, at least one further processor which communicates with the first and is suitable for carrying out training for the automated recognition of disruptive objects using data generated by the user. Furthermore, the processor of a module of the second type is advantageously communicatively connected to a data archive, with the data of the modules of the first type, the estimated values and assumptions about the physical and/or chemical nature of the disturbing object being able to be stored in the data archive on the one hand, and on the other hand after appropriate storage Feedback or sending back the data collected from the interfering object by concrete measurement and/or characterization of a module of third type. The module of the second type is also the one that trains the neural networks of the modules of the first and third type and makes them available--for example in the form of updates. An additional processor of such a module can be of the TPU and/or GPU type, for example.

As a result, the data on the current status, which is provided via the message, i.e. the provision of the data by one or more modules of the first type, can be compared with the data that is provided by the module or modules of the third type via the "truth". Data from the actual measurement and/or characterization results of the same interfering object are provided.By comparing this data, the cyber-physical system can automatically learn a great deal about the detection of the interfering objects.Therefore, the module of the second type is an advantageous embodiment a learning system, for example in the form of an artificial intelligence - "AI" - provided.

All the data in the system, not just that from the data archive, can be used to train the cyber-physical system so that the cyber-physical system is constantly improving. The machine learning processor or processors of the modules of the second type make their latest ML models available to the modules of the first and/or third type, for example in the form of updates, preferably automatically.

Advantageously, at least at the beginning, the data collection is carried out partly by people, so-called "user-driven data generated for the detection of disturbing objects", people who, as "users" of the park, specifically photograph rubbish there and thus provide the artificial intelligence with visual or optical or - depending on the detected recording device provide other data of rubbish that was not automatically classified as "interfering object" by the machine, but which, through data classified by the user, serve as the basis for suitable training of the neural networks of a second-type module belonging to the AI. After a training phase, ronal networks can be returned to the modules of the first and third type and then cause such disturbing objects to be classified automatically by the modules of the first and third type and automatically identified as "disrupting the object" by the cyber-physical system become recognizable.

The data collection by users at the beginning of the commissioning is also partly advantageous in order to refine the pre-trained generic neural networks more specifically for the area of application.

By putting the cyber-physical system into operation and generating data - machine and/or user-driven - a neural network of an AI, which is connected to the module of the second type, is trained. The trained network is then, for example, "deployed" in a drone, which can include both a module of the first type for detection and a module of the third type for elimination, i.e. transmitted and stored equipped self-driving robots, capable of automatically classifying debris as such within the area for which it is trained, based on the artificially collected and/or the humanly collected data and/or the training done The drone, for example, then develops the network through any amount of training data and/or receives updates from the module of the second type, which usually includes a more powerful, in particular generic, processor than the modules of the first and third types, which are housed in a drone, for example, that trains neural networks and then the trained networks can be fed back to a third-type module, such as the drone.

If a suitably equipped drone with a suitably trained network, or another recording device with a camera, automatically recognizes and classifies rubbish with its first-type module, it can do so without any detours via a central management system, which is implemented in particular in a second-type module, picking up and removing the rubbish via a third-type module, also implemented in the drone, such as a suitable manipulator, for example a gripper arm. For this reason, the cyber-physical system disclosed here can also be referred to as a "decentrally reacting" system, because after an appropriate familiarization phase of the individual modules, they are enabled to carry out cleaning work automatically and self-controlling without a central management system.

As mentioned, the user-driven generated data and/or the automatically generated data can be used to train the neural network of the artificial intelligence AI integrated in the cyber-physical system.

For example, after the refuse has been identified, a module of the third type can optically record it further before it is removed and generate further visual data on it, in particular, for example, with different lighting and/or from a different perspective. The data generated in this way can then be used again for training the system. In this way, the system can be further trained and improved and/or updated with each additional work step.

From the beginning, the cyber-physical system can also be used without people, but the person - or the user - is instrumentalized by mobile phones. So that vast amounts of unnecessary or unusable data do not flood the cyber-physical system, there is a collection point on the smartphone, for example, which can be pre-sorted according to "plausibility" via an app.

In doing so, new data is compared with data that is usable and correct and data that is unusable and wrong. Both correct and wrong data can be humanly or machine-identified. For example, a “false positive detection” can be provided by comparisons in the system. A "learning system" is an artificial intelligence that can be trained using data, such as the results of comparisons.

During operation, all images are uploaded to the data storage system and one or more neural networks are trained from this data storage system, which are then available to the cyber-physical system as AI and to the individual modules that have platforms, on which the neural networks can run can be distributed. Alternatively or in addition, the data storage system feeds one or more neural networks that have the same function but run on different platforms. The individual modules are part of devices that are connected, for example, via radio and/or cable support.

A process platform could be, for example, on a mobile device and/or part of a robot, a drone and/or a sensor in a drone, part of a sensor and/or another function included in a camera, for example to be trained.

Such a function can be a segmentation in an image, for example. The system recognizes something and points it out. "Excising" simplifies the classification of what has been excised.

The data storage system is preferably used as a database for training a neural network.

Machine and/or user-driven, i.e. human object recognition serves in particular as a preliminary stage to segmentation within the cyber-physical system.

According to an advantageous embodiment, the AI and/or the data storage system is used to to create so-called "heat maps" of the area covered by the physical system. The heat maps represent the area on a map in which areas are marked according to the frequency with which disturbing objects and/or rubbish can be found. In this way, the AI can identify locations where a lot of rubbish has already been found, and the cyber-physical system learns to search there more intensively.This can result in an enormous increase in the efficiency of the cyber-physical system, because areas in which rubbish is never found are only rarely found The problem is that the cyber-physical system is constantly evolving and can therefore advantageously provide updates to the individual modules from areas where, for example, 95% of the garbage has been found so far became.

In particular, the following modules of the first type, which serve to record and report and/or detect a disruptive object, are used:

Surveillance cameras, stationary and/or mobile, and/or users with smartphones that use a correspondingly programmed app to detect disturbing objects - if necessary and preferably segment and classify them and report them automatically - e.g. via the camera function Rubbish in parks. The surveillance system and/or the smartphone uses an app - which is connected, for example, to an artificial intelligence in such a way that it receives updates - to generate sensor and/or camera data that are automatically made available to a module of the second type. The modules of the first type are particularly advantageously equipped in such a way that they report the type of disturbing object in a classified manner, ie, for example, biowaste, glass waste, plastic waste, lost property, etc.. The modules of the first type include, for example, sensors and/or processors with stored visual information Comparative data that compare whether the surface of the detected disturbing object is firm, soft, shiny, matt, porous, liquid, dispersive, i.e. solid components in a liquid, muddy, etc. Other modules of the first type with detectors could record and compare smells, for example.

For example, a message classified in this way about the interfering object is forwarded by a first-type module via the corresponding communication interface, preferably with location data, to a second-type module, which, according to an advantageous embodiment, first checks this data for plausibility.

The plausibility can include several factors, for example the plausibility can first serve to determine whether the reported disturbing object can be there at all.

A message that has been checked for plausibility is then forwarded to a regulation and control system, which has at least one corresponding technical drive unit in the form of a third-type module, i.e. a device that can be remote-controlled and moves to specified coordinates automatically, and this continues via a manipulator as a means for picking up, such as a sucker, a gripping arm, etc., through which the device can pick up and/or transport an object that is in the way.

A module of the first type can, for example, be a mobile surveillance camera as part of a drone that is equipped with a camera. A third type of module can be a drone with one or more gripper arms, for example. According to an advantageous embodiment, for example, both modules, the first and the third type, are implemented in the same drone. The drone has, for example, a camera on the one hand, a gripper arm on the other and also a data infrastructure and interfaces via which the data from the camera is made available to the gripper arm via a suitable interface.

It is particularly advantageous that the data infrastructure, means of communication and interfaces, just like the camera and gripper arm, can be implemented in a drone, so that everything that is required to provide such an embodiment of the cyber-physical system according to the invention is physically available in which is realized a drone.

A "drone" is an unmanned vehicle, in particular an aircraft, which can be operated and navigated independently by a computer via remote control without a crew on board. A drone can also be an unmanned truck and/or a boat or an amphibious vehicle, for example A "drone" may also include a manipulator, such as a gripper arm and/or suction device. A drone within the meaning of the present invention can, for example, have implemented everything in one device, from the message to the controller and finally also the transaction using manipulators.

The module of the second type comprises in particular a management system that has or is connected to a geographic information system (GIS). The module of the second type can be implemented, for example, in the form of a processor that is part of a data processing unit of a remotely controllable drone.

According to an advantageous embodiment of the invention, the module of the second type comprises one or more processor(s) which are set up in such a way that they carry out a computer-aided check of the message by the module of the first type for plausibility.

A “management system” is basically any type of control that receives and processes from a module of the first type and controls and/or regulates a module of the third type. The management system as at least part of the module The second type can - physically seen - be part of a module of the first and/or third type. In particular, in - for example - a drone, a remote-controlled vehicle, any type of modules, interfaces and communication systems according to the present invention can be implemented. Communication with the IoT, just like with a smartphone, can be implemented or not.

"IoT" is the name of the Internet of Things, i.e. a collective term for technologies of a global infrastructure of the information society, which makes it possible to network physical and virtual objects with each other and to let them work together through information and communication technologies.

For example, a geographic information system is anchored in the IoT that enables the collection, processing, organization and presentation of spatial data, and in particular the location of a reported disturbing object.

In connection with the invention, a "module" can be understood, for example, as a drone, a remote-controlled vehicle, a robot, an automat, a camera, a sensor, a processor and/or a memory unit for storing program code. For example, the processor is special designed to execute the program code in such a way that the processor executes functions in order to implement or realize the method according to the invention or a step of the method according to the invention.The respective modules can also be designed as separate or independent modules.The corresponding modules can be used for this purpose for example include other elements.

These elements are, for example, one or more interfaces (e.g. memory interfaces, database interfaces, communication interfaces—e.g. network interface, WLAN interface) and/or an evaluation unit (e.g. a processor) and/or a memory unit , as a Data storage system that can be used as a basis for training neural networks and/or an artificial intelligence AI. For example, data can be exchanged (eg received, transmitted, sent or provided) by means of the interfaces. By means of the evaluation unit, data can be generated, compared, checked, processed, assigned or calculated in a computer-aided and/or automated manner, for example. By means of the storage unit, data can be stored, retrieved or made available in a computer-aided and/or automated manner, for example.

A module of the first type is any type of sensor and/or recording device that detects and reports a disruptive object, e.g. records reality, the "actual state" and generates data that can be processed using a computer. For example, a module of the first type can a camera function generate visual data, or record and analyze smells via a sensor function, which are then sent and/or transmitted via a module of the second type to a module of the third type.

A kind of "module of the first kind" is a smartphone of a visitor to the parking facility, who preferably has an app installed that is connected to the cyber-physical system and with which he can feed the system with "user-driven" data. This visitor then photographs the garbage or rubbish discovered by him/her, preferably via the app, the app carries out a plausibility check and sends this user-driven visual, optical and/or other data to the cyber-physical system, respectively a data storage system of it. Alternatively, for data collection without an app from a park visitor, another connection to a data storage system of the cyber-physical system could be established.

According to an advantageous embodiment of the module of the first type, a classification and/or a Segmentation of the disturbing object made, for example, the type of organic waste, leftover food, packaging remains, paper, glass, etc., which is also transmitted to the module of the second type, together with the message.

According to an advantageous embodiment, the message is accompanied by a so-called "intelligent blurring", so that the module of the first type initially recognizes, records and reports an interfering object, but at the same time is not prevented, other objects that are, for example, in the immediate vicinity, recognize, record and report disturbing objects.

Also known as "fuzzy search", intelligent fuzziness includes, for example, a search method that not only finds the exact character string, but also similar character strings. This technique can also be used to find disturbing objects in outdoor areas.

For example, a cyber-physical system configured and set up with intelligent fuzziness recognizes food leftovers, packaging and/or scattered, i.e. diffuse, contamination from confetti etc. and does not fly to the waste bin after every confetti it has collected and unload it , but collects the confetti in an area and removes a number of confetti at once.

In particular, an embodiment is also possible that can establish communication with the third-type module without a dedicated message as soon as the disturbing object is recognized, which then activates the third-type module without further testing and/or without further order and carries out the removal. With such a structure, the "module of the second type" in the cyber-physical system is, for example, one or more suitable interface(s) and/or the communication between the module of the first type and the module of the third type. The module of the second type receives the data from the module of the first type via a suitable interface and either forwards it directly to a module of the third type or processes it in the management system.

In particular, if the modules of the first, second and third type are not only implemented in a single device, such as a single drone, these are in particular a module or part of a module of the second type that is connected to the IoT and/or to an AI . The second type of module calculates - if necessary via an interface and communication to the IoT or AI - a plausibility, for example, can there be a dog mess in the middle of the river? - before further processing of the data takes place.

A connection to the IoT is also preferably provided in the second type of module because of the data archive. This in particular because even with a deployment of modules of the first, second and third type in a drone, this is, for example, a mini-deployment of a module of the second type, in the sense of a temporary representative (proxy) in offline scenarios.

For example, the drone would collect training data offline, i.e. without a connection to the IoT, but the training itself is cloud-based and therefore requires a connection to the IoT.

In the second type of module, for example, the plausibility-checked data from the first type of module or modules is compared with data from a database there. For example, trash, junk, leftover food, and/or dog poo captured by a camera can be identified and located using the IoT or other data infrastructure that includes a navigation system.

The module of the second kind, which is connected to the IoT, performs the following processes, for example:

- location of refuse detected by a module of the first type, - Checking the plausibility of the identified rubbish with the location,

- Selection - if necessary by means of an AI and/or other machine learning - from the known data on rubbish, garbage, faeces, etc., a type of module or modules of the third type, i.e. a type of removal e.g. drone, suction, sweeping and/or wiping robots and, where appropriate, combinations thereof,

- activation of the third-type module for removal and/or removal, in particular via an integrated or connected control and regulation system,

- control and/or guidance of the third-type module to the place where the disturbing object was located,

- regulation and control of the removal of the waste by one or more module(s) of the third type and

- If necessary, receiving and processing feedback from the third-type module on the physical and/or chemical characterization of the disturbing object and/or the success of the evacuation as training data for an artificial intelligence that is connected to the second-type module and/or with connected to this.

The module of the second type triggers a command and/or a controller that activates at least one module of the third type, for example a robot and/or a drone, controls it to the point and controls it there via appropriate drives and tools on the module of the third type Object, e.g. removes the dog poo and/or the rubbish, for example sucks up or picks it up. The controller then leads this module of the third type, together with the charge, to a station for disposal, for example a waste bin or the like.

According to another optionally also combined embodiment of the invention, the station for disposal is mobile, for example a moving, optionally also remote-controlled, in particular also automated, remote-controlled loader that picks up the rubbish and drives it to or from the trash can waste processing, such as a recycling station or similar, depending on the type and nature of the disruptive object.

A module of the third type always includes a manipulator with an engine and, for example, a gripper arm, a vacuum cleaner or a mop. For example, a module of the third type is a drone or a vacuum robot that specifically picks up the "reported" garbage and takes it to the station for disposal, for example to a truck, warehouse or dumpster.

According to an advantageous embodiment of the invention, the module of the third type transmits the result of removal via an interface back to the module of the second type, so that the module of the second type has feedback as to whether the solution found, ie the removal of the garbage with the activated actor was successful, less successful or not successful at all. Likewise, it is optionally provided that the module of the third type, that "knows the truth", measures the disturbing de object, weighs it, analyzes it physically and/or chemically and can thus provide data to the module of the second type, with which it can use the data from the detection of the disturbing object by visual or other recording by a module of the first type.These data can be stored in the data archive and made available there to an artificial intelligence.

According to an advantageous embodiment, the module of the second type includes such a processor, which includes an AI in order to automatically learn, for example, from the results of the respective actions of the modules of the third type, which manipulator is suitable for which rubbish at which location. These ML models, which result from automatic learning, are then made available to all other modules in the cyber-physical system.

In the following, the invention is further illustrated by a figure that schematically shows a system architecture of an embodiment. nes example of a possible cyber-physical system according to the invention, explained in more detail:

The disturbing object, comprising rubbish or found objects 100, can be seen in the middle a drone with a camera 104 and possibly a computer-aided machine learning model ML model system 105 is found and reported. The stationary surveillance system monitors - represented by arrow 106 - the park 101 and finds - see arrow 107 - the disturbing object 100. The CCTV system integrated in the CCTV system reports, segments and classifies the CCTV system shown here thanks to its ML Model 105, if necessary with the help of appropriate sensors (not shown), the disturbing object according to the type of nature, or the type of found item, but at least according to dimensions, surface quality, size, weight, etc. Depending on the maturity of the machine learning ML Model 105, which is integrated here, the classification, segmentation will do better or worse.

For example, the system shown here is in use due to the feedback 105, 127, 130 with artificial intelligence 118, which constantly trains the classification and/or segmentation that has taken place using data 115, 116, 117 recorded and classified by a camera 104 in the CCTV 103 continuously improved. This meta data 116 on the interfering object, as well as the data 117 with which the interfering object was recorded, for example the visual data from the camera, are sent to a management system M2.

On the other hand, a user 108, for example, strolls through the park with his smartphone 109 and photographs--see arrow 1--using a corresponding app 109, the disturbing object 100 with the smartphone 109. The app generates metadata 2 from the photo 111, such as classification, size, suspected weight and the order 110 and is likely to be the disruptive object 100 to segment, locate and describe. The app 110 may have corresponding sensors and/or comparison data for this purpose. On the other hand, the user can also use the app to enter his classification and his metadata 2 for the photo 111 or the disturbing object 100, so that these are also forwarded - e.g. via the IoT - to a management system M2 of the connected module of the second type M2, 115118 .

The one or more modules of the first type 103, 110 provide the visual data of the detection of the disturbing object 100, as well as any existing metadata 2 to one or more module (s) of the second type ready, for example, the data are the management system M2, which has a Geographic information system (GIS), as well as planning and control capabilities, provided. There, the visual, acoustic and other data belonging to the order are processed, in particular they are checked for plausibility and relevance - e.g. a dog pile can be in the middle of the pond or is the disturbing object in an area that is not suitable for cleaning is provided?

The use of the cyber-physical system can be detected simply by sending data, the recordings of disturbing objects including location information, to an appropriate communication interface, with the interface tracing the receipt of this data back to the sender and thus the authorization of the sender and/or or the sending device.

In the - preferably intelligent - management system, i.e. control and/or regulation system - M2, depending on the type of disturbing object 100 and depending on the location where this is located, the modules of the third type, such as drones, robots, for example suction and/or sweeping robot, truck, floating load receiver and/or transporter, activated and instructed to pick up and/or transport the disturbing object. Criteria for the selection of the module of the third type by the management system are, for example

- Responsibility for the location

- Responsibility for the type of disturbing object

- Accessibility for the selected device (land/water/air)

- Range - stationed close enough to the site

- Enough loading capacity for the disturbing object

- the module of the third type can charge/collect the disturbing object, e.g. depending on the gripping strategy, the mechanical resilience of the module of the third type and/or the carrying capacity,

- Possibility of the module of the third type being able to drive, swim and/or fly to the site via remote control, possibly automated remote control.

For example, the module of the third type can be a flying, driving and/or swimming drone, which is optionally coupled to one or more other drones, with which it coordinates itself automatically and optionally unloads the load, i.e. the interfering object 100 and/or the load Splits.

The module of the second type, ie z. B. the management system M2 is automatically activated by the provision and receipt of the data from the module or modules (s) of the first type.

The data is stored within the module of the second type in a data archive 115 on the one hand, for example in connection with the "order 1...n" 116 and/or the "actual image" 117, and on the other hand they are used to Intelligence "KI" 118 e.g. to enable the machine learning system for training the "ML" models 119 of a possibly integrated and/or connected AI and/or model archive 118.

A module of the second type, as in the embodiment shown in the figure, the management system M2 with, for example, a smart control system, comprises at least one processor that is configured so that it can access the data from the received data, as in - For example, the data from the data archive 115 and/or the results from the AI 118 calculates a relevance of the order after the plausibility check. The relevance is present, for example, in the form of a prioritization of the order in which the orders are to be processed.

The AI 118 feeds a model archive 135 and/or "AI" and/or artificial intelligence with continuously improved ML models 119. The training is triggered by retrieving the data from the data archive 115. In the data archive 115, the data from the modules of the first type 117, which represent the "actual state" on the one hand, and the data 116, which are collected through the characterization by the modules of the third type, on the other hand.

"AI" refers to the automation of intelligent behavior and machine learning "machine learning". Artificial intelligence attempts to emulate certain human decision-making structures using statistical methods and/or neural networks that are somehow modeled on the structures of the brain. For this purpose, e.g. according to an "iterative process", a decision is automatically made by the system again and again, implemented and the result evaluated. If the decision and subsequently the result was expedient, further decisions are made along the path, if not, this is the path Path classified as "wooden path" and avoided in future decisions.

Typical types of training for an AI are: a) Supervised learning by comparing the feedback from the learning module with the data that is actually available and b) By collecting data in a data archive, where the training effect is that over time more and more data is available with which the module can compare the data it receives.

In connection with the inventions, a “processor” can be, for example, a machine or an electronic circuit can be understood. A processor can in particular be a main processor (Central Processing Unit, CPU), a microprocessor or a microcontroller, a TPU (Tensor Processing Unit) and/or NPU (Neural Processing Unit), which is technically a hybrid ASIC and FPGA and/or DSP are similar. There are also COTS boards, for example Google Coral or Intel ^Movidius ...they are also known as VPUs...KI coprocessors. For example, it can also be an application-specific integrated ^circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can, for example, also be an IC (integrated circuit), in particular an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). Integrated circuit), or a DSP (Digital Signal Processor) or a graphics processor GPU (Graphic Processing Unit). A processor can also be understood to mean a virtualized processor, a virtual machine or a soft CPU.

It can also be a programmable processor, for example, which is equipped with configuration steps for executing said method according to the invention or is configured with configuration steps in such a way that the programmable processor implements the inventive features of the method or the modules, or other aspects and/or Partial aspects of the invention realized.

In terms of performance classes, the processors that can be used in the system are at least application processors, for example Cortex A. In the module of the second type, the management system within the ^cyber -physical system, processors of higher performance classes are advantageously used, for example those whose number of available cores is between 1 and 8. The number of cores in a processor indicates how many things the processor can lei can do and/or the clock frequency. CPU/TPU/GPU, for example, are used as generic processors.

In contrast to microcontrollers, e.g. K60, microprocessors cannot be used sensibly here because neural networks cannot be run sensibly on them.

In connection with the invention, "computer-aided" or "computer-aided" can be understood, for example, as an implementation of the method in which, in particular, a processor executes at least one method step of the method. For example, "computer-based" or "computer-based" also means "computer-implemented".

"Providing", in particular with regard to data, meta data and/or other information, can be understood in connection with the invention as computer-aided providing. The providing takes place, for example, via an interface (eg a database interface, a network interface, an interface to a storage unit). Via this interface, for example, when providing, corresponding data and/or information can be transmitted and/or sent and/or retrieved and/or received. In connection with the invention, for example, “providing” can also include a loading or a saving, for example, a transaction can be understood on with corresponding data. “Providing” can also be understood, for example, as a transmission (or a sending or a transmission) of corresponding data from a node to another node.

In the management system M2, a recovery strategy for the disturbing object 100 is calculated via the AI, if necessary, and at least one module of the third type 122 and/or 123 - a removal tool, e.g. drone, truck, suction, sweeping and/or wiping robot as well as possibly combination NEN of it needs-oriented - so for example depending on the classification of the disturbing object 100 - commissioned.

The modules of the third type 122, 123 are networked with the management system M2 and with one another and can communicate with one another in an automated manner directly and/or via the IoT and/or via the management system M2. In the embodiment shown here, the module of the third type 122 is a remote-controllable drone 122, which has a position determination 124, e.g. GPS, a camera 125, e.g. a 360° camera 125, an AI with a machine learning "ML" model 127 and Finally, a manipulator, such as a gripper arm 126 has.

In the example shown in the figure, the module of the third type 123 is a remote-controllable truck 123, which also has a position determination 128, a camera 129, an AI with a machine learning "ML" model 131 and a manipulator, such as a gripper arm 130 Like the manipulators 130 and 126, the module 122, 123 can be controlled automatically.

The commissioned modules of the third type 122 and 123 coordinate each other, find each other at the site and recover and transport the disruptive object 100. In this process, the disruptive object 100 is localized, photographed, and recovered. For example, the modules 122 and/or 123 also have means for characterization, for example for weighing and/or measuring the object 100 that is causing the problem. This data is used to train the machine learning system 118 and/or the data archive 115 of a module of the second type provided. The provided data 116 from the real measurement by the modules of the third type can then be provided to the machine learning system 118 and/or the model archive 135 of a module of the second type. The information and data 117, 116 collected in this way, physical and/or chemical characterization of the interfering object 100, are also given to the module of the second type, for example provided in the data archive 115 as data 116 which the machine learning system 118 accesses. This, for example, also with regard to a comparison with the data 117 of the actual state from the presumed and/or estimated data of the modules of the first type 103, 110.

If a truck 123 is on site that has a larger loading capacity than the interfering object 100 needs, other interfering objects (not shown) or other items are loaded onto it without the truck 123 driving back to the station beforehand. Communication between the modules is designed to always find the simplest solution. The modules of the third type 122, which, for example, still fit on the truck 123 in terms of weight/size, can also be brought back to a base station by the latter.

The method for cleaning with a cyber-physical system according to an exemplary embodiment of the invention is characterized in that the modules of the third type 122, 123 provide data for characterizing the disturbing object 100, which they the data archive 115, 116 and / or the provide machine learning system 118 of a module of the second type for training.

By capturing the approximately estimated order data regarding the physical dimension of the disturbing object, the plausibility check 121 in the smart management system M2, and the storage in the data archive 115, 117 as well as the comparison with reality through the recorded data 116 from weighing and measuring when collected by the Modules of the third type 122 and/or 123, the cyber-physical system collects training data around its own machine learning ML models 119, 105,

127 and 131 through training and to increase their accuracy in operation.

The ML models 119 updated in the module of the second type 118 are communicated "into the field" and, for example, by "0- ver-the-Air" OTA update into the third type 122 and/or 123 modules where they are in the form of the ML models 127 and/or 131.

The ML models 119 are not digital copies of each other, but are always the result of the respective machine learning process that takes place in the machine learning system 118 and/or is stored in the model archive 118, where the machine learning system looks at the data, for example served from the data archive 115 and the result is provided in the form of the respective ML model 119 tailored to the models of the first or third type. These ML models 119 are advantageously also stored again in the model archive 135 . In this way, transfer learning is also possible.

As a result, the cyber-physical system - especially one in a park where the system has already been trained - learns over time automatically all types of disturbing objects 100 and can automatically detect impurities in monitoring mode via modules 103, 122 and 123 detect, report and rectify.

But users who report to the cyber-physical system with their smartphone or app can also access the extended capabilities, e.g. in the form of ML models via OTA updates.

It is quite possible that a single drone 122, as well as a camera 125 located in or on it, can be used both as a module of the first type for detecting and reporting a disruptive object and as a module of the third type for collecting and transporting through the cyber physical system is used.

Due to the autonomous improvement of the cyber-physical system in use, the present invention provides for the first time a possibility for automated cleaning of the green areas, outdoor facilities and/or parks. The invention relates to a cyber-physical system -CPS- for the automated detection and removal of disturbing objects in parks, outdoor areas, ditches, etc. The invention also relates to a method for automated cleaning of parks, roadsides, outdoor areas and green areas, especially in inner cities. Modules of the first, second and third type are communicatively connected via the IoT in such a way that modules of the first type record the current status, and the data generated from this is sent to the modules of the second type via the

Provide IoT and the modules of the second type then calculate a target state and activate modules of the third type, if necessary via the IoT, in order to create the target state. The modules of the third type, which have manipulators, e.g. a gripper arm, a sucker, generally a recording device, etc., collect the disturbing object, can characterize it if necessary and transfer the characterization data to the ML models of the Provide modules of the second type for training before these modules of the third type remove the disturbing object. In particular, due to the availability of the actual characterization of the disturbing object and the possibility of comparison with the estimated data of the modules of the first type, the cyber-physical system can constantly improve and specialize on the green area in focus.

Claims

patent claims

1. Cyber-physical system for the automated cleaning of the edge areas of traffic routes and/or green spaces, modules of the first and third type, comprising at least one communication network and associated interfaces, with at least

- a module of the first type automatically and/or user-driven generates data that reflects an actual state with a disruptive object,

2. Cyber-physical system according to claim 1, wherein a module of the first type (103, 110) is a module of the second type

(M2, 115, 118) and a module of the third type (123, 122) are realized in a single device.

3. Cyber-physical system according to claim 2, wherein the device is a drone (122).

4. Cyber-physical system according to one of the preceding claims, in which the module of the second type (M2, 115) has a data archive that is communicatively connected to the machine learning system (118).

5. Cyber-physical system according to one of claims 1 or 2, wherein a module of the second type comprises a processor in a management system (M2) which is suitable and configured to carry out a plausibility check on the data received from a module of the first type (103, 110) are provided to perform.

6. Cyber-physical system according to one of the preceding claims, wherein the module of the first type (103, 122, 123) comprises a 360° camera (104, 129, 125).

7. Cyber-physical system according to one of the preceding claims, in which a module of the first type (103, 122, 123, 110) is a drone (122).

8. Cyber-physical system according to one of the preceding claims, in which the modules of the first type, modules of the second type and modules of the third type are implemented in a single drone (122).

9. Cyber-physical system according to one of the preceding claims, wherein a module of the second type comprises a management system (M2).

10. Cyber-physical system according to one of the preceding claims, wherein a module of the second type comprises a data archive (115).

11. Cyber-physical system according to one of the preceding claims, wherein a module of the second type (M2) comprises a geographic information system (120).

12. Cyber-physical system according to one of the preceding claims, wherein a module of the second type comprises a learning system and/or model archive (135).

13. A method for the automated cleaning of green areas by means of a cyber-physical system according to any one of claims 1 to 12, in which

- an actual state of a spot in the green area with a disturbing object (100) is recorded,

- whereby the data on a disruptive object is recorded automatically and/or user-driven, at least the reporting of the data generated by the user by means of transmission and provision of the data may correspond to a management system (M2) in a module of the second type (M2, 115, 118,135 ) he follows,

- a module of the third type (122, 123) with a manipulator (126, 130) is activated by the management system (M2) directly or by means of at least one processor configured and set up for this purpose and/or by suitable communication without the intermediary of the management system, and at least a module of the third type (122, 123) that hits, swims and/or flies towards, picks up and/or removes the disturbing object.

14. The method as claimed in claim 13, in which a module of the second type (M2, 115, 118) comprises a regulation and control system (M2).

15. The method according to any one of claims 13 or 14, in which a plausibility check (121) of the data provided by the first modules (103, 110, 122, 123) in the management system (M2) before the activation of a module of the third type ( 122, 123) is performed.

16. The method according to any one of claims 13 to 15, wherein the modules of the third type (103, 122, 123) provide data for the characterization of the disturbing object (100), which they provide to the data archive (115, 116) and/or the

Provide learning system (119) of a module of the second type for training.

17. The method according to claim 16, wherein the learning system

(119) and/or model archive (135) of a second module (M2, 115, 118,135) using over-the-air "OTA" update ML models (119, 105, 131,

127) to the modules of the first and third type (103, 110, 122, 123).