DE102020213560A1 - Method and system for monitoring a transport container for goods in an aircraft and aircraft - Google Patents

Abstract

A method for monitoring a transport container for goods in an aircraft comprises capturing video image data of the transport container using a video camera installed in the aircraft, determining an intervention in the transport container based on the captured video image data using a data processing device installed in the aircraft and outputting an intervention report by the Data processing device when an intervention in the transport container is detected.

Description

The present invention relates to a method and a system for monitoring a transport container for goods in an aircraft and an aircraft, in particular a passenger aircraft.

In aviation, sales services provided on board are becoming increasingly important for airlines. For example, it is common to offer high-value goods for sale during the flight. Typically, these goods are stored in a mobile transport container or trolley provided for this purpose. During the sale, the cabin crew moves the trolley inside the cabin to the passengers to offer the goods and, in the case of a purchase, to hand them over to the passengers.

In flight phases in which no sale takes place, the trolley with the goods is usually stowed in a storage location, e.g. in a storage compartment provided for this purpose on a shelf arrangement. During such flight phases, the trolley is usually unattended and generally freely accessible to passengers and cabin crew. Therefore, it cannot be ruled out that unauthorized persons may access the goods. Other mobile transport containers, such as pieces of luggage, or stationary transport containers, such as storage compartments above passenger seats, can also be unattended at times, so that there is a risk of unauthorized intervention.

Aircraft generally have a video surveillance system for monitoring the passenger cabin, cameras being distributed in the cabin in order to record video image data. The video image data are generally shown on a display which is arranged, for example, in the cockpit or in an area where the flight attendants stay. Such video surveillance systems are, for example, in EP 1 440 883 A1 or the DE 10 2007 051 196 A1 disclosed.

It is the object of the present invention to find solutions with which interventions in a transport container in a cabin of an aircraft can be traced in a simple manner.

This object is solved in each case by the subject matter of the independent claims.

According to a first aspect of the invention, a method for monitoring a transport container for goods in an aircraft is provided. The method includes capturing video image data of the transport container and, if applicable, the environment of the transport container using at least one video camera installed in the aircraft, determining an intervention in the transport container based on the captured video image data using a data processing device, which can optionally be installed in the aircraft, and a Outputting an intervention report by the data processing device if an intervention in the transport container is determined.

According to a second aspect of the invention, a system for an aircraft for monitoring a transport container for goods is provided. The system according to the invention has at least one video camera for capturing video image data of the transport container and, if applicable, the area surrounding the transport container and a data processing device with a signal connection to the video camera, which is set up to process the video image data captured by the video camera in a method according to the first aspect of the invention .

According to a third aspect of the invention, an aircraft, in particular a passenger aircraft, is provided with a system according to the second aspect of the invention.

One of the ideas on which the invention is based is to monitor a transport container, such as a trolley, in an aircraft using a video camera permanently installed in the aircraft and to carry out an automated evaluation of a video recorded by the video camera, with the evaluation detecting if the transport container was reached into, in particular whether the receiving area of the transport container s, in which the goods are stored, was reached into.

According to the invention, the analysis or evaluation of the image data provided by the video camera is automated by means of a data processing device, which can optionally be accommodated in the aircraft. It is also conceivable that only one storage medium is installed in the aircraft, on which the video image data is stored, and that the evaluation takes place on an external data processing device. It would also be possible to send the video image data from the aircraft to an external data processing device, for example using wireless data communication. The data processing device can have, for example, a processor and a data memory that can be read by the processor, software being stored in the data memory that can be executed by the processor and causes it to carry out an evaluation algorithm in order to determine an intervention in the transport container. an intervention or reaching into the transport container is represented by a typical data pattern in the video image data, for example by a data pattern representing a typical gesture or movement. The presence of such a data pattern is determined using the data processing device. It should be noted that the video camera does not necessarily have to capture images from the interior of the transport container. Rather, it is sufficient if video image data is recorded from at least one area in front of the interior of the transport container in which the goods or objects in general are stored, so that it can be concluded from the data patterns contained in this video image data that someone is reaching into the interior of the trolley .

If a data pattern representing an intervention in the trolley is determined during the evaluation of the video image data, an intervention report or notification is generated and output by the data processing device. The notification of an intervention can, for example, be a data record which contains at least the information that an intervention has taken place. The intervention notification can be stored in a data memory, e.g. the data memory of the data processing device or another data memory of the aircraft, optionally together with the captured video image data.

An advantage of the invention is that an intervention in the transport container can be detected in a simple manner. In particular, the intervention can be determined automatically, with sensors not necessarily having to be provided on the transport container itself. Since the transport container is monitored via one or more video cameras, it is also easier to identify people who reach into the transport container, particularly if the video camera has an optical sensor. Furthermore, the output or generation of an intervention notification in combination with the video surveillance offers the advantage that the intervention notification simplifies a visual check of the video material.

Advantageous refinements and developments result from the dependent claims referring back to the independent claims in connection with the description with reference to the figures.

According to some embodiments it can be provided that the determination of the intervention includes an analysis of the video image data by the data processing device for the presence of an intervention feature in an area in front of the transport container through which the goods can be removed from the transport container.

The area in front of the transport container, also referred to as the detection zone, can be, for example, a space defined by the projection of a circumference of the transport container, e.g. a cuboid space which is located in front of a boundary of the transport container through which goods can be removed from the transport container. The transport container can, for example, have compartments that can be closed by doors and/or pull-out drawers. The detection zone is located in front of these compartments and/or drawers.

The intervention feature can very generally be a data pattern in the video image data, which represents an intervention in the transport container. For example, this can be a typical movement and/or a typical object, in particular a body part of a person. Optionally, the engagement feature includes at least one of the following optical features represented by the video image data: a shape of a human hand and/or a human forearm, an optical signature of a human hand and/or a human forearm, a predetermined movement pattern.

The presence of the intervention feature can be determined, for example, using a method based on artificial intelligence, AI. For example, the data processing device can have a monitoring processor, which is supplied with the video image data recorded by the camera. The monitoring processor is connected bidirectionally to a KI processor, which checks the video image data for deviations from a reference set of rules stored in a reference set of rules memory and generated by a set of rules generator based on self-learning algorithms. The monitoring processor can output the intervention notification if the deviations determined by the AI processor are above one or more predeterminable deviation threshold values, with the deviation being above the deviation threshold value in the case of an intervention feature.

According to some specific embodiments, it can be provided that the intervention report includes a data record with one or more of the following contents: a time, a date, a position within the aircraft, a position of the aircraft, an identification number of the aircraft. The time, the date, the position of the aircraft and the identification number of the aircraft can be retrieved from a central computer of the aircraft by the data processing device, for example. The position within the aircraft can be based, for example will be determined from image information, which is represented by the video image data.

According to some embodiments, the method can further include generating an output of video images based on the captured video image data on a display and displaying a marker in the output of the video images if an intrusion into the transport container is determined. The display can be provided, for example, on a flight attendant panel, a mobile device such as a tablet, or in the cockpit of the aircraft. The video image data recorded by the camera can be played back on the display in real time, for example. It is also conceivable that the video image data is stored in a data memory, e.g. the data memory of the data processing device, and then played back. The marking can be, for example, a visual element displayed in the output, e.g., a symbol, a color coding of the output, a frame or the like. This makes it easier for the viewer of the display to recognize an intervention.

According to some embodiments, the output of the video images can be generated based on a recording of video image data captured in a predetermined recording period, e.g. from video image data stored in the data memory of the data processing device, with a graphical progress indicator, e.g. a bar, being displayed in the output of the video images, which represents a beginning and an end of the recording period and a current point in time within the recording period, and wherein the progress indicator is marked at locations which correspond to a point in time at which an intrusion into the transport container was determined. The visual highlighting of points in time in the progress display at which interventions were determined makes it easier for a user to evaluate the recording.

According to some embodiments, the method can include transmitting the intervention report to a mobile terminal device in the aircraft and the mobile terminal device issuing a warning signal. For example, the intervention report can be transmitted to a smartphone or another mobile terminal device, which is assigned to a purser or another crew member, for example. The terminal generates a warning signal, e.g. an acoustic and/or haptic and/or visual warning signal, to inform the crew member that an intervention in the transport container, in particular in the form of a trolley, has been detected. This makes it easier for the wearer of the end device to check whether the intervention was carried out by an authorized person.

According to some embodiments, the method can include determining an authorization for the determined intervention based on the captured video image data by means of the data processing device, with the intervention message only being output if the determined intervention is authorized, or with the intervention message being output with the information that the intervention is authorized. For example, in certain phases of the flight, different people or groups of people can be authorized to intervene in or to remove goods from the transport container. For example, a certain part of the cabin crew can be authorized during a flight phase in which a sale of goods from the transport container takes place. Outside of this flight phase, for example, only a small part of the cabin crew, e.g. only the purser, may be authorized to intervene. According to some embodiments, if the intervention is authorized, the intervention report can be dispensed with, which has the advantage that the number of intervention reports generated is reduced. This is not advantageous for an evaluation of the intervention report by human users, but also because storage space and computing power can be saved.

According to some embodiments, determining the authorization can include determining an authorization feature in the video image data. The authorization feature can very generally be a data pattern in the video image data, which represents that the person removing the goods belongs to an authorized group of people. Optionally, the authorization feature can be formed by one or more of the following optical features: clothing of the person reaching into the transport container, an optical marking worn by the person reaching into the transport container, biometric features that identify the person reaching into the transport container.

The presence of the authorization feature can be determined using a method based on artificial intelligence, KI, for example. For example, the data processing device can have a monitoring processor, which is supplied with the video image data recorded by the camera. The monitoring processor is connected bidirectionally to a KI processor, which checks the video image data for deviations from a reference set of rules stored in a reference set of rules memory and generated by a set of rules generator based on self-learning algorithms. The monitoring processor can output the intervention notification if the deviations determined by the AI processor are above one or more specified values ren deviation thresholds are, in the case of an authorization feature, the deviation is above the deviation threshold.

According to some embodiments, it can be provided that the transport container is identified by means of the data processing device on the basis of the captured video image data. Optionally, the transport container can be identified among a plurality of other containers. The transport container can optionally be identified using an optical marking of the transport container, e.g. in the form of QR codes, barcodes or the like, using optical and/or geometric properties of the transport container as such, such as dimensions, arrangement of handles or the like, and/ or based on a position of the transport container relative to the other containers. Optionally, in these embodiments, an AI-based method can also be used to identify the transport container, as has already been described above.

According to some embodiments, provision can be made for video image data to be captured by a large number of video cameras arranged distributed in the aircraft, and for a position of the transport container inside the aircraft to be determined by means of the data processing device using the captured video image data and, optionally, using a position of the video cameras. The large number of cameras transmits the recorded image data to the data processing device, which determines or detects the object “transport container” in the image data, e.g. as described above based on external features of the transport container. If the transport container has been recognized as such, the data processing device further determines its location within the aircraft, e.g. by comparing other image features in the vicinity of the transport container, for example a trolley, with a stored reference image of the interior of the aircraft. Alternatively or additionally, each camera can be assigned an identifier and a position within the aircraft, for example, with the identifier and position of the camera being determined that captures the image data in which the transport container is recognized. As a result, the position of the transport container can be determined from the position and optionally from a known orientation or a known field of view of the camera. An advantage of determining the position of the transport container is that the amount of data that is examined for the presence of an intrusion is reduced. Furthermore, the determination of an intrusion is also facilitated for cases in which the transport container is moved, e.g. before, after or during a sales transaction.

According to some specific embodiments of the system, the data processing device can have a first interface for connecting to a central computer on the aircraft and/or a second interface for connecting to a communications network on the aircraft. The first interface can be, for example, a wired interface set up for connection to a data bus, e.g., an ARINC 429 interface, an ARINC 825 interface or a similar interface. For example, the second interface can be a wireless interface, e.g., a WiFi interface, a 4G interface or the like. Of course, the second interface can also support different data transmission protocols, e.g. both 4G and WiFi and/or other protocols.

According to some embodiments of the aircraft, it can have a cabin monitoring system. The cabin monitoring system comprises a large number of video cameras installed distributed within a cabin of the aircraft for capturing video image data and a display signal-connected to the video cameras for generating an output of video images based on the captured video image data. For example, the cabin monitoring system can be a so-called CVMS, with "CVMS" standing for the English expression "Cabin Video Monitoring System". According to some embodiments, the system for monitoring a transport container can be part of the cabin monitoring system of the aircraft, with at least one of the video cameras of the cabin monitoring system being aligned in the direction of a parking area for the transport container. The storage area can be arranged, for example, in the area of a galley of the cabin. The integration of the monitoring of the transport container in a cabin monitoring system advantageously facilitates retrofitting in existing aircraft. Furthermore, a high level of functional integration is achieved, which is advantageous for the weight and manufacture of the aircraft.

• 1 eine schematische Blockdarstellung eines Luftfahrzeugs gemäß einem Ausführungsbeispiel der Erfindung;
• 2 eine schematische Draufsicht auf eine Bordküche eines Luftfahrzeugs gemäß einem Ausführungsbeispiel der Erfindung;
• 3 eine schematische, perspektivische Ansicht eines Transportbehälters in Form eines Trolleys, welcher durch ein System und/oder ein Verfahren nach einem Ausführungsbeispiel der Erfindung überwacht wird;
• 4 eine schematische Darstellung einer Ausgabe von Videobildern in einem System und/oder Verfahren gemäß einem Ausführungsbeispiel der Erfindung; und
• 5 ein Flussdiagramm eines Verfahrens gemäß einem Ausführungsbeispiel der Erfindung zum Überwachen eines Transportbehälters für Waren in einem Luftfahrzeug gemäß einem Ausführungsbeispiel der Erfindung.

The invention is explained below with reference to the figures of the drawings. From the figures show:

• 1 a schematic block diagram of an aircraft according to an embodiment of the invention;
• 2 a schematic plan view of a galley of an aircraft according to an embodiment of the invention;
• 3 a schematic, perspective view of a transport container in the form of a trolley, which by a system and / or Method according to an embodiment of the invention is monitored;
• 4 a schematic representation of an output of video images in a system and/or method according to an embodiment of the invention; and
• 5 a flowchart of a method according to an embodiment of the invention for monitoring a transport container for goods in an aircraft according to an embodiment of the invention.

In the figures, the same reference symbols designate identical or functionally identical components, unless otherwise stated.

In the following description, reference is made to self-learning algorithms which are used in an artificial intelligence system (AI system). Generally speaking, a self-learning algorithm emulates cognitive functions that are humanly associated with human thinking. By adding new training information, the self-learning algorithm can dynamically adapt the knowledge gained from old training information to the changed circumstances in order to recognize and extrapolate patterns and regularities in the entirety of the training information.

In self-learning algorithms within the meaning of the present invention, all types of training that form the human knowledge gain can be used, such as supervised learning, partially supervised learning, independent learning on the basis of generative, non-generative or deep contradictory networks ("adversarial networks", AN), reinforcement learning, or active learning. Feature-based learning (“representation learning”) can be used in each instance. The self-learning algorithms within the meaning of the present invention can in particular carry out an iterative adjustment of parameters and features to be learned via feedback analysis.

A self-learning algorithm within the meaning of the present invention can be based on a support vector classifier (“support vector network”, SVN), a neural network such as a convolutional neural network (“convolutional neural network”, CNN), a Kohonen network, a recurrent neural network , a time-delayed neural network (TDNN) or an oscillatory neural network (ONN) , a random forest classifier, a decision tree classifier, a Monte Carlo network or build a Bayesian classifier. A self-learning algorithm in the sense of the present invention can use hereditary property algorithms, k-means algorithms such as Lloyd's or MacQueen's algorithms or TD learning algorithms such as SARSA or Q-Learning.

The invention is described below primarily with reference to a transport container in the form of a trolley, ie a container that can be moved on rollers. Within the scope of the present disclosure, however, a transport container can be any type of container that is suitable for accommodating objects, in particular for accommodating objects that can be moved by hand by a single person. Such objects can in particular weigh less than 10 kg and/or have maximum dimensions of approximately 60 cm by 50 cm by 30 cm. For example, these can be everyday items such as cosmetics, tobacco, spirits, medicines or clothing, consumer electronics items, jewelry, and so on. Accordingly, the transport container is not limited to a trolley, but can, for example, also be a suitcase, a bag, a storage compartment in the aircraft, such as a so-called hat rack, which is arranged above the passenger seats, or a transport box.

1 shows a block diagram of an aircraft 100 purely by way of example. As in FIG 1 shown by way of example, the aircraft 100 has a system 1 for monitoring a transport container 110 for goods and optionally a cabin monitoring system 200 .

The aircraft 100 can be a passenger aircraft, for example, which has a passenger cabin. A plurality of seats (not shown) may be arranged in the passenger cabin. A galley 160, also referred to below as a galley, can also be accommodated in the passenger cabin, in which various transport containers 110, 116 for transporting goods within the cabin are parked.

A galley 160 is in 2 purely as an example and shown only schematically. The galley 160 can have, for example, a shelf arrangement 161 which is arranged on the floor 102 of the passenger cabin and which defines one or more adjacent shelf compartments 162 . As in 2 shown as an example, mobile transport containers 110, 116, also referred to as trolleys, can be accommodated in the shelves 162. The totes 110, 116 may have a body 110A, 116A defining an interior space and wheels connected to the body 110A, 116A 110B, 116B, so that the transport container 110, 116 can be moved on the floor 102. Goods can be accommodated in the interior space defined by the body 110A, 116A, such as food and beverages or high-quality merchandise such as watches, tobacco products, cosmetic products or the like. The goods can be removed from the respective transport container 110, 116 through an access opening which can be closed, for example, by a door 110C, 116C or from which a drawer 110D, 116D can be pulled out, as is shown in 2 is shown schematically.

During a flight, the transport containers 110, 116 can be removed from the shelf arrangement 161 by the cabin crew and driven to the passenger seats, e.g. in order to offer the passengers goods for sale or to offer drinks and food. Outside of these sales phases, the transport containers 110, 116 are generally parked on the shelf arrangement 161 or on another storage area provided for this purpose in the cabin.

As in 1 shown schematically, the system 1 can have one or more video cameras 2 and a data processing device 3 . As in 1 also shown, the system 1 can be part of a cabin monitoring system 200 of the aircraft 100 . The cabin monitoring system 200 has a large number of video cameras 2 and at least one display 150 . The video cameras 2 of the system 1 for monitoring the transport container 110 can thus be formed by cameras of the cabin monitoring system 200 . Optionally, the data processing device 2 can take over functions of the cabin monitoring system 200 as well as functions of the system 1 .

The video cameras 2 are set up to capture video image data. For example, the video cameras 2 can have an optical sensor, e.g. a CMOS sensor, and an optics arrangement which focuses light reflected from objects in the cabin of the aircraft 100 onto the sensor, with the sensor being set up to output corresponding video image data. The video cameras 2 can thus be implemented in the usual way. As an alternative or in addition to an optical sensor, the video cameras 2 can have thermal sensors or proximity sensors, such as ultrasonic sensors, so that abstract video image data and/or video image data in non-visible frequency ranges can also be generated.

As in particular in 2 shown, the video cameras 2 can be permanently installed, for example, on a ceiling 104 of the cabin or at another location inside the cabin, for example in partition walls or the like. The video cameras 2 can be positioned and aligned over the entire cabin, for example in such a way that video image data of an area in which passenger seats are arranged and/or video image data of an area in which the galley 160 is arranged are recorded. In particular, at least one of the video cameras 2 is aligned in the direction of a storage area for the transport container 110, as is shown in FIGS 1 and 2 is shown schematically.

The data processing device 3 can in particular have a processor 30 and a data memory 31 that can be read out by the processor 30 . Processor 30 may include one or more CPUs or similar computing units. For example, the data processing device 3 can have a monitoring processor and an AI processor, which are bidirectionally connected. The data memory 31 can in particular have a non-volatile storage medium, such as an HDD memory or an SSD memory.

The data connection device 3 can, as in 1 shown schematically, be permanently installed in the aircraft 100, for example in the floor 102, in the ceiling 104 or on another accommodation structure in the passenger cabin. Of course it is also conceivable that the data processing device 3 is accommodated outside the passenger cabin, for example in a cargo hold.

As in 1 shown schematically, the data processing device 3 can have an input interface 34 via which the cameras 2 are signal-connected to the data processing device 3 . The input interface 34 can be, for example, a wired interface, eg a USB interface, an Ethernet interface, an ARINC 429 or 825 interface or the like. Optionally, the data processing device 3 can also have a first interface 32, which can be implemented as a wired interface, for example as a USB interface, an Ethernet interface, an ARINC 429 or 825 interface or the like. The first interface 32 can, as in 1 shown as an example, be signal-connected to a central computer 120 of the aircraft 100 in order to retrieve various data, such as date, time, flight number, aircraft number, current position of the aircraft from the central computer 120. As an alternative or in addition to the first interface 32, a second interface 33 can be provided, as shown in 1 is shown as an example. The second interface 33 can optionally be wireless Interface can be implemented, for example as a WiFi interface or a 4G interface, or the second interface 33 can include one or more communication interfaces, each of which supports different protocols. As in 1 shown by way of example, the second interface 33 can be connected to a communication network 130 of the aircraft 100, for example in order to be able to send data to and/or receive data from devices that are connected to the communication network 130. In 1 is shown by way of example that a mobile terminal 140, for example a smartphone, is connected to the communication network 130.

The display 150 is configured to generate an output R2 of video images based on video image data. For example, the display 150 can be an OLED display, an LED display, an LCD display, or the like. The display 150 can be a display of a mobile terminal device, such as a tablet PC, or a permanently installed display, such as a so-called flight attendant panel or the cockpit. As in 1 Shown purely by way of example, the display 150 can be signal-connected to the first interface 32 of the data processing device 3 , the data processing device 3 being set up to provide the video image data generated by the video camera or cameras 2 at the first interface 32 . Thus, the display 150 is signal-connected to the video cameras 2 through the data processing device 3 . Alternatively, the display 150 can also be signal-connected directly to the video cameras 2 . In general, the display 150 is signal-connected to the video cameras 2 . In 4 an output R2 is shown as an example, which can be generated by the display 150 based on captured video image data.

5 shows an example of the sequence of a method M for monitoring a transport container 110 for goods in an aircraft 100. The method M is explained below purely by way of example with reference to the above system 1. In particular, the data processing device 3 can be set up to process the video image data captured by the video cameras 2 in accordance with the method M described below. For example, software can be stored in the data memory 31, which causes the processor 30 to process the video image data according to the method M and to generate corresponding outputs.

In a step M1 of the method, video image data of the transport container 110 are recorded by means of at least one video camera 2 . For example, the transport container 110 can be parked on a parking area, e.g. as in 2 shown in the shelf structure 161 of the galley 160. However, the transport container 110 can also be parked at another location or moved through the cabin, for example by the cabin crew. The video image data 110 captured by the video camera 2 is transmitted to the data processing device 3, for example to its input interface 34. In 1 shows by way of example that the data processing device 3 is permanently installed in the aircraft 100 . However, it is also conceivable for only one storage medium, in particular a non-volatile storage medium, to be installed in the aircraft 100, which is connected to the video cameras 2 in order to store the captured video image data. This storage medium can then be temporarily removed from the aircraft 100 and/or connected to an external data processing device. It would also be possible to send the video image data from the aircraft 100 to an external, for example ground-based data processing device, for example via a communication interface (not shown).

In an optional step M2, the transport container 110 can be identified M2 as a container to be monitored. For example, it can be provided that only the transport container 110 is to be monitored, e.g. because it contains valuable items such as goods for sale, while monitoring other transport containers 116 is not necessary, e.g. because they contain consumables such as drinking cups, serviettes or the like. An identification M2 of the transport container 110 can thus optionally take place among a plurality of further containers 116 . In step M2, this is done using the data processing device 3 based on the captured video image data.

For example, the processor 30 can execute an image processing algorithm that analyzes the video image data for the presence of certain features that identify the transport container 110 . For example, the identification M of the transport container 110 can be based on an optical marking 111, which, as in 2 visibly attached to the transport container 110 as shown by way of example. The optical marking 111 can, for example, be a bar code or QR code, a colored marking or the like. Alternatively or additionally, optical and/or geometric properties of the transport container 110, such as its outline, its dimensions or an arrangement of its doors 110C and drawers 110D, can be used as identification features. Alternatively or additionally, the transport container 110 can also be determined on the basis of its position relative to the other containers 116. For example, it can be stored in the algorithm that the transport container 110 to be monitored is always the trolley 110 parked on the far left in the top view of the shelf arrangement 161 .

In step M2 it can optionally be provided that video image data is recorded with several or all of the plurality of video cameras 2 distributed in the aircraft 100 . In a further optional step M3, a position of the transport container 11 within the aircraft 100 can then be determined using the data processing device 3 based on the video image data recorded by the plurality of video cameras. For example, the processor 30 of the data processing device 3 can execute an algorithm which is designed to identify the trolley 110 in the recorded video image data and also to compare the video image data representing the area around the trolley 110 with a stored reference image of the interior of the aircraft. Alternatively or additionally, each camera 2 can be assigned an identifier and a position within the aircraft 100, with the identifier and position of that camera 2 being determined which captures the image data in which the trolley 110 is recognized. For example, the identifier of a respective camera 2 can be contained in the video image data transmitted by it to the data processing device 3 or can be transmitted together with it. A position can be stored in the data memory 31 for each camera identifier, for example.

As in 5 As further shown, the method M proceeds after step M3 to step M4, in which an intervention in the transport container 110 is determined using the recorded video image data by means of the data processing device 3 . 3 shows purely schematically the transport container 110 during an intervention by a person. As in 3 shown by way of example, a hand X44 reaches into a detection zone 4 in front of the access opening of the trolley 110. The detection zone 4 can be defined, for example, by a cuboid space, as is shown in 3 is shown as an example. If a like in 3 If the situation shown or another process representing an intervention is contained in the video image data, this is recognized by the data processing device 3 . A process representing an intervention is contained in the video image data as an intervention feature X4. The intervention feature can thus very generally be a data pattern in the video image data, which represents an intervention in the trolley 110 . In step M4, the video image data are therefore analyzed by the data processing device 3 for the presence of an intervention feature X4 in a region 4 in front of the transport container 110, ie the detection zone 4. The engagement feature X4 can be, for example, a shape of a human hand X44 and/or a human forearm, as described in 3 is shown as an example. Alternatively or additionally, an optical signature of a human hand X44 and/or a human forearm or a predetermined movement pattern can be recognized as an intervention feature X4.

If an intervention in the transport container 110 is detected in step M4, the method M optionally first goes to step M5, as shown in 5 shown by way of example, or directly to step M6.

As in 5 is shown, an optional step M5 can also be carried out, in which an authorization of the determined intervention is determined using the recorded video image data by means of the data processing device 3 . Similar to the determination of an intervention in step M4, the authorization can be determined by analyzing the video image data by an algorithm executed by the processor 30, which recognizes certain data patterns in the video image data from which it can be concluded whether the intervention in the trolley 110 person is authorized to intervene or not. Determining the authorization can thus generally include determining an authorization feature X5 in the video image data. The authorization feature X5 can very generally be a data pattern in the video image data, which represents the affiliation of the person removing the goods to an authorized group of people, i.e. to a group of people who are authorized to reach into the transport container 110 and optionally remove goods from the transport container 110 . The authorization feature X5 can be defined, for example, by the clothing of the person reaching into the transport container 110 . For example, it can be provided that the uniform of the cabin crew is recognized, that certain ranks of the cabin crew identified on the uniform or other clothing features are recognized. Alternatively or additionally, an optical marking X55 worn by the person reaching into the transport container 110, for example a bracelet or the like as in 3 shown schematically, can be recognized as authorization feature X5. According to a further option, biometric features that identify the person reaching into the transport container 110 can be recognized. This may include, for example, the processor 30 executing a face recognition algorithm. Authorization can still be changed during different flight phases. For example, the data processing device 3 can receive flight phase signals from the central computer 120, which characterize a current flight phase, such as taxi, takeoff, landing, climb, descent, cruise or the like. The persons or groups of persons authorized to intervene can use the data processing device 3 are determined based on the flight phase signal depending on the flight phase.

In step M6, which, as in 5 shown, can follow step M5 or is carried out directly after step M6, the data processing device 3, in particular the processor 30, issues an intervention report. The intervention report is generally a data record containing at least the information that an intervention has taken place. Optionally, the intervention report can also include one or more of the following contents as a data record: a time, a date, a position of the trolley 110 within the aircraft 100, a position of the aircraft 100, an identification number of the aircraft 100. The position of the trolley 110 within the Aircraft 100 may have been determined in step M3, for example. The data processing device 3 can call up the data time, date, position of the aircraft 100 , identification number of the aircraft 100 for example via the first interface 32 from the central computer 120 .

The processor 30 can, for example, output the intervention message to the data memory 31, where it is stored, preferably together with the captured video image data. Alternatively or additionally, the processor 30 can output the intervention report at the first and/or the second data interface 32, 33 of the data processing device 3, as will be explained below.

If the optional step M5 has been carried out, it can be provided that in step M6 the intervention message is only output if it was determined in step M6 that the identified intervention is authorized, or that the intervention message is output with the information that the intervention is authorized. In the latter case, the data record of the intervention report also contains information that the intervention is authorized.

In a further optional step M7, which can be carried out independently of steps M2 to M6, an output R2 of video images is generated on a display, for example the display 150 of the cabin monitoring system 200, based on the recorded video image data. As already mentioned in connection with 1 was explained, the data processing device 3 can, for example, output the video image data at the first data interface 32 and transmit it to the display 150, which displays the video images represented by the video image data.

Optionally, in step M8, the method M can include displaying a marker X2 in the output R2 of the video images if an intrusion into the transport container 110 is determined in step M4. 4 shows schematically and purely by way of example an output R2 while an intervention in the transport container 110 is taking place. In 4 this is symbolically represented by the hand X44. As in 4 is shown, a symbol, for example, can be superimposed on the output R2 as the marking X2. For example, the processor 31 can create a corresponding entry in the video image data before it is transmitted to the display 150 or stored in the data memory 31 .

In step M8, it can optionally be provided that the output R2 of the video images is generated based on a recording of video image data captured in a predetermined recording period. For example, the data processing device 3 can store the video image data captured in a predetermined recording period in the data memory 31 and transmit the stored video image data to the display 150 to generate the output. In this case, a graphic progress indicator P, for example a bar as in 4 shown as an example, are displayed. The progress indicator P represents a beginning P1 and an end P2 of the recording period and a current point in time P3 within the recording period. In 4 for example, the hatched area of the progress bar shows the time period of the recording that is before the current time. The progress indicator P can, as in 4 represented symbolically by bars, are marked at points X3 which correspond to a point in time at which an intervention in the transport container 110 was determined. In 4 a current point in time P3 of the output R2 is shown as an example, at which an intervention in the transport container 110 takes place. Therefore, at time P3, a dash or bar is superimposed as marking X3. To generate the marking X3 of the progress indicator P, the data processing device 3 can, for example, output the intervention notifications to the display 150 or at least the times of the intervention optionally contained in the respective intervention notifications, so that the display can generate the marking X3 itself. Alternatively, the marker X3 can be written by the data processing device 3 as a data entry in the video image data.

As in 5 is also shown, the intervention report can be transmitted M9 to a mobile terminal device 140 in the aircraft 110 in a step M9. For example, the processor 30 can output the intervention report at the second interface 33 which transmits the intervention report to the mobile terminal device 140 via the communication network 130 . The mobile terminal device 140 can be a smart phone, for example or another terminal device assigned to a cabin crew member. In a further step M10, the mobile terminal device 140 can generate an optical and/or a haptic and/or an acoustic warning signal based on this. Alternatively, the terminal 140 can also only store the intervention report. It can optionally be provided that an approval signal can be generated by means of the mobile terminal device 140 and is transmitted to the second interface 33 of the data processing device 3 via the communication network 130 . The data processing device 3 can optionally delete the intervention message in response to the approval signal, e.g. in the data memory 31.

When identifying the transport container 110 in step M2, when determining an intervention in the transport container 110 in step M4 and when determining the authorization of the determined intervention in step M5, the video image data are analyzed for the presence of specific features. For this purpose, in particular, an algorithm for the automated analysis of the video image data can be carried out, which is based on artificial intelligence, AI. In this case, the data processing device 3 may have a supervisory processor and AI processor, which are bidirectionally connected. The monitoring processor and the AI processor can be formed, for example, by different processor cores of the processor 30 or separate processors. The data processing device 3 can also have a set of rules generator based on self-learning algorithms and a reference set of rules memory. The set of rules generator can be stored in the data memory 31 as a software module. The reference set of rules memory can be formed by part of the data memory 31 . The algorithm can include, for example, receiving the video image data by the monitoring processor, checking the video image data for deviations from a reference set of rules stored in a reference set of rules memory of an AI system and generated by the set of rules generator by the AI processor, and outputting indicator signals by the monitoring processor, if the deviations determined by the AI processor are above one or more predefinable deviation threshold values.

Although the present invention has been explained above using exemplary embodiments, it is not limited thereto but can be modified in a variety of ways, and combinations of the above exemplary embodiments are also conceivable.

For example, the video cameras 2 do not necessarily have to be installed in the passenger cabin, but it is also conceivable that they are installed in a cargo hold of the aircraft 100 . In this case, the transport containers can be suitcases, for example, which are stored in the cargo hold. The method described above can thus be used to determine when an intervention in a suitcase takes place, e.g. when it is opened. In this case, too, an authorization of the intervention can be checked, as described above.

For example, certain ground personnel, such as customs officials, may be authorized to intervene and others may not.

Furthermore, it is possible in the manner described above to determine the intervention in hat racks in a passenger cabin. An authorization can be checked here, for example, by assigning specific hat racks to specific people or groups of people. For example, video image data of all passengers during a phase in which they are seated in their assigned seats can be collected as reference data. When passengers move around the aircraft and reach into a hat rack, they can be identified using the reference data. In this case, the authorization feature can be formed, for example, by the person staying within a certain radius of the space assigned to him/her.

Furthermore, the automated evaluation of the image data makes it possible to recognize unusual conditions of rooms, people or animals. For example, if the video camera has a thermal sensor, it can be checked whether a temperature of the room, the person, the animal or the transport container is within a predetermined range. Other unusual states that can be determined, for example, are states of restlessness or being in unauthorized zones.

Reference List

1

system

2

video camera

3

data processing device

4

area

30

processor

31

data storage

32

first interface

33

second interface

100

aircraft

102

floor

104

ceiling

110

transport container

110A

body

110B

Wheels

110C

door

110D

Drawer

116

more containers

116A

body

116B

Wheels

116C

door

116D

Drawer

120

mainframe

130

communication network

140

mobile device

150

screen

160

galley

161

shelving arrangement

200

Cabin Monitoring System

M

procedure

M1-M10

process steps

P

progress indicator

P1

Beginning of the recording period

p2

end of the recording period

P3

current time

R2

output of video images

X2

mark

X3

mark

X4

engagement feature

X5

authorization feature

X44

hand

X55

mark

QUOTES INCLUDED IN DESCRIPTION

This list of the documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• EP 1440883 A1 [0004]
• DE 102007051196 A1 [0004]

Claims (15)

Method (M) for monitoring a transport container (110) for goods in an aircraft (100), comprising: Capturing (M1) video image data of the transport container (110) by means of a video camera (2) installed in the aircraft (100); Determination (M4) of an intervention in the transport container (110) based on the recorded video image data by means of a data processing device (3); and Outputting (M6) an intervention report by the data processing device (3) if an intervention in the transport container (110) is determined. Procedure (M) according to claim 1 , wherein the determination (M4) of the intervention involves analyzing the video image data by the data processing device (3) for the presence of an intervention feature (X4) in an area (4) in front of the transport container (110) through which the goods can be removed from the transport container (110). , and wherein the intervention feature (X4) preferably comprises at least one of the following optical features represented by the video image data: a shape of a human hand (X44) and/or a human forearm, an optical signature of a human hand (X44) and/or a human forearm, a predetermined movement pattern. Method (M) according to one of the preceding claims, wherein the intervention report comprises a data record with one or more of the following contents: a time, a date, a position within the aircraft (100), a position of the aircraft (100), an identification number of the aircraft (100). Method (M) according to any one of the preceding claims, additionally comprising: generating (M7) an output (R2) of video images based on the captured video image data on a display (150); and Displaying (M8) a marker (X2) in the output (R2) of the video images if an intrusion into the transport container (110) is detected. Procedure (M) according to claim 4 , wherein the output (R2) of the video images is generated based on a recording of video image data captured in a predetermined recording period, wherein in the output (R2) of the video images a graphical progress indicator (P), for example a bar, is displayed which has a beginning ( P1) and an end (P2) of the recording period as well as a current point in time (P3) within the recording period, and wherein the progress indicator (P) is marked at positions (X3) which correspond to a point in time at which an intervention in the transport container ( 110) was determined. Method (M) according to any one of the preceding claims, additionally comprising: Transmission (M9) of the intervention report to a mobile terminal (140) in the aircraft (110); and Outputting (M10) a warning signal by the mobile terminal. Method (M) according to any one of the preceding claims, additionally comprising: Determining (M5) an authorization of the determined intervention based on the recorded video image data by means of the data processing device (3); wherein the tampering report is issued only if the determined tampering is authorized, or wherein the tampering report is issued with the information that the tampering is authorized. Procedure (M) according to claim 7 , wherein determining (M5) the authorization includes determining an authorization feature (X5) in the video image data, wherein the authorization feature (X5) is preferably formed by one or more of the following optical features: clothing of the person reaching into the transport container (110). , an optical tag (X55) worn by the person reaching into the transport container (110), biometric features identifying the person reaching into the transport container (110). Method (M) according to one of the preceding claims, additionally comprising: Identification (M2) of the transport container (110), in particular among a plurality of other containers (116), by means of the data processing device (3) based on the recorded video image data. Procedure (M) according to claim 9 , wherein the identification (M2) of the transport container (110) based on an optical marking (111) of the transport container (110), based on optical and / or geometric properties of the transport container (110) as such and / or based on a position of the transport container (110) relative to the other containers (116). Procedure (M) according to claim 9 or 10 , video image data being recorded by means of a large number of video cameras (2) arranged distributed in the aircraft (100), and a position of the transport container (110) inside the aircraft (100) being determined by means of the data processing device (3) on the basis of the recorded video image data ( M3). System (1) for an aircraft (100) for monitoring a transport container (110) for Goods, with: a video camera (2) for capturing video image data of the transport container (110); and a data processing device (3) which is signal-connected to the video camera (2) and is set up to process the video image data captured by the video camera (2) in a method (M) according to one of the preceding claims. System (1) after claim 12 , wherein the data processing device (3) has a first interface (32) for connection to a central computer (120) of the aircraft (100) and/or a second interface (33) for connection to a communication network (130) of the aircraft (100). Aircraft (100), in particular passenger aircraft, with a system (1) according to one of Claims 12 or 13 . Aircraft (100) to Claim 14 comprising: a cabin monitoring system (200) with a large number of video cameras (2) installed distributed within a cabin of the aircraft (100) for capturing video image data and a display (150) signal-connected to the video cameras (2) for generating an output (R2) of video images based on the captured video image data; wherein the system (1) for monitoring a transport container (110) is part of the cabin monitoring system (200) of the aircraft (100); and wherein at least one of the video cameras (2) of the cabin monitoring system (200) is aligned in the direction of a storage area for the transport container (110), which is preferably arranged in the area of a galley (160) of the cabin.

Images