DE102019208871A1 - Device and method for determining the interior state of a vehicle - Google Patents

Abstract

The described device or the method for determining a use-related state of at least one component (150-175, 180-200) arranged in the interior of a vehicle (110) provides that at least one sensor (100, 105) in the vehicle (110) for capturing at least one area of a vehicle interior and preferably a data transmission unit (115) for wireless transmission (125) of the captured data from the vehicle (110) to an external computer system (130), the transmitted data using the external computer system (130) be evaluated. In this case, the external computer system (130) preferably comprises a functional unit (135) for determining suitable measures for possible elimination of an inadequate state of the at least one component (150-175, 180-200).

Description

The invention relates to a device and a method for detecting or determining a use-related state of the interior of a vehicle.

State of the art

Nowadays, there is little or no transparency about the individual usage history or the corresponding usage status of vehicle interiors. Therefore, the resulting mechanical or optical state of a vehicle interior or a passenger cell and the degree of wear and tear on vehicle interior equipment cannot be determined automatically. Such vehicle interiors are e.g. Dashboards, vehicle seats or upholstery, vehicle interior panels, window panes, floor mats as well as components for vehicle operation that are subject to particularly high mechanical loads during driving, e.g. the steering wheel or steering wheel, the gear selector lever or a touch-sensitive screen of a possibly existing infotainment system.

The vehicle manufacturers also do not offer any sensor systems on board a vehicle that allow the vehicle owner or operator to accurately assess the current state of a named vehicle component located in the vehicle interior. In the case of rental vehicles, complaints about damage to or contamination of the vehicle interior are at most reported to the respective vehicle operator by a subsequent user of the vehicle. After an inspection by the operator, these deficiencies may be charged to the previous user who caused them. However, this approach is not very conclusive and also requires a lot of personnel. In addition, this form of checking by a subsequent user at the start of use leads to a temporary, unpredictable non-usability of the vehicle in the event of severe impairment of the interior.

Out EP 1 339 570 A1 a device and a method for the camera-based position detection of elements or components or objects in a vehicle emerge. An optical marking is attached to components of the vehicle, for example to a vehicle seat. A camera that captures the marking photographically is also installed in the vehicle. The position of the marking recorded in this way is determined by means of an evaluation unit. The camera, designed as a stereo camera with two image sensors, also records spatial information of the image area to be captured. For example, the volume distribution of the object on the seat is determined from the contour of each detected object by subtracting the contour of the seat surface. This enables the object to be classified into, for example, child seats and people of different weight classes.

Also go out DE 10 2014 214 352 A1 discloses a method and an arrangement for operating an observation system for occupants, the direction of view or the drowsiness of the driver being monitored by means of a camera device in order to increase the safety in traffic and to improve the comfort of use for the driver. For example, the driver's head position is determined in order to adapt or align the orientation of an image generated by a “head-up” display depending on the detected head position, so that the driver always has the displayed data in view. In order to be able to track the gaze of the driver with high accuracy, the camera device must be calibrated. Internal camera parameters such as distortion or the installation position are taken into account.

Disclosure of the invention

The present invention is based on the idea of specifying a device and a method by means of which the state of the interior or the passenger compartment of a vehicle is recorded, documented and evaluated by a person with a significantly reduced or even completely avoidable testing effort compared to the prior art in order to derive any necessary (maintenance) measures.

The increase in forms of mobility such as "Car sharing", especially with frequently changing vehicle users, this aspect of vehicle use is becoming increasingly relevant. In addition, fully autonomous driving in the area of "car sharing" and so-called "ride hailing" leads to an even more intensive, temporary use of vehicles with significantly higher numbers of users and an even more frequent change of user without these vehicles being able to be checked regularly by one person . With ride hailing, city dwellers in particular can find and book mostly private carpooling via various app providers.

Similar problems can also be found with other forms of mobility such as in public vehicles, e.g. in the case of rail vehicles (passenger trains, trams), buses, automated shuttle vehicles, cable cars, elevators, or the like.

According to a first aspect of the device proposed according to the invention this an arrangement of at least one sensor for data acquisition of at least one area of a vehicle interior. The data recorded by sensors are evaluated by means of a computer system and suitable measures for possible remedying of an inadequate condition of the vehicle interior determined by the computer system on the basis of the recorded sensor data are determined or proposed. The computer system can be a vehicle-internal computer which, in particular in the case of future autonomous vehicles, provides sufficient computing capacity or computing power for evaluating the sensor data with regard to the usage-related state of the vehicle interior.

According to a second aspect of the device proposed according to the invention, it comprises a data transmission unit for the wireless transmission of the data recorded by sensors from the vehicle to an external computer system. By means of the external computer system, these data can be evaluated with high computing power and suitable measures for possible remedying of an inadequate condition of the vehicle interior determined by the external computer system on the basis of the recorded sensor data can be determined or proposed.

The proposed device differs from the prior art in particular in that only temporary data acquisition and thus no continuous data acquisition is required and that, instead of people, the vehicle interior and its usage-related condition are recorded. In addition, it can be avoided in particular that the data acquisition takes place while the vehicle is in motion.

• - Einen oder mehrere Sensoren, welche in dem Fahrzeuginnenraum angeordnet sind und diesen zumindest teilweise räumlich erfassen, z.B. im sichtbaren Licht oder im UV-Bereich erfassende Fotokameras und/oder chemische Sensoren zur Erfassung insbesondere von im Fahrzeuginnenraum sich entwickelnden Gerüchen;
• - eine Datenübertragungseinheit zur drahtlosen Übertragung der erfassten Sensorsignale an ein externes Rechnersystem, z.B. eine Cloud-Computing Plattform.

According to a further aspect, the proposed device preferably comprises the following two technical functional units arranged in or on the vehicle:

• One or more sensors which are arranged in the vehicle interior and which at least partially detect it spatially, for example photo cameras and / or chemical sensors which detect in visible light or in the UV range for detecting, in particular, odors developing in the vehicle interior;
• - A data transmission unit for wireless transmission of the detected sensor signals to an external computer system, for example a cloud computing platform.

According to a further aspect, the proposed device comprises a computer-aided evaluation system which is preferably arranged in the external computer system and to which the recorded sensor data are transmitted and processed there electronically. The evaluation system is preferably designed as a self-learning KI system and comprises an artificial neural network (ANN).

The ANN can e.g. with photographs of exemplary damage and / or soiling of the vehicle interior, e.g. the seat upholstery or the seat covers of the vehicle, or the like, must be trained in advance. On the basis of this data, e.g. currently captured photographs can be (in) classified into various damage or pollution classes. During operation of the vehicle, based on this, currently captured photographic material can also be classified or evaluated accordingly with regard to the severity of any damage or contamination that may be present and implemented accordingly using a system of measures described below.

According to yet another aspect, the device proposed according to the invention additionally comprises one or more gas sensors, by means of which the interior air can be examined for molecules relevant to smell or pollution. As gas sensors e.g. chemical and / or physical sensors known per se for the detection of gaseous substances, which convert chemical information from the ambient air into an electrically usable signal, can be considered. The sensor signals recorded in this way can also be trained in advance by means of an ANN, with the sensor signals being able to be assigned to different, odor-relevant contaminants or types of contamination in the vehicle interior. On the basis of an appropriately set up AI system, corresponding, currently recorded sensor data can then be assigned to a specific type of contamination and thus implemented using the system of measures described below.

According to a further aspect, one or more retrofittable photo cameras, for example on the inside mirror, on the vehicle roof lining, on the inside of the A, B or C-pillar of the vehicle chassis, on one or more vehicle seats, and / or on the center console of the vehicle be arranged. The cameras either cover a large angular range, for example by means of a wide-angle lens or by swiveling the camera, in order to be able to record, for example, the general state of contamination in the interior of the vehicle. Alternatively, a single camera can also cover a spatially limited area of the interior, for example the driver's seat or the dashboard. In the latter case, several cameras are preferred provided in order to be able to cover or capture the entire interior as possible.

It should be noted here that cameras arranged in the interior of the vehicle for the stated purpose must, however, be operated for reasons of data protection law or technology so that they can e.g. only take photos when there is no occupant on board the vehicle if possible. Suitable times for this are e.g. unlocking or locking the driver's door before a planned or after use of the vehicle.

It should also be noted that camera recordings or the like can be evaluated both by human observers and by deterministic or self-learning electronic evaluation algorithms. The evaluation of information on the part of the gas or odor sensors is, however, preferably carried out using electronically executable algorithms for reasons of the complexity of the data recorded.

According to a further aspect, the device comprises a system of measures, which is preferably also designed as a self-learning ANN system. Based on the results of the above-mentioned evaluation of the recorded sensor data, the system of measures automatically suggests necessary measures, e.g. Cleaning or repair measures on the parts of the vehicle interior that are affected.

As an alternative or in addition to the measures mentioned, technical interventions can be carried out on the vehicle according to a further aspect, e.g. a technical inspection, maintenance measures, repair measures or the activation or strengthening of interior ventilation of the vehicle in order to at least temporarily remove or suppress any odors that may be present.

Alternatively or additionally, purely organizational measures, e.g. blocking or preventing a sublet of the vehicle. Again as an alternative or in addition, the vehicle operator can e.g. Claims for damages can be asserted against the responsible party.

According to a first aspect of the method proposed according to the invention, the mechanical and / or optical state of at least one vehicle component arranged in the vehicle interior is recorded at least in one area of the vehicle interior using sensors and / or camera technology. The data recorded by sensors are processed and, on the basis of the results of the processing of these data, a possibly inadequate condition of the vehicle interior or at least one of the components mentioned is determined.

According to a further aspect of the method proposed according to the invention, the data captured by sensors or camera technology are initially only preprocessed in the vehicle and the corresponding preprocessed data are transmitted wirelessly from the vehicle to an external computer system. These data are evaluated by means of the external computer system and suitable measures are proposed for possible rectification of an inadequate condition of the vehicle interior determined by the external computer system on the basis of the transmitted sensor data.

According to yet another aspect of the method proposed according to the invention, the data transmitted wirelessly to the external computer system are evaluated by means of an evaluation system arranged in the external computer system, which is preferably also designed as a self-learning ANN system. The ANN can e.g. with photographs of exemplary damage and / or soiling of the vehicle interior, e.g. the seat upholstery or the seat covers of the vehicle, or the like, can be trained or learned or (in) classified in advance. On the basis of this data, e.g. currently captured photographs can be classified into various damage or soiling classes. During operation of the vehicle, based on this, currently captured photographic material can be classified or evaluated accordingly with regard to the severity of the respective damage or contamination and implemented accordingly using suitable measures to remedy these defects.

Suitable measures are e.g. Consider cleaning or repair measures on the respective components or parts of the vehicle interior. As an alternative or in addition to the measures mentioned, technical interventions can be carried out on the vehicle, e.g. a technical inspection, maintenance measures, repair measures or the activation or strengthening of interior ventilation of the vehicle in order to at least temporarily remove or suppress any odors that may be present.

According to a further aspect of the method proposed according to the invention, feature vectors can be used in the classification of the recorded sensor or camera data on the basis of empirically predeterminable types of damage or contamination, for example using a per se well-known "Random Forest" classifier. As is well known, a "random forest" classifier consists of several uncorrelated decision trees which are formed using a random mechanism (so-called "randomization") during an underlying learning process. For a classification, each decision tree makes a decision, with the class with the most votes determining the final classification.

Particularly computationally intensive calculation steps in the above-mentioned classification or determination of the interior state of a vehicle affected here can thus be carried out on an (external) computer arranged outside the vehicle, e.g. a cloud computing platform. Less computationally intensive calculation steps, on the other hand, can already be carried out in the vehicle and only correspondingly preprocessed and thus correspondingly compressed data can be transmitted to the external computer, in particular to reduce or minimize the data volume of the data to be transmitted.

In addition, parameters and hyperparameters of a named classifier can be learned or trained on the basis of the named training data using known machine learning algorithms, with corresponding training data, preferably specific to an individual vehicle or vehicle type, at a respective test stand can be generated.

The proposed method thus enables a precise and, in particular, temporally comprehensive analysis of the sensor data concerned here with regard to the interior state of a vehicle. As a result, a complete usage history can even be created for a specific vehicle, which can include a wide variety of contamination and / or damage events within the entire vehicle life cycle. With regard to their significance for the value of the vehicle, these events can include the mechanical or optical condition of certain vehicle interior components as well as the traceability and severity of damage or contamination in the interior of the vehicle caused by vehicle abuse.

The invention can be used accordingly in particular in a land vehicle or motor vehicle, but in principle also in watercraft, rail vehicles, aircraft / spacecraft or the like, which are subject to a use-dependent risk of wear, contamination and / or damage.

The computer program is set up to carry out each step of the method, in particular when it runs on a computing device or in a control device arranged on the respective vehicle. It also enables the implementation of the method in an already existing control unit of the vehicle without having to make structural changes to the control unit. The machine-readable data carrier on which the computer program is stored is provided for this purpose. By uploading the computer program to a microprocessor or microcontroller of such a control device, the latter is set up to carry out the method.

Further advantages and configurations of the invention emerge from the description and the accompanying drawings.

It goes without saying that the features mentioned above and those yet to be explained below can be used not only in the respective specified combination, but also in other combinations or on their own, without departing from the scope of the present invention.

Figure list

• 1 shows an embodiment of the device according to the invention in a schematic side view of a vehicle interior.
• 2 shows an embodiment of an evaluation process that can be carried out in an external evaluation unit on the basis of a combined block / flow diagram.
• 3 shows an exemplary embodiment of a preprocessing, which can be carried out in a vehicle, of acquired sensor data on the basis of a combined block / flow diagram.
• 4th shows an exemplary embodiment of process steps running in more detail in an external evaluation unit for evaluating acquired sensor data, using a flow chart.

Description of exemplary embodiments

In the 1 The device shown comprises at least one, but preferably several sensors 100 , 105 for detecting the condition of the interior of a vehicle 110 characterizing data. The sensors 100 , 105 are preferred in the interior of the vehicle 110 arranged. As sensors 100 , 105 for example photo cameras, photo-optical sensors, optical IR or UV sensors, In the visible or non-visible wavelength range working or sensitive laser sensors and / or chemical sensors, for example odor-sensitive gas sensors, into consideration.

In addition, is in the vehicle 110 a communication unit 115 provided for preferably wireless data transmission of the acquired sensor data, in order to transfer the acquired data as promptly as possible to a computer-aided evaluation unit external to the vehicle 120 to be able to transfer. The communication unit 115 can transmit the data using cellular radio or another wireless transmission technology, eg "Wireless LAN". Alternatively, one in the vehicle can be used for data transmission 110 standard already existing transmission unit can be used or at least shared.

The external evaluation unit 120 evaluates the received data 125 out to the external, ie optical and / or odor-related condition of the vehicle interior, and / or possibly additionally the mechanical functional condition of an interior component of the vehicle 110 to be able to determine as precisely as possible. The external evaluation unit 120 is implemented in an external computer system, in the present embodiment in a schematically indicated “cloud computing” platform 130 .

It should be emphasized here that a mechanical functional state determined in this way of a (functional) component required to operate the vehicle can also be used to detect changes in the functional state of the component caused by the operation of the vehicle by the driver or a passenger and changes caused thereby to be able to distinguish the externally visible state of the component from changes in state actually to be determined in the present case, so that these do not have to be taken into account in the subsequent evaluation. Such operational changes in the appearance can e.g. Changes in the angle of rotation of a steering wheel, the position of a shift lever or an open window. In this case, respective changes in the functional state of such a functional component can additionally be correlated or checked for plausibility with corresponding operating data from the vehicle electronics, or such functional components can not be taken into account at all in the evaluation method or excluded from it.

The external evaluation unit 120 supplies the results of the evaluation to one, preferably also with regard to the vehicle 110 externally established, suitable measures to remedy an inadequate condition of the vehicle interior proposing or implementing functional unit 135 . This functional unit 135 is set up in the present exemplary embodiment to determine suitable measures based on experience or learning on the basis of detected contamination and / or damage to the vehicle interior and to a third party, for example the vehicle owner or lessor 137 to propose or to the vehicle 110 transfer back 140 . The implementation of the measures can also be carried out by the functional unit 135 already initiated.

The in 1 The schematically illustrated embodiment of the device are two digital cameras as sensors 100 , 105 provided, which are arranged or positioned in the vehicle interior in such a way that even with this relatively small number of cameras, the best possible overview of the entire vehicle interior is made possible. So instead of the two cameras 100 , 105 a single camera can also be provided which photographically captures the entire vehicle interior by means of a panorama lens.

Such is the first camera equipped with a wide-angle lens 100 so opposite the longitudinal axis 112 of the vehicle 110 roughly in the middle of the vehicle roof 145 of the vehicle 110 arranged to be the front interior of the vehicle 110 almost completely covered in terms of camera technology. In particular, the first camera enables 100 the detection of the driver 150 and passenger seat 155 as well as the front vehicle floor 160 from the top view, the dashboard 165 , including the steering wheel 170 , as well as the front window (or windshield) 172 , the front side windows 173 as well as the front area 175 of the vehicle sky 145 .

Correspondingly, it is also recorded with respect to the longitudinal axis 112 of the vehicle 110 centrally arranged, second camera 105 the rear area (so-called "rear") of the vehicle 110 , the rear seats in the present embodiment 180 and the footwell in front of it 185 includes. The second camera also records 105 the parcel shelf 190 and the rear side windows 195 as well as the rear window 200 .

It should be noted in this regard that with a correspondingly adapted lens optics of the respective camera 100 , 105 on the one hand, the largest possible image section should be made possible. On the other hand, an unintended detection of people or objects outside the vehicle is also intended 110 can be effectively prevented through one or more of the vehicle windows. Alternatively, it can be provided that after an image capture in the respective photograph, Image areas possibly protected by copyright before the images are transmitted to the external evaluation unit 120 , for example by means of a corresponding, per se known electronic image processing method, are automatically hidden.

There is a continuous image acquisition by means of said cameras 100 , 105 In addition, if existing data protection laws would generally be violated, provision can be made for the image acquisition to take place only when the vehicle starts and / or ends 110 or by the beginning and / or the end of a rental of the vehicle 110 is triggered. In the present exemplary embodiment, it is triggered by a corresponding one with the on-board electronics of the vehicle 110 connected electronic trigger unit 202 . Because at these times the vehicle is 110 locked and there are no people in the vehicle 110 . At these times are from the cameras 100 , 105 only once pictures taken each time. This can thus ensure that the state of the interior of the vehicle 110 is documented graphically at least immediately before and after the vehicle is used.

Either a radio signal from the vehicle rental company or a signal from a “car sharing box” that may already be present in the vehicle can also be used to trigger the image acquisition over time. Such a box already contains the corresponding communication interfaces. In addition, it can be provided that event-related live recordings (photo and / or video recordings) are initiated, e.g. triggered by a vehicle alarm system or the like.

It goes without saying that the aforementioned data protection and copyright problems are already caused by the use of sensors that do not take pictures or that do not produce photographs, e.g. Laser sensors, or UV sensors, can be effectively prevented. In addition, technologies already exist in the so-called “in-car monitoring” area to record objects, movements of objects, object temperatures, or the like without creating a visual image of the respective object. Similar to body scanners that are already in use at some airports, in which people are screened with electromagnetic radiation in the THz (terahertz) frequency range, this detection can take place.

The said digital cameras 100 , 105 can also be operated in spectral ranges outside the visible light spectrum, eg in the IR or UV near range. The near-UV range is particularly advantageous for the superficial or near-surface detection of organic substances, for example on the seat cushions 150 , 155 , 180 of the vehicle 110 . This also means that the vehicle is operated at night 110 or images of sufficient quality can be created when the lighting conditions are inadequate, it is known that corresponding lighting or flash functions can be integrated into the cameras 100 , 105 be integrated, e.g. based on LED technology.

In addition to the cameras 100 , 105 For example, electronic gas sensors (not shown here) serving as air quality sensors, so to speak, can be provided, by means of which the air in the vehicle interior can also be examined for pollution-relevant molecules. These sensors are, for example, known (for example metal oxide-based) VOC sensors (VOC = Volatile Organic Compounds), which react with an electronic signal to certain molecules that dock on the sensors' receptors. To learn or train an ANN to evaluate corresponding sensor data, the sensors can be used on various, empirically predeterminable, contamination-relevant molecule combinations. Examples of such molecule combinations are tobacco smoke, feces, leftover food, rotting garbage, or the like.

The odor or air quality sensor (s) can advantageously be integrated with one of the two or both cameras 100 , 105 to be ordered. Depending on the installation situation of these sensors, the natural convection of the indoor air can be used to increase the supply of molecules to the odor sensor (s).

Alternatively, it is possible to supply the interior air to the respective sensor surface by means of a fan integrated into the camera or sensor housing. In both cases, the camera or sensor housing is equipped with corresponding openings for the entry and exit of air from the vehicle interior.

Alternatively, the odor sensor (s) can be arranged in an intake duct of the vehicle ventilation system for air circulation. For the sensor measurement, a corresponding interface is then used to switch to recirculation mode (at least) for the duration of the measurement.

In the present embodiment is in the vehicle 110 additionally a preprocessing unit 205 arranged, which has a processor unit (eg a microcontroller) or control electronics 210 as well as an interface 215 to the sensors 100 , 105 includes. The preprocessing unit 205 is also in terms of communication or data technology with said communication unit 115 connected 223 and preferably with its own energy supply (e.g. rechargeable battery) 220 equipped which in operation of the vehicle 110 For example, it can be charged via its on-board electrical system.

The control electronics 210 further comprises a data memory 225 for the intermediate storage of the data from the sensors 100 , 105 captured sensor data. The data obtained in this way, in particular image data and, if applicable, electronic signals from the odor sensors mentioned and, if applicable, further sensors, are stored in the data memory after being temporarily stored 225 in the vehicle 110 pre-processed and pre-evaluated with regard to any deviations or anomalies that may be present in the data, specifically in comparison with reference data.

If there is a deviation from the reference data, then the extensively recorded sensor data are sent to the external evaluation unit in the manner described 120 transmitted. If, however, no deviation from the reference is found, only a corresponding zero signal is transmitted, which is sent to the external evaluation unit 120 signals that no abnormalities or anomalies were detected in the current sensor data. This preprocessing process is used for data transmission to the external evaluation unit 120 the intended data volume advantageously reduced to a minimum. The preprocessing can be based on data of the object recognition by means of the cameras 100 , 105 as well as to data of the odor or pollution detection by means of a named gas or air quality sensor.

In the 2 is an embodiment of the in the external evaluation unit 120 (please refer 1 ) running evaluation process is shown on the basis of a combined block / flow diagram. The evaluation process includes that indicated by a first dashed line 301 delimited method steps for determining the usage-related interior state or vehicle components or objects of the vehicle arranged in the interior, as well as those indicated by a second dashed line 302 delimited procedural steps for the implementation of measures based on the results of the determination.

In the exemplary embodiment, for the sake of simplicity, it is assumed that in the vehicle 110 arranged sensors only those in 1 shown two cameras 100 , 105 for generating said image data. The one from the vehicle 110 to the evaluation unit 120 transmitted wirelessly 300 Image data are initially processed in a multi-stage process (according to the first dashed line 301 ) evaluated according to the procedural steps described below.

In step 305 compares the evaluation unit 120 the first image data thus transmitted 300 , which the front vehicle interior area with the interior components 150-175 as well as the rear vehicle interior area with the interior components 180-200 comprise, by means of image recognition software known per se, with second image data representing a target state of these two vehicle interior areas 310 . The second image data 310 are formed in the present exemplary embodiment by reference recordings which are on an (individually) affected vehicle or a corresponding vehicle type, preferably at the time of delivery of the vehicle from the factory 110 , were recorded. Based on the comparison 305 the current state of the interior is inferred.

If in the comparative test step 305 an empirically predeterminable threshold value 315 excessive deviations between the first image data 300 and the second image data 315 have shown, then these deviations are evaluated with regard to the current degree of soiling and / or damage 320 and corresponding measures, described in more detail below, for further treatment of the vehicle 110 initiated 325 .

The one from the evaluation unit 120 In the present exemplary embodiment, detected deviations are additionally marked in the respective digital image files for simplified further processing 330 , for example highlighted by appropriate borders, colored shading or other visual markings.

• - Rücksitz umgeklappt,
• - Kopfstütze fehlt,
• - Unrat im Fahrzeug,
• - Fleck auf Polster,
• - Sitzfläche aufgerissen.

In the present exemplary embodiment, an object recognition and object classification function is also included in the image recognition software 335 integrated. This enables object-specific recognition or determination results and, as already described, recognition results which are dependent on a functional state of a respective object. The quality of detection of object-specific soiling of objects or components of the vehicle interior can thereby be improved. The classification function 335 makes it possible to use reference object classes to assign detected deviations to corresponding object categories. Possible object categories are, for example:

• - rear seat folded down,
• - headrest missing,
• - rubbish in the vehicle,
• - stain on upholstery,
• - Seat torn open.

The image information mentioned can, in conjunction with a possibly in the vehicle interior arranged odor sensor, can be enriched by corresponding additional sensor data. For example, it can be classified as to whether a recognized stain on a seat cushion is water, leftover food or vomit. In addition, the cameras 100 , 105 Undetectable, hidden soiling, such as leftovers or rubbish under a seat, can be detected.

The status data determined as described are subsequently, according to the by the second dashed line 302 delimited process steps 360-375 , implemented in suitable measures to remedy or eliminate the corresponding defects in the vehicle interior.

Suitable measures can either be determined in advance on the basis of a specifiable catalog of measures or, as in the present exemplary embodiment, by means of an ANN-based one that is capable of learning 360 Optimization process, which is based on the effectiveness of already implemented measures 365 learns 370 and thereby constantly improving. On the basis of this process, suitable measures are initiated automatically or independently and, if necessary, communicated to service personnel in various ways 375 , e.g. via smartphone app, SMS, or the like.

• - Technische Inspektion,
• - Reinigung der verunreinigten Komponenten bzw. Objekte,
• - Reparatur der beschädigten Komponenten bzw. Objekte,
• - Anpassung der Lüftungsregelung des Fahrzeugs.

The following technical interventions can be considered as suitable measures:

• - technical inspection,
• - cleaning of the contaminated components or objects,
• - repair of damaged components or objects,
• - Adaptation of the vehicle's ventilation control.

• - das Blockieren des Fahrzeugs für eine Weitervermietung,
• - das Verbringen des Fahrzeugs zur Instandhaltung,
• - das Stellen von rechtlichen Ansprüchen an den jeweiligen Verursacher,
• - das Einleiten eines Versicherungsfalls,
• - oder dergleichen,

vorgesehen sein. Hierzu ist das gezeigte Maßnahmensystem 302 ggf. mit weiteren Systemen verbunden, z.B. mit einem Buchungssystem, und kann auch mit diesem System Informationen austauschen.Purely administrative measures can also be taken as further measures, e.g.

• - blocking the vehicle for subletting,
• - taking the vehicle for maintenance,
• - making legal claims to the respective polluter,
• - the initiation of an insured event,
• - or similar,

be provided. The system of measures shown is for this purpose 302 possibly connected with other systems, eg with a booking system, and can also exchange information with this system.

In the 3 is an embodiment of a procedure for the aforementioned preprocessing of the to the external evaluation unit 120 data to be transmitted, specifically for reducing or compressing the respective amounts of data to be transmitted, shown on the basis of a combined block / flow diagram.

Currently available or cached camera or sensor data 400 are with the mentioned reference data 405 compared 410 . In the comparison 410 empirically specifiable threshold values are used to detect relevant abnormalities or deviations 415 based on. Relative changes or corresponding percentage values for a minimum deviation of, for example, 30% of the pixel values affected in each case in a vehicle component can thus be specified as threshold values. If the respective threshold value is exceeded, the corresponding data are sent to the external evaluation unit 120 transferred for more detailed analysis or evaluation 420 . If the threshold value is not exceeded, the procedure is only carried out again when new sensor data is available 425 .

In the 4th is an exemplary embodiment of process steps for executing an evaluation algorithm, in particular for determining a named state for assessing the superficial or optical usage-related condition of the front seat cushions 150 , 155 and the rear seat cushion 180 based on photographically created image data 500 shown. Additionally available sensor data 505 of a gas or odor sensor mentioned. The sensor data 505 serve in particular from the image data 500 obtained information about the interior condition of the vehicle with the sensor data 505 to correlate and to be able to subject the information obtained to a plausibility check.

• - Geometrische Form der fotografisch erfassten Änderung;
• - Räumliche Position der erfassten Änderung;
• - Farbgebung der erfassten Änderung;
• - Flächenanteil/-ausdehnung der erfassten Änderung.

From the image data 500 and the sensor data 505 feature vectors are formed 510 . The following features, for example, are extracted from the image data:

• - Geometric shape of the photographed change;
• - spatial position of the change recorded;
• - Coloring of the recorded change;
• - Area share / extent of the recorded change.

• - Chemische Zusammensetzung der sensorisch erfassten gasförmigen Substanz;
• - Chemische Verbindungsklasse/-gruppe der sensorisch erfassten gasförmigen Substanz;
• - Gasanteil der erfassten Substanz in Bezug auf die Innenluft in %.

The following features, for example, are extracted from the sensor data:

• - Chemical composition of the gaseous substance detected by sensors;
• - Chemical compound class / group of the gaseous substance detected by sensors;
• - Gas content of the detected substance in relation to the indoor air in%.

A classification takes place on the basis of the feature vectors formed in this way 515 the respective data in accordance with the corresponding possible wear and contamination classes created in advance 520 . The classification 515 takes place in the present exemplary embodiment using what is known as a “random forest” classifier, in which case only the superficial state of the vehicle seats is to be classified for purposes of simplification.

In the case of the random forest classifier, as is known, a learning process is formed from a plurality of uncorrelated decision trees, using a random mechanism. For the classification, each decision tree makes a decision, with the class with the most votes determining the final classification. The parameters and hyperparameters of the classifier can be learned or trained on the basis of training data, with corresponding training data being able to be created on a test stand or in real road traffic.

The mentioned different types of usage-related properties of the respective components or objects can thus be identified, classified and the resulting degree of soiling, wear and / or damage derived. The severity can e.g. local contamination of a seat cushion can be quantified as “low”, “medium” or “high”. In this way, the current interior state of the entire vehicle can be reliably determined.

The usage-related properties can also be grouped into different soiling, wear and / or damage groups, each group corresponding to a different type of soiling or wear and tear or damage to an interior component of the vehicle concerned here.

Using the e.g. Using an artificial neural network (ANN), machine learning processes can also be used to identify certain patterns and regularities in the data recorded by sensors, in order to draw reliable conclusions from the (real) measured data about previous events affecting the interior of the vehicle to be able to. To train or learn the ANN, e.g. on a test stand, determined training data on the contamination, wear and / or damage states typically affected here are used as a basis.

From the feature extraction 510 and possibly the data resulting from the aforementioned ANN-based method, different conclusions can finally be drawn about the vehicle history with regard to the vehicle interior. In this way, the current state of the interior can be determined 525 , for example by specifying a number of points corresponding to the stated severity ("score"). This number of points can also be compared with the current mileage 530 of the vehicle are correlated 535 .

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 1339570 A1 [0004]
• DE 102014214352 A1 [0005]

Claims (18)

Device for determining a usage-related state of at least one component (150-175, 180-200) arranged in the interior of a vehicle (110), characterized in that at least one sensor (100, 105) in the vehicle (110) for detecting at least one, the area of the interior of the vehicle (119) comprising at least one component (150-175, 180-200) is arranged, wherein the recorded sensor data is provided by means of a computer system with regard to the usage-related state of the at least one component (150-175, 180-200 ) of the interior can be evaluated. Establishment according to Claim 1 , characterized in that a data transmission unit (115) for wireless transmission (125) of the recorded data from the vehicle (110) to an external computer system (130) is arranged in the vehicle (110), wherein the external computer system (130) is used for the The transmitted data can be evaluated with regard to the usage-related state of the at least one component (150-175, 180-200) of the interior space. Establishment according to Claim 1 or 2 , characterized in that an internal or the external computer system (130) has a functional unit (135) for determining suitable measures for possible elimination of an inadequate state of the at least one component (150-175, 180-200). Setup according to one of the Claims 1 to 3 , characterized in that the at least one sensor (100, 105) is formed by at least one photo camera and / or video camera and / or a laser sensor for photographic or laser-based detection of the respective area of the interior in visible light and / or in the UV range . Establishment according to Claim 4 , characterized in that the at least one sensor (100, 105) is additionally formed by at least one gas sensor for detecting a gas composition in the interior. Establishment according to Claim 5 , characterized in that the at least one gas sensor is formed by a VOC sensor. Device according to one of the preceding claims, characterized in that the external computer system (130) has an electronic evaluation unit (120) which evaluates the data transmitted by the vehicle (110) by means of an artificial neural network. Establishment according to Claim 7 , characterized in that the functional unit (135) comprises an artificial neural network for determining suitable measures. Method for determining a usage-related state of at least one component (150-175, 180-200) arranged in the interior of a vehicle (110) by means of a device according to one of the preceding claims, characterized in that at least one area of the interior uses sensors (100, 105 ) recorded data are evaluated and, based on the results of the evaluation, a possibly inadequate state of the at least one component (150-175, 180-200) is concluded (525). Procedure according to Claim 9 , characterized in that the data recorded by sensors are preprocessed in the vehicle (110) and that the preprocessed data are transmitted wirelessly from the vehicle (110) to an external computer system (130), by means of which the transmitted data are evaluated, in order to determine suitable measures for the possible remedy of a possibly inadequate condition of the interior. Procedure according to Claim 9 or 10 , characterized in that the external computer system (130) evaluates the transmitted (125) data by means of an artificial neural network, which on the basis of empirically specifiable photographic and / or laser-based image reference data and / or from empirically specifiable sensor data of a gas sensor on typical damage and / or Soiling of the at least one component (150-175, 180-200) of the interior is trained or taught in advance. Procedure according to Claim 11 , characterized in that the typical damage and / or soiling of the at least one component (150-175, 180-200) of the interior space are classified in advance into different damage or soiling classes. Procedure according to Claim 12 , characterized in that at least in one area of the interior, data currently recorded by sensors (100, 105) are evaluated on the basis of the damage or contamination classes created in advance with regard to the severity of the respective damage or contamination. Method according to one of the Claims 9 to 13 , characterized in that photographic and / or laser-based image data and chemical sensor data of a gas sensor are recorded by sensors (100, 105) and that the image data and chemical sensor data recorded in this way during the Evaluation with regard to the presence of damage or contamination of the at least one component (150-175, 180-200) of the interior can be correlated with one another. Method according to one of the Claims 9 to 14th , characterized by the following steps: feature extraction (510) for providing characteristic features in the sensor data using feature vectors to be generated, classification (515) of the sensor data using the feature vectors generated. Procedure according to Claim 15 , characterized in that the classification is carried out using a "random forest" classifier. Computer program which is set up, each step of a method according to one of the Claims 9 to 16 perform. Control unit of a vehicle, which is set up, by uploading the computer program according to Claim 17 on a microprocessor or microcontroller of the control device the method according to one of the Claims 9 to 16 execute.

Images