GB2534287A

GB2534287A - Method for providing sensor data provided by sensor devices by vehicles

Info

Publication number: GB2534287A
Application number: GB1523088.1A
Authority: GB
Inventors: Busenbark Christopher
Original assignee: Daimler AG
Current assignee: Mercedes Benz Group AG
Priority date: 2015-12-30
Filing date: 2015-12-30
Publication date: 2016-07-20
Also published as: GB201523088D0

Abstract

The invention relates to a method for providing sensor data 30 provided by respective sensor devices 20 of respective vehicles 10, the method comprising: by at least one server 28 receiving the sensor data provided by the sensor devices; and by the server providing the received sensor data via at least one computer network 34 to which the server is connected thereby making the received sensor data available for terminal devices connected to the computer network. A graphical user interface may be displayed on the terminal devices such that users can select the provided data. The data may comprise video or still images of a portion of the surroundings of the vehicle captured by a camera, and may also comprise audio data captured by a microphone within the vehicle. The data may be recorded and stored in a memory of the server for broadcast at a later time, and may also be classified as public or private. The method allows the driver of a vehicle to share live video and audio data with users at remote locations.

Description

Method for Providing Sensor Data provided by Sensor Devices by Vehicles The invention relates to a method for providing sensor data provided by respective sensor devices of respective vehicles.

DE 103 18 410 Al shows a method and a system for transmitting video data from vehicles. Moreover, US 8 835 045 B2 shows a vehicle surrounding monitoring device comprising at least one camera installed in an own vehicle to obtain an image around an own vehicle. The vehicle surrounding monitoring device further comprises an obstacle detection device installed on in the own vehicle configured to detect an obstacle in an obstacle detection range. Moreover, the monitoring device comprises an image processing device configured to convert a camera image imaged by the camera into a view point converted image seen from a virtual view point above the own vehicle.

It is an object of the present invention to provide a method by means of which sensor data from vehicles can be provided particularly advantageously.

This object is solved by a method having the features of patent claim 1. Advantageous embodiments with expedient developments of the invention are indicated in the other patent claims.

The invention relates to a method for providing sensor data provided by respective sensor devices of respective vehicles. For example, the respective sensor device is configured to capture at least a portion of the surroundings of the respective vehicle so that the sensor data provided by the respective sensor device characterize or comprise the captured portion. For example, the respective sensor device comprises at least one camera configured to capture at least one image or at least one video of said portion. Alternatively or additionally, the respective sensor device can comprise at least one microphone configured to capture noise or sound in the interior of the vehicle and/or in the surroundings of the vehicle. Thus, the sensor data characterize or comprise the captured image, video and/or sound.

The method according to the present invention comprises a first step of, by at least one server, receiving the sensor data provided by the sensor devices. The method comprises a second step of, by the server, providing the received sensor data via at least one computer network to which the server is connected thereby making the received sensor data available for terminal devices connected to the computer network. Thereby, the sensor data received by the server can be accessed by the terminal devices so that users of said terminal devices can access the sensor data via the terminal devices. For example, a platform such as a digital platform or a social network platform can be created, said platform being able to parse the sensor data provided by the sensor devices so that the sensor data is available for consuming.

For example, the sensor data characterize a status of the vehicle and/or at least a portion of the surroundings of the respective vehicle so that, for example, via said platform users other than the driver of the vehicle can experience what the driver of the respective vehicle experienced or is experiencing by means of the sensor data. The method of the present invention allows the user or the driver of the vehicle to realize and share their vehicle's sensor data again and again. Moreover, the method allows to parse shared, in particular publicly shared sensor data. Moreover, the method can allow safety improvements based off of vehicle sensor data that has been involved in an accident and much more. Moreover, the shared sensor data can be used to improve algorithms.

Further advantages, features, and details of the invention derive from the following description of a preferred embodiment as well as from the drawings. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figures and/or shown in the figures alone can be employed not only in the respectively indicated combination but also in other combination or taken alone without leaving the scope of the invention.

The drawings show in: Fig. 1 a schematic side view of a vehicle having at least one sensor device configured to provide sensor data which can be streamed and shared via computer network; and Fig. 2 a graphical user interface provided by a digital platform via which the streamed and shared sensor data can be consumed.

In the figures the same elements or elements having the same functions are indicated by the same reference signs.

Fig. 1 shows in a schematic side view a vehicle 10 in the form of a motor vehicle which is driven along a road 12 by a user 14 of the vehicle 10. For example, the vehicle 10 is a passenger vehicle having a body 16 bounding an interior 18 of the vehicle 10. The user 14 is the driver of the vehicle 10 which further comprises at least one sensor device 20. The sensor device 20 is configured to detect at least a portion of the surroundings 22 of the vehicle 10. For example, the sensor device 20 comprises at least one camera configured to capture a video of said portion of the surroundings 22 while the vehicle 10 is driven along the road 12. Moreover, for example, the sensor device 20 comprises at least one microphone configured to capture sound or noise in the interior 18. The sensor device 20 is configured to provide sensor data characterizing or comprising the captured video and the captured sound. Moreover, the vehicle 10 comprises at least one operating element 24 which is, for example, configured as a button arranged on the steering wheel 26 of the vehicle 10.

In the following, a method for streaming and, thus, sharing said sensor data provided by the sensor device 20 is described. Said method comprises a first step of detecting that the operating element 24 is operated by the user 14. For example, the user 14 pushes the button thereby operating the operating element 24. The method further comprises a second step in which, as a result of detecting that the operating element 24 is operated by the user 14, the sensor data provided by the sensor device 20 are wirelessly streamed to at least one server 28. In Fig. 1, the sensor data which are wirelessly streamed to the server 28 are indicated by 30. The server 28 is configured to receive the streamed sensor data 30 and broadcast the received sensor data 30 as a live stream 32 via at least one computer network 34 to terminal devices 36 connected to the computer network 34. Since the sensor data 30 are streamed, the sensor data 30 are also referred to as streaming data. The sensor data 30 comprise the captured video and sound so that the terminal devices 36, in particular users of the terminal devices 36, can access the sensor data 30. This means the respective terminal device 36 can receive and play back the sensor data 30 so that the video and sound captured by the sensor device 20 can be played back by the respective terminal device 36. For example, the respective terminal device 36 comprises a display 38 configured to show the video captured by the sensor device 20. Moreover, the respective terminal device 36 can comprise at least one speaker 40 configured to play the sound captured by the sensor device 20.

When the user 14 operates the operating element 24 again the streaming of the sensor data 30 to the server 28 as well as the broadcasting are ceased as a result of detecting that the operating element 24 of the vehicle 10 is operated by the user 14. In other words, the sensor data 30 are contents which can be consumed immediately by the users of the terminal devices 36 through, for example, on-demand subscriptions and/or streaming services such as a channel via which the streamed and shared sensor data 30 can be accessed.

For example, the respective terminal device 36 is a desktop computer, a tablet, a smart phone, a television device, a mobile device or another terminal device. By means of said method a hot content sharing, i.e. a live streaming of the sensor data 30 can be realized. For example, the streaming of the sensor data 30 is a live stream of short duration lasting, for example, about one minute in order to spare 3G or 4G monthly bandwidth. If, for example, the bandwidth is unlimited longer durations are possible. Starting the streaming and the broadcasting is particularly simple since the streaming and broadcasting can be started and stopped by operating the operating element 24. Thereby, respective experiences of the user 14 while driving along the road 12 can be shared with persons being said users of the terminal devices 36.

Said sharing of the sensor data 30 is not limited to live streams since, for example, the sensor data 30 can be recorded and saved in, for example, a memory device 41 of the server 28. Thus, the sensor data 30 can be accessed and uploaded at a later time after the sensor data 30 had been provided by the sensor device 20.

Preferably, the server 28 is configured to, when receiving the streamed sensor data 30, transmit at least one respective notification message to the terminal devices 36, wherein the notification message is indicative of the begin of the streaming. Thus, the users of the terminal devices 36 can be made aware of the beginning of the streaming so that the users of the terminal devices 36 can consume the live stream immediately. Thus, targeted users can choose which platform to view the live stream on after receiving. In said method a one-way-interaction (user 14 to audience) is realized since the user 14 can share their experience with the users of the terminal devices 36.

Fig. 2 shows a graphical user interface (GUI) of a platform via which the users of the terminal devices 36 can access the sensor data 30 by means of the terminal devices 36. For example, said platform is a digital platform and/or configured to provide a chat room for said users of the terminal devices 36. As can be seen from Fig. 2, the graphical user interface comprises a virtual map 42 showing available contents 44. For example, the graphical user interface is displayed on the display 38 so that the user of the respective terminal device 36 can view the graphical user interface by means of the display 38. For example, the respective user of the respective terminal device 36 can choose between the available contents 44 by, for example, clicking on respective icons 46 assigned to the respective available contents 44. For example, the respective content 44 comprises a live stream and, thus, sensor data that are currently streamed and shared by a user of a respective vehicle. Alternatively or additionally the respective content 44 can comprise recorded sensor data stored in the memory device 41. Thus, the respective user of the respective terminal device 36 can access the respective sensor data via the graphical user interface shown in Fig. 2. Thereby, the respective user of the respective terminal device 36 can browse the contents 44 and thus respective live streams and/or recorded and stored sensor data provided by respective vehicles.

Preferably, broadcaster options allow the user 14 to specify whether the sensor data 30, in particular the live stream, is public or private. Public live streams can be viewed by anyone connected to the platform. Default live streams shown in the graphical user interface are the current featured ones and the respective user can browse others by selecting different markers and/or icons on the map 42. This means the sensor data are broadcast in dependency on the sensor data 30 being classified to be public or private, so that, for example, private live streams or sensor data can be viewed by selected people only. Moreover, the users of the terminal devices 36 can experience browsing saved sensor data and, thus, saved streams. Past streams will be saved to allow for viewing them at any time after the live stream is over. Just like a public live stream, a publicly saved stream can be viewed by anyone. Default saved streams shown on the map 42 are the current public ones and user can browse others by selecting different markers on the map 42, wherein said markers can comprise the icons 46. In other words, by selecting the respective icons 46 the users of the terminal devices 36 can view respective live streams and/or recorded and stored sensor data. Furthermore, Fig. 3a and 3b show a flow diagram illustrating said method. This means Fig. 3a shows a first part of the flow diagram, and Fig. 3b shows a second part of the flow diagram, wherein the first and second parts form the flow diagram as a whole.

Moreover, the server 28 is used to realize a method for providing sensor data provided by respective sensor devices of respective vehicles. This means the respective vehicles can stream or transmit the respective sensor data such as the sensor data 30 to the server 28 which receives the respective sensor data from the respective vehicles. As described with respect to the vehicle 10, the sensor device 20 and the sensor data 30, the respective sensor data received by the server 28 comprise respective sounds, images and/or videos captured by the respective sensor device of the respective vehicle. This means said method for providing the sensor data comprises a first step of, by the server 28, receiving the sensor data provided by the sensor devices. In a second step of said method, by the server 28, the received sensor data are provided via the computer network 34 to which the server 28 is connected thereby making the received sensor data available for the terminal devices 36 connected to the computer network 34. By the server 28, the graphical user interface to be displayed on the terminal devices 36, in particular the displays 38, is provided such that the user of the terminal devices 36 can select the respective contents 44, i.e. the provided sensor data via the graphical user interface 42.

The idea behind the method for providing the sensor data is as follows: the upcoming generation is going to be used to consuming content whenever and wherever they are. A high percentage of content consumed will be of content users follow. The rest will be of content the users discover. These keywords are the ones that drive current popular content consuming platforms. Social media content consuming is very passive. Users choose when they want to consume the content that people they are following produce, contents such as status updates, videos, photos, shared links, etc. There is no need of having to consume content as it is produced because no one will miss the content since the content will, most of the time, be available for users to consume the content when the users can/want to consume it.

Live streaming content, however, is the polar opposite. Live streaming content will be missed if it is not consumed before the producer ends it. The appeal of consuming live content is that live content leaves its consumers feeling superior to people who did not consume live streaming content when it actually happened live. Having that feeling of superiority and the fear of missing out on something is why consumers come back to live content of their favorite producers. Nowadays, consumers are able to know the exact moment a producer goes live via a notification or notification message. For example, the notification or said notification message is an email, SMS and/or notification on a smart phone. The content of live streaming content and monetizing off of it seems niche but big companies see the popularity.

A further idea behind said method for providing sensor data provided by sensor devices of vehicles is to create a platform such as a digital or social network platform, said platform being a virtual space where users can consume vehicle sensor data and an end point to upload vehicle sensor data to. Not everyone who visits the platform needs to create an account and become a user. However, becoming a user allows them to follow other users and be notified when other users go live, participate in chat rooms, and have their own end point where they can stream live vehicle sensor data to.

For example, upon creating an account, the user must answer a set of question in case they ever choose to use the platform as an end point for live streaming of vehicle sensor data. A first one of said questions is, for example. do you want your live vehicle sensor data to be public or private? In the following, public sensor data are described: all of the user's followers will be notified when the user starts streaming data to the platform, and any user of the platform can view the user's channel while they are streaming data to the platform. In the following, private sensor data will be described: none of the user's followers will be notified when the user starts streaming data to the platform, none of the users of the platform can view the user's channel while they are streaming data to the platform, and a select list of users can be set up to be notified when the user starts streaming data to the platform. For example, the selected users can be loved ones and, or other people who can view the channel and, thus, have access to the sensor data only.

The user can now browse the platform and view its features such as: live leaderboard of fastest country, live leaderboard of slowest country, live visualization of world's g-forces, map of current users publicly live streaming their sensor data, and map of the most popular publicly shared sensor data. If the user selects a publicly shared live stream of the sensor data that includes cabin cameras and microphones they would then be able to see what that user is seeing and hear what the user is saying. However, it is important to note that the interaction is only one way: vehicle-to-audience. This means that the occupants of the vehicle 10 cannot interact with the users browsing the platform. The users browsing the platform have a chat room available to them to speak or chat amongst themselves but the occupants of the vehicle 10 cannot view the chat room. It is possible that a user is live streaming vehicle sensor data that does not include the cameras and microphones. This experience would just be a visualization of the data that is being shared. For example: if accelerometers, GPS (global position system) and speed is being shared, a visualization of just that will be shown and could suffice for the purposes using the platform. The person streaming to the platform might just want to notify a loved one that they are in the vehicle and on the way home.

list of reference signs vehicle 12 road 14 user 16 body 18 interior sensor device 22 surroundings 24 operating element 26 steering wheel 28 server sensor data 32 lifestream 34 computer network 36 terminal device 38 display speaker 41 memory 42 map 44 content 46 icon

Claims

Claims 1. A method for providing sensor data (30) provided by respective sensor devices (20) of respective vehicles (10), the method comprising: by at least one server (28): receiving the sensor data (30) provided by the sensor devices (20); and by the server (28): providing the received sensor data (30) via at least one computer network (34) to which the server (28) is connected thereby making the received sensor data (30) available for terminal devices (36) connected to the computer network (34).
2. The method according to claim 1, wherein, by the server (28), a graphical user interface (42) to be displayed on the terminal devices (36) is provided such that users of the terminal devices (36) can select the provided sensor data (30) via the graphical user interface (42).
3. The method according to claim 1 or 2, wherein, by the server (28), the sensor data (30) are broadcast as a live stream (32) to the terminal devices (36) via the computer network (34).
4. The method according to any one of the preceding claims, wherein the sensor data (30) are recorded and stored in a memory device (41) of the server (28).
5. The method according to any one of the preceding claims, wherein the sensor data (30) are made available in dependency on the sensor data (30) being classified to be public or private.