DE102020107998A1 - Method, system, computer program and storage medium for operating a loudspeaker system for a vehicle - Google Patents

Abstract

A method, system, computer program and storage medium for operating a loudspeaker system (12a, 12b, 12c) for a vehicle are specified. The vehicle (10) comprises an optical system (11) for capturing image data which are representative of occupants of the vehicle (10). In the method, a desired rest state of an occupant of the vehicle (10) is determined as a function of the image data, which is representative of a presumption that the occupant currently wants a quiet environment, and in the event that it has been determined that the occupant is is in the desired state of rest, a volume of the loudspeaker system (12a, 12b, 12c) that can be measured at a head position of the occupant is reduced.

Description

The invention relates to a method and system for operating a loudspeaker system for a vehicle. Furthermore, a corresponding computer program and computer-readable storage medium are specified.

One object on which the invention is based is to specify a method and a corresponding system for operating a loudspeaker system for a vehicle which contributes to comfortable use of the vehicle and safe operation of the vehicle.

The object is achieved by the independent patent claims. Advantageous refinements are characterized in the subclaims.

According to a first aspect, the invention relates to a method for operating a loudspeaker system for a vehicle.

The vehicle includes an optical system for capturing image data representative of occupants of the vehicle. In the method, a desired rest state of an occupant of the vehicle is determined as a function of the image data. The occupant's desire to rest is representative of a presumption that the occupant currently wants a quiet environment. In the event that it has been determined that the occupant is in the desired resting state, a volume of the loudspeaker system that can be measured at a head position of the occupant is reduced.

For example, image analysis algorithms can be used to identify properties of people such as closed or open eyelids, moving or immobile limbs, moving or immobile lips, etc. Advantageously, the proposed method can be used to automatically adjust the volume within the vehicle upon successful detection of a sleeping or talking person, thus helping to prevent the person in question from being disturbed by inadequate volume or that a driver of the Vehicle would have to make a manual volume adjustment that distracts from the traffic situation.

A volume of the loudspeaker system that can be measured at a head position of the occupant is understood to mean an amplitude of a sound pressure level generated by the loudspeaker system, which could be measured at the head position of the occupant as a reference location. The head position can be determined using image analysis algorithms, which is particularly useful in vehicles with a large number of possible locations and positions. In today's motor vehicles, the location of the occupants and their position is severely restricted by the seat configuration, so that an estimate of the head position is preferably carried out or stored here.

In an advantageous embodiment according to the first aspect, the optical system is designed as an interior camera.

In an advantageous embodiment according to the first aspect, the loudspeaker system has at least three loudspeaker units. At least one of the loudspeaker units is set up to provide sound to a front area of the interior of the vehicle. At least one of the loudspeaker units is set up to provide sound to a rear area of the interior of the vehicle. At least one of the loudspeaker units is set up to provide sound to a left area of the interior of the vehicle. At least one of the loudspeaker units is set up to provide sound for a right area of the interior of the vehicle.

The volume of the loudspeaker system that can be measured at the head position of the occupant is reduced by adjusting a balance between the at least one loudspeaker unit for sounding the front area and the at least one loudspeaker unit for sounding the rear area. Alternatively or additionally, the volume of the loudspeaker system that can be measured at the occupant's head position is reduced by adjusting a balance between the at least one loudspeaker unit for sounding the left area and the at least one loudspeaker unit for sounding the right area.

A loudspeaker unit set up to cover an area is understood here and below to mean that an amplitude of a sound pressure level generated by the corresponding loudspeaker unit is higher in the area during normal operation than in other areas of the interior of the vehicle. For this purpose, the corresponding loudspeaker unit is arranged, for example, in the area and / or directed towards this area.

• - einen Schlafzustand, in dem der Insasse des Fahrzeugs als schlafend erkannt wird;
• - einen ersten Gesprächszustand, in dem der Insasse des Fahrzeugs als im Gespräch befindlich mit einer Person unmittelbar außerhalb des Fahrzeugs erkannt wird;
• - einen zweiten Gesprächszustand, in dem der Insasse des Fahrzeugs als im Telefongespräch befindlich erkannt wird;
• - einen dritten Gesprächszustand, in dem der Insasse des Fahrzeugs als im Gespräch befindlich mit einem weiteren Insassen des Fahrzeugs erkannt wird.

In an advantageous embodiment according to the first aspect, the desired rest state of the at least one occupant of the vehicle comprises at least one of the following states:

• a sleep state in which the occupant of the vehicle is recognized as sleeping;
• - a first conversation state in which the occupant of the vehicle as in conversation located with a person immediately outside the vehicle is recognized;
• - A second conversation state in which the occupant of the vehicle is recognized as being on the phone;
• a third conversation state in which the occupant of the vehicle is recognized as being in conversation with another occupant of the vehicle.

In an advantageous embodiment according to the first aspect, an eyelid condition of the occupant and / or limb activity of the occupant and / or oral activity of the occupant is or are monitored by means of the optical system.

In the event that the eyelid condition is representative of the fact that the occupant's eyes are closed beyond a specified period of time and / or that the occupant's limb activity and / or the occupant's oral activity falls below or falls below a specified activity within the specified period of time the occupant is assigned the sleeping state.

For example, a duration of 5 minutes or more can be considered as a predetermined time period.

In an advantageous embodiment according to the first aspect, oral activity of the occupant and / or a viewing direction of the occupant is or are monitored by means of the optical system and / or an opening state of a window or door of the vehicle is monitored by means of a window or door sensor, which is representative of the fact that the window or door is open.

In the event that the occupant's oral activity exceeds a specified activity within a specified period of time and / or the occupant's line of sight is directed towards the window or door of the vehicle and / or the window or door of the vehicle is in the open state , the first call status is assigned to the occupant.

A period of 10 seconds or more can be considered here as a predefined time span, for example.

In an advantageous embodiment according to the first aspect, oral activity of the occupant and / or use of a telephone, headset or headphones by the occupant is or are monitored by means of the optical system.

In the event that oral activity exceeds a predefined activity within a predefined period of time and / or the occupant has determined that a telephone, headset or earphone is being used, the occupant is assigned the second conversation state.

A period of 10 seconds or more can be considered here as a predefined time span, for example.

The use of a telephone is understood to mean, for example, that the telephone is held in one hand of the occupant and / or is located on the occupant's ear. The use of the headset or headphones is understood to mean, for example, that the headset or the headphones are arranged on the head of the occupant.

In this context, the products of Jungo Connectivity Ltd. are merely an example. ("CoDriver for Car OEMs and Tier-1s") and Sensetime ("Driver Analysis System"), which implement optical recognition of smartphones.

In an advantageous embodiment according to the first aspect, oral activity of the occupant and / or a line of sight of the occupant is or are monitored by means of the optical system and an oral activity of a further occupant and / or a line of sight of the further occupant is monitored by means of the optical system.

In the event that the oral activity of both occupants exceeds a specified activity within a specified period of time and / or the direction of gaze of both occupants is facing each other, the third conversation state is assigned to both occupants.

In this context, a seating position of the two occupants can also be recorded, for example. In the event that the two occupants are both sitting in the same row of seats in the vehicle, loudspeaker units of the loudspeaker system can be turned down to provide sound for the entire row of seats.

According to a second aspect, the invention relates to a system for operating a loudspeaker system for a vehicle.

The vehicle includes an optical system for capturing image data representative of an occupant of the vehicle. The system is set up to carry out the method according to the first aspect.

According to a third aspect, the invention relates to a computer program comprising commands that are used when the computer program is executed cause the computer to carry out the method according to the first aspect.

According to a fourth aspect, the invention relates to a computer-readable storage medium on which the computer program according to the third aspect is stored.

Embodiments of the invention are explained in more detail below with reference to the schematic drawings.

• 1 ein Ausführungsbeispiel eines Systems zum Betreiben eines Lautsprechersystems für ein Fahrzeug,
• 2 ein Ablaufdiagramm zum Betreiben des Lautsprechersystems, und
• 3 eine beispielhafte Ausgestaltung des Schritts S3 des Ablaufdiagramms gemäß 2.

Show it:

• 1 an embodiment of a system for operating a loudspeaker system for a vehicle,
• 2 a flow chart for operating the speaker system, and
• 3 an exemplary embodiment of the step S3 of the flow chart according to 2 .

Elements of the same construction or function are provided with the same reference symbols in all the figures.

The volume of a loudspeaker system for a vehicle can be adjusted when media or voice announcements are played back, for example, as a function of the speed of the vehicle. The focus here is primarily on fading out ambient or rolling noises that come to the fore at high speeds. In other words, the loudspeaker system is adapted in such a way that a higher volume is selected at high speed.

In the event that rear passengers fall asleep, a volume of the loudspeakers assigned to the rear rows of seats currently has to be adjusted manually. However, this distracts the driver from the traffic situation. In the following, a method is described which takes into account situations in which an automatic reduction in volume is considered useful. If it is detected that passengers are (falling) asleep or are having a conversation, for example through a side window of the vehicle, the volume is automatically adjusted.

• - der Passagier schläft,
• - der Fahrer durch das Seitenfenster mit jemandem spricht,
• - ein Baby einschläft,
• - eine Türe geöffnet wird, etc.

Using optical systems such as an interior camera and optional additional sensors (e.g. door contact, window lifters), different activities / states of passengers in the interior of a vehicle can be recognized. Image analysis algorithms are also able to recognize characteristics of people such as age, gender, ethnicity, etc. For example, activities / states can be assigned to individual passengers that describe, for example, that

• - the passenger sleeps,
• - the driver is talking to someone through the side window,
• - a baby falls asleep,
• - a door is opened, etc.

On the basis of the detected status / activity, the overall volume can then be reduced or minimized either in the entire interior of the vehicle or for individual seats. With the interior camera it can be recognized whether a passenger is sleeping and on which seat they are sitting. On the basis of this information, the measurable volume on the seat concerned can then be reduced, for example by means of balance control and / or volume control.

1 Figure 3 shows an embodiment of a system 1 for operating a loudspeaker system for a (motor) vehicle 10.

The system 1 includes a camera 11 that are in an interior of the vehicle 10 is arranged and designed to capture image data that are representative of passengers of the vehicle 10 , as well as a speaker system with a front speaker 12a as well as a left rear loudspeaker 12b and a right rear speaker 12c . In addition, the system 1 more in the vehicle 10 arranged sensors (not shown here), which are set up to provide sensor data that are each representative of a state / activity of a detected passenger.

The system 1 is also one with the camera 11 , the speakers 12a-12c and, if applicable, the control unit coupled to the optional additional sensors 13th assigned. In a data and program memory of the control unit 13th is a program for operating the speaker system according to 1 saved, which is based on the flowchart of the 2 is explained in more detail.

The program starts in one step S1 , in which, for example, variables are initialized. The program is in one step S2 continued in the image data of the indoor camera 11 be determined. In a subsequent step S3 a desired rest state is determined as a function of the image data for at least one passenger, in particular for each passenger, which is representative of a presumption as to whether the corresponding passenger currently wants a quiet environment. Optical recognition of a passenger's condition in the passenger compartment is therefore fundamental. If none of the passengers have a wish-to-rest state was determined, the program is exemplified in step S2 continued or the program ended. Otherwise the program will run in one step S4 continued. Details of the step S3 are based on 3 explained.

In the step S4 the loudspeaker system is controlled in such a way that the measurable volume is reduced at a presumed or determined head position of the corresponding passenger determined to be in the desired resting state. For this purpose, a seat assigned to the corresponding passenger is determined by way of example, and a loudspeaker assigned to the seat is determined 12a-12c deactivated or turned down. Based on the exemplary vehicle 10 according to 1 could in the event that the identified passenger has a seat in the front right of the vehicle 10 occupied, the front speaker 12a and / or the right rear speaker 12c can be turned down while the left rear loudspeaker 12b continues to operate unchanged.

In other words, an (automatic) volume control takes place upon successful detection of a passenger in the desired idle state. The regulation can also be referred to as “balance” and / or “fader” adjustment. Alternatively, it is also conceivable to reduce the overall volume of the loudspeaker system. The program is then ended in a step S5.

3 shows a possible embodiment for determining the desired rest state of the passengers of the vehicle 10 according to step S3 the 2 .

In a step S31, the image data are used to determine an eyelid condition, a line of sight, a limb activity and an oral activity of a first passenger of the vehicle 10 determined. In addition, the image data are used to determine whether the first passenger is currently using a telephone, a headset or headphones. In addition, for example, a window or door sensor is used to determine an open state of a window or a door of the vehicle 10 determines which is representative of whether the window or the door is open.

In a subsequent step S32, it is checked whether the eyelid condition is representative of the fact that the eyes of the first passenger are closed for a predetermined period of time. In this case, the program is continued in a step S33. Otherwise, the program is continued in a step S35.

In step S33 it is checked whether the limb activity of the first passenger falls below a predefined activity within a predefined period of time. In this case, the program is continued in a step S34. Otherwise the program in step S2 continued.

In step S34 it is checked whether the oral activity of the first passenger falls below a predefined activity within a predefined period of time. In this case, the first passenger is assigned a sleep state which is representative of a desired rest state in which the first passenger is recognized as sleeping, and the program in step S4 continued. Otherwise the program in step S2 continued.

In step S35 it is checked whether the oral activity of the first passenger exceeds a predefined activity within a predefined period of time. In this case, the program is continued in a step S36. Otherwise the program in step S2 continued.

In step S36 it is checked whether the line of sight of the first passenger is towards a window or a door of the vehicle 10 is directed. In this case, the program is continued in a step S37. Otherwise, the program is continued in step S38.

In step S37 it is checked whether the corresponding window or the corresponding door is open. In this case, a first conversation state is assigned to the first passenger, which is representative of a desired idle state in which the first passenger is in conversation with a person immediately outside the vehicle 10 is recognized, and the program in the step S4 continued. Otherwise the program in step S2 continued.

In step S38 it is checked whether the first passenger is using a telephone, a headset or headphones. In this case, the first passenger is assigned a third call status, which is representative of a desired idle status in which the passenger is recognized as being on the phone, and the program in step S4 continued. Otherwise, the program is continued in a step S39.

In step S39, for example, the image data are used to check whether there are other passengers in the interior of the vehicle 10 are located. In this case, the program is carried out repeatedly for each additional passenger in step S31 (not shown in greater detail for reasons of clarity).

In the event that there are no other passengers in the interior of the vehicle 10 is located, for which eyelid state, direction of view, etc. have not yet been determined, a check is made in step S39 as to whether the direction of view of two passengers coincides in such a way that the two passengers are facing each other. In this case, the two passengers are each assigned a third call state, which is representative of a desired idle state in which the two passengers of the vehicle 10 are recognized as being in conversation, and the program in the step S4 continued. Otherwise the program in step S2 continued.

List of reference symbols

1

system

10

vehicle

11

Indoor camera

12a-12c

speaker

13th

Control unit

S1-S4

Program steps

Claims (11)

Method for operating a loudspeaker system (12a, 12b, 12c) for a vehicle (10), wherein the vehicle (10) comprises an optical system (11) for capturing image data which are representative of occupants of the vehicle (10), wherein - As a function of the image data, a desired rest state of an occupant of the vehicle (10) is determined, which is representative of a presumption that the occupant currently wants a quiet environment, and - In the event that it has been determined that the occupant is in the desired resting state, a volume of the loudspeaker system (12a, 12b, 12c) that can be measured at a head position of the occupant is reduced. Procedure according to Claim 1 , wherein the optical system (11) is designed as an indoor camera. Method according to one of the preceding claims, wherein the loudspeaker system has at least three loudspeaker units (12a, 12b, 12c), at least one of the loudspeaker units (12a) being set up to cover a front area of the interior of the vehicle (10), at least one of the loudspeaker units ( 12b, 12c) is set up for sounding a rear area of the interior of the vehicle (10), at least one of the loudspeaker units (12b) is set up for sounding a left area of the interior of the vehicle (10) and at least one of the speaker units (12c) is set up for sounding of a right area of the interior of the vehicle (10) is set up, and the volume of the loudspeaker system, which can be measured at the head position of the occupant, by adjusting a balance between the at least one loudspeaker unit (12a) for sounding the front area and the at least one loudspeaker unit (12b, 12c ) for sound reinforcement of the hi nter area and / or by adapting a balance between the at least one loudspeaker unit (12b) for sounding the left area and the at least one loudspeaker unit (12c) for sounding the right area is reduced. Method according to one of the preceding claims, wherein the occupant's desire to rest comprises at least one of the following states: a sleep state in which the occupant of the vehicle (10) is recognized as sleeping; a first conversation state in which the occupant of the vehicle (10) is recognized as being in conversation with a person immediately outside the vehicle (10); a second conversation state in which the occupant of the vehicle (10) is recognized as being on the phone; a third conversation state in which the occupant of the vehicle (10) is recognized as being in conversation with another occupant of the vehicle (10). Procedure according to Claim 4 , an eyelid condition of the occupant and / or a limb activity of the occupant and / or an oral activity of the occupant being monitored by means of the optical system (11), and in the event that the eyelid state is representative of the occupant's eyes being over a predetermined period of time are closed and / or the limb activity of the occupant and / or the oral activity of the occupant falls below or falls below a respectively specified activity within the specified time period, the occupant is assigned the sleep state. Method according to one of the Claims 4 or 5 , wherein the occupant's oral activity and / or a line of sight of the occupant is or are monitored by means of the optical system (11) and / or an open state of a window or door of the vehicle (10) is monitored by means of a window or door sensor , which is representative of the fact that the window or the door is open, and in the event that the oral activity of the occupant exceeds a predetermined activity within a predetermined period of time and / or the direction of view of the occupant towards the window or the door of the vehicle (10) is directed and / or the window or the door of the vehicle (10) is in the open state, the occupant is assigned the first conversation state. Method according to one of the Claims 4 until 6th , wherein by means of the optical system (11) an oral activity of the occupant and / or use of a telephone, headset or headphones by the Occupant is or are monitored, and in the event that the oral activity exceeds a specified activity within a specified period of time and / or the occupant has been determined to use a telephone, headset or headphones, the occupant is assigned the second conversation state. Method according to one of the Claims 4 until 7th , wherein an oral activity of the occupant and / or a line of sight of the occupant is or are monitored by means of the optical system (11) and an oral activity of another occupant and / or a line of sight of another occupant is or are monitored by means of the optical system (11). and in the event that the oral activity of both occupants exceeds a specified activity within a specified period of time and / or the direction of gaze of both occupants is facing each other, the third conversation state is assigned to both occupants. System for operating a loudspeaker system (12a, 12b, 12c) for a vehicle (10) with an optical system (11) for acquiring image data which are representative of an occupant of the vehicle (10), the system being set up, the method to be carried out according to one of the preceding claims. Computer program comprising instructions which cause the computer program to be executed by a computer, the method according to one of the Claims 1 until 8th to execute. Computer-readable storage medium on which the computer program is based Claim 10 is stored.

Images