WO2023070142A1

WO2023070142A1 - Portable digital audio device for capturing user interactions

Info

Publication number: WO2023070142A1
Application number: PCT/AT2022/060372
Authority: WO
Inventors: Ulf Hakan Benny Sällberg; Mikhael BEIM
Original assignee: Birdkids Gmbh
Priority date: 2021-10-28
Filing date: 2022-10-27
Publication date: 2023-05-04
Also published as: AT525698A1

Abstract

The invention relates to a portable audio device (1) for converting user interactions (3) that can be captured in the form of user input signal(s) (2) into a digital data stream for conversion into an acoustic signal (5) and/or a control signal (6) for controlling an acoustic signal (5) at a later stage, the audio device (1) being designed for conversion while taking into account specifications that can be configured according to a configuration data set (7).

Description

P22952pct

PORTABLE DIGITAL AUDIO DEVICE TO CAPTURE USER INTERACTIONS

The invention relates to a portable digital audio device for detecting user interactions, which can be used for later conversion into an acoustic signal and/or a control signal for controlling an acoustic signal.

The following examples from the patent literature are presented:

US2016124559A1 relates to a device comprising an array of touch zones, each touch zone capable of measuring the compressive force exerted by a user and its variation over time in three axes (X,Y positions, Z for pressure), each independently capture. The device is able to calculate and generate initiation, continuation and release events per touch zone and subsequent changes per axis (X,Y,Z) corresponding to the moment a user starts interacting with a Touch zone begins, changes interaction, or ends interaction at that zone. For the initial touch event, the device generates a MIDI note-on (=note on) with a note velocity value consisting of the pressing force and the note number as configured for this zone, a MIDI note- Off is generated when the touch zone is released, and for the continuation event, one control change signal can be assigned per axis change-over-time, mostly MIDI parameter changes over time of class loudness, pitch , and timbre which are musically processed outside of the device. This may at first glance appear very similar to the event generator according to the present invention, which also intends to generate engage, change and release events depending on the state of user interaction with a sensor. Depending on the choice of target application, the detection can be reduced to two axes per touch zone. Furthermore, US2016124559A1 describes at least one method for optimizing the calculations of simultaneous events and their prioritization in processing, but only within the framework of the specific hardware equipment and its requirements. A closer look, however, shows that the present invention has some distinguishing features and thus represents significant advantages over this prior art: - The method with which the events "Engage (Activate)", "Change (Modify)" and "Release (Release)" are calculated can be recorded in the present invention in a granular way by the user - and differentiated for further processing released, or partially omitted. Therefore, the basic behavior of a device according to the invention (when events are generated in response to user input and consequently the rate of events per second) is user controllable. This allows the user to find a trade-off between battery/power consumption and event granularity.

- In contrast to the prior art mentioned, in which a fixed arrangement of touch sensors (force sensitive) is intended, our event generator is directly suitable for processing any sampled/digital one- or multi-dimensional sensor signal and not just force-measuring sensors. Any sensors such as gyroscope or acceleration sensors can be used to record user interaction.

The provision of configurable event generation in the present invention reduces the processing power required to process sensor data and reduces the bandwidth required to transmit control data since data is only generated in response to the generated events.

- The assignment of the generated events is user-configurable in the present invention. This means that a user is not limited to assigning a MIDI note with a velocity of e.g. B. corresponds to a compressive force. A user can configure a sensor's Engage event to output any other effect or control that affects sound generation in an external instrument.

- The present invention also allows the user to map many output generators (processing elements) onto the same generated event. So pressing one key can lead to many output actions.

- The triggering of events by our event generator can be further mapped by our sequencer to certain processing elements that a device behavior that goes beyond the simple sending of control data.

US9076419B2 includes a configurable control unit consisting of several sensors and different sensor types. The device uses the signals from sensors to interpret user interactions and generate control data. The device is user configurable using external software, allowing the user to assign what output actions are possible for each input sensor. The user can also control the sensitivity of each sensor, e.g. B. to control at which pressure force the output should be generated. The device described in this document is intended exclusively for the output of control signals within the framework of MIDI. Here are a few points where the present invention offers advantages over this prior art:

- The prior art is limited to mapping input sensor events to MIDI output actions. It is not possible to configure the device to perform any processing in response to an event. The present invention allows the user to map any input event to any processing element, and since the processing elements in the present invention can be more freely configured, the output actions of the present invention are not limited to generating simple MIDI outputs, but can also programmed to provide, for example, MIDI polyphonic expression (MPE) functionality.

- Since the prior art mentioned is limited to sending out only basic MIDI commands, it requires host software on a PC in order to connect digital audio workstation programs. It is not possible to configure the device of the patent as an independent interface to digital audio workstations, nor to control contemporary wireless devices and applications via, for example, MIDI-over-Bluetooth. This is a major advantage of the present invention that the output actions in response to events are encapsulated by freely programmable processing elements and thereby can be adapted to specific communication formats required by digital audio workstations. - The patent is limited to a device that runs on an external power source. The present invention should be portable and battery powered.

- The patent is limited to touch and force sensors, while the event generation according to the present invention can process any one-dimensional sensor signal, including e.g. B. Gyroscope and accelerometer data.

- The prior art allows the user to control the sensitivity of the sensors using one value for both Engage/Release/Change events, whereas our invention explicitly allows the user to set the Engage, Release and Engage To control thresholds separately, with the advantage that a user can separately control the required actuation force (Engage Threshold) of a force sensor and the required change force (Change Threshhold) for the output of change events.

- The present invention allows priority execution of assigned events to processing elements (which may be configured to generate MIDI or other form of control signals for a standard protocol such as OSC), therefore the present invention ensures a certain quality of service for mappings (Assignments) with high priority over low-priority mappings, so that e.g. B. Certain functions are guaranteed low latency compared to other functions. This is particularly relevant in connection with wireless transmission methods with a limited bandwidth, such as e.g. B. bluetooth low energy (BLE), an advantage.

US2014214189 relates to a DJ radio controller consisting of an accelerometer, a control button and a radio transmitter. The DJ radio controller detects the depression of the control button and gets an output value from the accelerometer. Based on this output value, the DJ radio control generates a control signal and transmits this control signal to influence an external audio signal. The DJ radio controller offers a fairly simple user interface, consisting of buttons that when pressed initiate a transmission of accelerometer data. A limiting factor of a device according to US2014214189A1 is the inflexible and restrictive use of the existing sensors. It is not possible for the user to greatly change the settings of the US2014214189A1 series device compared to the present invention. only the predefined, limiting interaction properties can be changed and subsequently assigned to external audio devices.

US2020098343A1 relates to a human interface device including a processor with memory. A plurality of switch banks, also coupled to the processor, a rotary selector and a slider, both coupled to the control circuitry such that the control circuitry is able to communicate the position of the rotary selector and slider to the processor. The rotary selector and the slider are placed on the interface device to form a connection with the switch bank, a converter coupled to the processor, said converter containing a number of LED sources spatially positioned on a position ring, a series of non-transitory processor instructions ready for the processor rules to provide MIDI commands through a BLE port. Although the device according to US2020098343A1 offers advantages such as visual feedback through the LED sources positioned on the positioning ring, the device is limited in the setting options for artistic expression outside of visual design (the inventor's intention is to control three-dimensional operations in VR, A/R applications ), namely to satisfy the real-time processing and mapping needs of interactions in musical performances. Conversely, the present invention is very well suited to controlling visual applications and manipulations in virtual 3D spaces (A/R, VR) in a variety of ways, since the processing is subject neither to type-specific input sensors nor to a rigid allocation methodology.

US2019251937A1 describes a fingerless digital music effects control glove consisting of a glove-mounted device with a cable sensor to measure movement of the user's wrist, an accelerometer, and a flexible pressure sensor or switch on the glove's palm. The device transmits data wirelessly to digital audio software running on an external computer, laptop or smartphone, allowing the user to manipulate various parameters of the resulting sound or piece of music. The fingerless digital music effects control glove is configured to be recognized by the external computer as a MIDI or audio device, making it compatible with existing digital audio software. The glove is lightweight, flexible and fingerless, allowing unrestricted movement of the user's fingers while playing a musical instrument or other musical device. A disadvantage of this control glove according to US2019251937A1 is its limited differentiation of possible events when capturing user interactions, their lack of processing and limited assignment of parameters. This means that there is only very limited configurability.

US2010022183A1 describes a system and method for real-time radio transmission of a digital audio signal and control data. A transmitter can be linked to the audio source. The transmitter includes a processor to combine the control data with the digital audio signal from the audio source, and the processor transmits the combined control data and audio signal over the air to a receiver. The receiver includes a processor which is used to receive the combined control data and audio signals and to process the control data to perform a user defined function. A significant disadvantage of a device according to US2010022183A1 is that it is not a radio control device for external devices for creating artistic expressions, but rather an independent real-time processing of sound in which the receiving unit depends on the sensor data provided by the transmitting unit.

WO2006133207A2 relates to an audio signal controller for controlling and a method for controlling the application of special audio effects to an audio signal, where the audio effect controller includes a sensor that detects motion and associates the detected motion with the formation of audio signals by using the sensor Control signal generated in response to the detected movement and this control signal is forwarded to an audio effects unit for controlling the application of an audio effect to an audio signal. The use of orientation sensors serves as an example where an orientation sensor is attached to an instrument to provide orientation sensor data to the effects unit that modifies the instrument's acoustic expression, for example, tilting the guitar can be used to activate modulation effects on the guitar sound. A disadvantage of WO2006133207A2 is that it requires a device that is limited to the use of orientation and motion sensors and that the possibility of reconfigurability is limited to simply mapping sensor events and sensor data to controllable musical input parameters of the instrument. US8686276B1 relates to techniques developed for recording and performing musical performances on portable devices. The techniques developed enable the recording, encoding and use of gestural data for the execution of a musical performance. In some implementations, a sign stream enables the audible performance of the musical performance to be encrypted locally at the portable device on which the musical performance is recorded, typically in real time. In other representations, a sign stream efficiently encodes the musical performance for transmission from the portable device on which the musical performance was recorded to (or to) an external device on which the musical performance is recorded (or can be). Indeed, in some representations, a gesture stream is recorded and encoded so that it can be recorded locally and on an external device using substantially identical or equivalent circumstances of digital synthesis of the musical instrument. Although US8686276B1 includes a device which, as a remote control device, changes music and sound on similar devices in the network of devices, this prior art does not provide for general control using, for example, MIDI or any other data protocol from external devices (which are not correspond to the device types in the patent).

US2018137847A1 includes a hardware segment that electronically captures the physical inputs of a musician playing his or her musical playing style or articulated intent; a software segment that interprets and formulates the data output from the hardware segment for application to any digitally modulated synthesizer or recording sampler; and an optional advanced method for sampling or digitally recording articulated acoustic instrument sounds. The device according to US2018137847A1 is limited in comparison to the present invention for general control of external digital applications in that the requirement of using a matrix of at least 16x16 mechanical sensors and a slight reconfigurability, which the prior art invention provides, affects the assignment from sensor events to output MIDI commands.

WO2012170344A2 describes how a wireless sensor network for musical instruments is provided, which enables a musician to use natural performance gestures (alignment, pressure, tilt, etc.) to a computer. User interfaces and computer modules are also provided that allow a user to utilize the data transmitted by the wireless sensor network to complement and/or augment artistic expression. A particularly advantageous feature of this prior art is the possibility of calibrating and defining a sensor data range and of supplying correctly assigned sensor data to external devices. Although advantageous in the formation of artistic expressions in external devices, WO2012170344A2 requires the use of at least two technically separate devices; a transmitting unit with a sensor and a receiving device and the configuration of this device is limited to the assignment of sensor output data, either pure sensor data or processed and generated sensor events, to musical input parameters, which represent severely limiting properties from WO2012170344A2.

The object on which the invention is based is to provide a portable digital audio device for detecting user interactions that can be used for later conversion into an acoustic signal and/or a control signal for controlling an acoustic signal. The audio device should be configurable by the user and have a higher degree of configurability than the existing devices and solutions already have to a certain extent. The freely designable configurability of the audio device makes the user's artistic work in creating sounds and music much easier. As a result, the audio device should have such an exceptionally freely designable configurability that the user will find previously unexplored ways of generating sounds and music. In addition, the necessary electronic energy consumption and the bandwidth required by the audio device during data transmission should be minimal in view of the configurability of the device.

The object of the invention is achieved by a portable digital audio device according to claim 1.

Further advantageous embodiments of the invention are specified in the dependent claims. The invention will be explained in more detail with reference to drawings showing exemplary and non-limiting embodiments.

Fig. 1 shows a schematic representation of a portable digital device according to the invention

Audio device in one embodiment.

Fig. 2 shows a schematic representation of a portable digital device according to the invention

Audio device in another embodiment.

Fig. 3: shows a schematic representation of a further embodiment according to

Invention.

4 shows a schematic representation of an exemplary detail of the invention, in which at least the number of pre-stored and linkable sound generation and/or processing operations and an input event list of potentially possible input events are stored in an internal data memory.

Fig. 5: shows the sequence of individual steps that can be carried out by the invention.

FIG. 6: shows an exemplary data flow diagram for an exemplary sensor

Invention.

Fig. 7: shows an embodiment of a force sensor for the detection of the

User's finger pressure force, which force sensor can be used in the present invention.

Fig. 8: shows and advantageous embodiment of a in the invention

Audio device located position sensor.

Fig. 9 shows an advantageous embodiment of a data transmission interface that can be used in the invention.

Fig. 10 shows an embodiment of a light source that can be used for a feedback function of the invention. Fig. 11: shows a top view of an embodiment of an audio device according to the invention, wherein a light source is located below a semi-transparent silicone push button.

12: shows a top view of the audio device, with a loudspeaker being integrated in the housing and emitting acoustic signals through housing recesses.

Unless otherwise stated, the same reference symbols denote the same features in the following figures.

1 shows a schematic representation of a portable digital audio device 1 according to the invention in one embodiment. The following is proposed for solving the task mentioned at the outset: a portable digital audio device 1 for converting user interactions 3 that can be detected as user input signal(s) 2 into a digital data stream 4 for subsequent conversion into an acoustic signal 5 and/or a control signal 6 for controlling an acoustic signal 5, the audio device 1 being set up to carry out the conversion taking into account specifications that can be configured in accordance with a configuration file 7 .

FIG. 2 shows a further preferred embodiment of the invention, with the control signal 6 of the audio device 1 being routed externally in order to control an external acoustic signal 5 . According to the exemplary illustration in FIG. 2, the invention also allows a combination of internal and external conversions of detectable user interactions 3 into acoustic signals 5 and/or control signals 6 for later conversion into an acoustic signal 5.

It is proposed - as shown in Fig. 3 for a preferred embodiment - that the audio device 1 comprises at least the following: a housing 8, a number of sensors 9 arranged in or on the housing, a first internal data memory 10 arranged in the housing 8 , a processing unit 11 arranged in or on the housing 8 with an arithmetic processor 12 and a processing memory 13 and a power source 14 for the electrical supply of the number of sensors 9, the internal data memory 10 and the processing unit 11.

According to the invention, it is additionally proposed that the number of sensors 9 arranged in or on the housing includes at least one input sensor for continuously detecting a user interaction 3 . The continuous detection of a user interaction 3 has the advantage that the user can generate acoustic signals 5 and/or control signals 6 for controlling an acoustic signal 5 in a more granular manner compared to, for example, the binary granularity that a digital on/off button offers. In order to elucidate the advantage of continuously capturing user interactions 3, the following example is used, in which it is assumed that one of the input sensors 9 is a directional sensor, and the audio device 1 is set up in such a way that the directional sensor outputs output data, which in this case represent the user input signal 2, a corresponding acoustic signal 5 can be influenced, e.g. the pitch of the acoustic signal. The user can thus carry the audio device 1 in his hands and continuously influence the pitch of the acoustic signal 5 by moving the audio device 1 . Alternatively or in addition to this, the audio device 1 can have a force sensor as an input sensor 9, and the audio device 1 is set up in such a way that the pressure exerted by the user on the force sensor, depending on the pressure intensity, is converted into a stronger or weaker acoustic signal 5 or control signal 6 for controlling a Acoustic signal 5 is converted, or continuous changes in the compressive force are converted into continuous changes in the acoustic signal or the control signal. This advantageous example explains the intuitive approach of how the user can interact with the audio device 1 in order to influence an acoustic signal.

According to the invention, a conversion of the user interactions 3 detected by the input sensors 9 as user input signals 2 into a digital data stream 4 with a resolution of at least 7 bits is proposed. This advantageously means that the further and later processing of the user input signals 2 can be implemented in digital form. The further and later processing, some of which takes place in the arithmetic processor 12 proposed according to the invention, can thus enjoy the well-known advantages of digital processing. An example of such an advantage of digital processing is that the functionality of software or configurable hardware can be changed by simple reprogramming or reconfiguration can be done. Typically, the proposed computing processor 12 can either execute predetermined software to implement the functionality proposed according to the invention, or the computing processor 12 can also be implemented from configurable hardware, e.g. by a Field Programmable Gate Array (FPGA), which is an example of such configurable hardware is. A combination of a plurality of software and configurable hardware units of the computing processor 12 can also be advantageous, where modern microcontrollers and digital processors typically consist of multiple computing cores or have configurable hardware peripheral units.

The conversion of input sensors 9 from recorded user interactions 3 into user input signals 2, which thereby become further digital signals that can be processed, can, as is generally known in the technical field and is also permissible with the present invention, either already be integrated in the input sensor 9 and the Input sensor 9 thereby already supply a digital user input signal 2, or - should the input sensor 9 not have such a possibility for such a conversion - the audio device (1) can have a conversion set up according to the conversion task in order to provide digital user input signals 2 therefrom. There is also the possibility that both input sensors 9 with integrated converters and input sensors 9 without integrated converters are present, in which case the audio device 1 has a conversion of the analog input signals from input sensors 9 without integrated converters into digital user input signals 2.

The invention proposes in particular, as also shown in FIG. 4, that at least a number of pre-stored and linkable sound generation and/or processing operations 15 and an input event list 16 of potentially possible input events 17 are stored on the internal data memory 10, wherein according to the invention, for each potentially possible input event 17 at least one link 18 to at least one sound generation and/or "operation" 15 can be defined by the user of the audio device 1, with each link 18 being assigned a priority value 19 in accordance with a configuration file 7 stored on the audio device assigned. In other words, potentially possible input events 17 are linked to “operations” (15) in a selectable manner by the user, and the link is provided with a priority value 19 . The user has the choice due to the freedom of configuration assign no, one, or even several links to potentially possible input events 17 and provide them with a priority value 19 . As will be described in more detail in the next section, a certain user interaction 3 can be correspondingly assigned to an input event 17 . For example, the audio device 1 can be set up such that when the user exerts a force sensor with a force above a definable force threshold, one of the potential possible input events 17 corresponding to the event is activated. The links 18 associated with this activated input event 17 are thus also activated and the “operations” 15 defined accordingly for these activated links 18 are processed in the arithmetic processor 12 . This ensures that the input event list 16 that can be linked and defined is assigned to a series of “operations” 15 of potentially possible input events 17 . The fact that the user can freely design and configure these links 18 and the stored "operations" 15 greatly facilitates the user's artistic work when creating sounds and music, and the methodology proposed by the invention also means that the audio device 1 has such exceptional free-form configurability that the user can find previously unexplored ways of creating sound and music. No other existing audio device offers this high degree of freely designable configurability.

In particular, it is proposed that the priority values 19 correlate with the latency sensitivity of the sound generation or processing operations 15 assigned by the link 18 . The advantage of this is that “operations” that have a higher latency sensitivity are also assigned corresponding priority values 19, so that when the “operations” are processed later, their priority values lead to faster processing of the “operations”.

The audio device 1 according to the invention also has a processing unit 11 arranged in or on the housing 8 with an arithmetic processor 12 and a processing memory 13, the processing unit 11 having access to the internal data memory 10, the processing unit 11 being set up to do so using the arithmetic processor 12 to execute an algorithm 20 stored in the processing memory 13, according to which algorithm 20 the following steps are carried out, as also shown in FIG. 5: a) Calling up a pre-definable trigger generator 21 depending on the current configuration file 7, the trigger generator 21 for each sensor 9 having assignment criteria 22 for assigning input events 17 to user input signals 2 that can be detected by the sensors 9, with the reduction of energy consumption and bandwidth requirements of the audio device 1, the assignment criteria 22 of the trigger generator 21 according to step a) include limit values for the user input signals 2 detected by the sensors 9, so that input events 17 are only switched to an active state for those signals 2 that exceed the limit values, b) comparison of the priority values of the links 18 associated with the recorded input events 17 that are active and to which a link 18 to a sound generation and/or processing operation 15 is also assigned at the same time, and determining the order in which the linked and simultaneously active input events 17 assigned sound generation and/or or processing operation 15 are to be processed according to the level of the priority values 19, c) and execution of the sound generation and/or processing operations 15 assigned to the linked and simultaneously active input events 17 in the order defined according to the previous step b) for the generation of a digital data stream 4 to the later Conversion into an acoustic signal 5 and/or a control signal 6 for controlling an acoustic signal 5.

An advantageous embodiment of the trigger generator 21 according to the invention follows, the trigger generator 21 having three potentially possible input events 17 for each sensor 9; Activation Entry Event, Change Entry Event, and Inactivation Entry Event. The advantageous embodiment of the trigger generator 21 is illustrated using a state machine in a very energy-saving manner. In this example, the state machine has only two states per sensor 9: active and inactive, and these are assumed depending on whether there is a user interaction; if a user interaction is detected, an active state exists; or if there is no user interaction, then the inactive state is assumed. The User input signal 2 for a sensor 9 will shortly be denoted as x[t], where t corresponds to the time of a sample value of the user input signal 2 . This advantageous embodiment uses a variable X per sensor (see FIG. 6), which can be changed by the state machine. Assignment criteria that can be defined by the user are named TE, TD and TM in this example, which have the following meaning:

• TE is a limit value which, as soon as x[t] > TE is present, causes the state machine to change from the inactive state to the active state. Likewise, as soon as x[t] < TE, the state machine changes from the active state to the inactive state. As soon as the state machine changes from the inactive state to the active state, the activation input event is activated and as soon as the state machine changes from the active state to the inactive state, the deactivation input event is activated, in both cases the variable X = x[t] is assigned to the signal sample value to be saved at the time of the triggering of that input event.

• TD is another limit considered only in the active state, where the calculation and comparison abs(x[t] - X) > TD is performed to detect a change in the user input signal in the active state. The calculation abs(x[t] - X) calculates the difference magnitude between the current signal sample value x[t] and the signal sample value X from the last time the input event was triggered. As soon as this difference magnitude exceeds the limit value TD, the change input event is activated and X=x[t] is set.

• The introduction of a waiting time TW after the activation of an input event is particularly advantageous in order to prevent multiple activations of an input event possibly caused by measurement noise.

The trigger generator according to this embodiment only requires a maximum of a few calculations and one comparison operation per signal sample value and is therefore extremely energy-efficient and can be easily implemented in a computer processor. In addition, the input events that are assigned to the sensors 9 and can potentially be activated are only then switched to active mode as soon as the respectively corresponding user input signal exceeds the limits (TE and TD) which are definable by the user. In an exemplary embodiment, the limit values can be set higher in order to then activate the input events less frequently. Increasing the limit values and the input events that occur less frequently as a result leads to a reduction in the processing effort when processing linked "operations" leads to additional power savings and also significantly reduces the bandwidth required for later transmission of the digital data stream generated by the linked "operations". can. The user thus has the choice of how often and simply how often input events assigned to a sensor should be activated, and subsequently how often linked operations should be processed.

The embodiment of the trigger generator described above is implemented particularly advantageously if each sensor has assignment criteria that can be individually defined and an individual state machine.

According to step b) of the algorithm proposed above, it is proposed in particular that the priority values 19 be compared in such a way that those links 18 that are assigned to linked and simultaneously active input events 17 are processed in descending order with regard to their latency sensitivity, so that the link with with the highest latency sensitivity is processed first. If it is possible to process the links in parallel (e.g. by multi-core processors), they can advantageously also be processed in parallel. Subsequent links can also be processed in parallel (and thus taken at the same time). However, it should be ensured that a link with higher sensitivity is not processed after a link with lower sensitivity.

Advantageously, according to step b) of the algorithm proposed above, it is additionally proposed that each priority value is assigned a time function that is set up to increase priority values of links of active input events 17 that have not yet been executed by forming a time integral from the original priority value over the Duration is formed until the processing of its link, and the calculated value resulting from this integral calculation as a temporally dynamic priority value in the calculation in the ranking according to step b) as a priority value is used. The big advantage of this is that "operations" with a low latency sensitivity are also queued up and processed at some point, should there always be "operations" with a higher latency sensitivity.

The invention also advantageously proposes that the sound generation operations are internal sound generation operations and include at least one selection from the following list:

- Generation of a sound signal whose pitch and volume can be parameterized and controlled,

- Generation of a sound signal whose spectrum can be parameterized and controlled,

- Modulation of a sound signal, whereby the modulation strength and modulation width can be parameterized and controlled,

- Filtering of a sound signal, with filter parameters such as filter type, filter frequencies, filter gain being parameterizable and controllable,

- Delay of a sound signal, whereby the duration of the delay can be parameterized and controlled.

In other words, the invention proposes that internal sound generation operations are operations commonly used in signal generation, signal modulation and signal filtration. Internal tone generation operations in the present invention can either be digital operations or analog operations that can be controlled digitally. The signal generation, in which the pitch and tone strength can be controlled, is advantageously also carried out in combination with the control of the signal spectrum. The signal generator is then advantageously aligned in such a way that it can selectably generate different signal types, the signal types comprising at least sinusoidal signals, sawtooth signals and square-wave signals. In a similar way, the invention advantageously proposes that the modulation function and the filtering be aligned in such a way that parameterizable features of those functions can be selected and controlled by the user are. In the modulation function, the modulation strength and modulation width can be controlled, or the signal that can be used as the modulation signal source can advantageously be linked dynamically by control. In the case of filtration, at least the filter type (e.g. low-pass, band-pass, band-stop, high-pass), those cut-off frequencies that are used in the current filter type, and the filter gain can be parameterized and controlled. Since the configuration of the internal sound generation operations can be selected by the user and can be stored in a configuration file 7 and called up again, the invention permits a particularly advantageous dynamic system configuration.

The invention advantageously proposes that the sound generation operations comprise externally suppliable sound generation operations, these externally suppliable sound generation operations comprising at least one selection from the following list:

- Conversion of the signal value of the user input signals detected by the sensors into a signal value that is permissible for later conversion into an acoustic signal and/or a control signal for controlling an acoustic signal and/or into a protocol format that may be required for further data transmission,

- arithmetic and combinatorial processing of current and previous signals detected by the user input signals detected by the sensors capable of generating a modified signal usable for later conversion into an acoustic signal and/or a control signal for controlling an acoustic signal.

In other words, the invention advantageously proposes that external sound generation operations are operations and/or arithmetic and combinatorial processing that process and convert the signal values detected by the sensors, so that these processed and converted signal values can be used accordingly for further use . The further use can be, for example, either direct conversion into an acoustic signal and/or conversion into a control signal for controlling an acoustic signal and/or conversion into a protocol format that may be required for further data transmission. Since the configuration of the external sound generation operations can be selected by the user and can be stored in a configuration file 7 and called up again, the invention also allows a particularly advantageous dynamic system configuration in this case.

The processing operations can include at least one selection from the following list:

- a state machine capable of changing the state of the state machine based on current signal values, the user input signals detected by the sensors, and/or the internal state of the state machine and/or other values from the processing memory 13, each state the state machine is capable of performing selectable sound generation and/or processing operations.

- An event generation that is able to trigger a selectable input event forcibly and to emulate this input event therewith in order to process the assignment to the emulated input event to sound generation and processing operations in the predetermined prioritization list.

In other words, processing operations are, for example, operations that can influence internal states and generate events. It is not absolutely necessary for the processing operations to convert signal values directly into an acoustic signal and/or control signal, rather this happens in the sound generation operations linked by the processing operations and thus executed.

Since the configuration of the processing operations can be selected by the user and can be stored in a configuration file 7 and called up again, the invention also allows a particularly advantageous dynamic system configuration in this case.

The division into internal sound generation operations, external sound generation operations and processing operations made possible by the invention has the advantage that the user gets a particular clarity as to which operations are available and also a special overview of the associated configuration options with regard to the selectable operations.

According to the invention, it is proposed that the audio device 1 has at least one touch sensor for detecting manual user interactions 3 carried out on the audio device 1 by touching/pressing the touch sensor and the audio device 1 at least one position sensor for detecting on the audio device 1 through movement and/or or direction and/or change in position of the position sensor.

According to the invention, user input signals 2 are provided from the user interactions 3 detected by the sensors. The user input signals 2 or their signal values can be used by the user as control signals 6 for the above-mentioned listing of both internal and external sound generation operations as well as processing operations 15 . The user therefore has the choice and possibility of linking the user input signals to the operations by storing the configuration in the configuration file 7 . Through the stored configuration, the user creates a link between user input signals 2 and the selected executing operations. This configuration, which can be selected by the user, forms a major advantage of the invention, since new links between user interactions and executions that could not be represented before are now possible. Examples of such links of the user-created configuration that are advantageous due to the invention are given below:

- The user's finger pressure force is recorded by a force sensor and linked to the sound strength parameter of a signal generator. The link is configured and the recorded pressure force signal is processed and converted in such a way that the sound strength of the acoustic signal generated by the signal generator forms a function of the pressure force, with an increasing finger pressure force being represented as an increasing sound strength.

- The vertical position of the audio device 1 is detected by a position sensor and linked to the pitch parameters of a signal generator. The link is configured and the recorded vertical position signal is processed and converted in such a way that the pitch of the the acoustic signal generated by the signal generator forms a function of the orientation in relation to the vertical position.

Fig. 7 shows an advantageous embodiment of a force sensor for detecting the finger pressure force of the user, a so-called Force Sensing Resistor (FSR) pressure force sensor being used to detect the finger pressure force exerted by a correlating electrical resistance value, the detected resistance being value is in turn converted into a user input signal (2). In Fig. 7, a cross-sectional image of the audio device 1 is shown, with the user exerting a finger pressure force F7a on a silicone button F7b. The silicone push button F7b protrudes from a recess in the upper half of the housing 8 and distributes the exerted finger pressure force F7a on the underlying FSR pressure force sensor F7d. The applied finger pressure force F7a as well as the finger pressure force F7a distributed through the silicone button F7b on the FSR pressure force sensor F7d is illustrated in FIG. 7 by arrows. The FSR compression force sensor F7d is mounted on an underlying circuit board F7e, which is placed between the upper case half F7c and the lower case half F7f.

8 shows a position sensor located in the audio device 1, the position sensor in this case detecting the angles Fx, Fy and Fz mentioned here about the Cartesian coordinate system that is locally aligned for the audio device 1. In other words, Fx, Fy and Fz correspond to the roll, pitch and yaw angles of the audio device 1. The user input signals 2 formed from these angles can advantageously be used by the user as described above as control signals 6 for the above-mentioned listing of internal and external sound generation operations as well as processing operation 15 can be used configurably.

Advantageously, sensors other than those listed above can also be used, namely sensors that are able to detect user interactions 3 and convert them into user input signals 2, the detected user input signals 2 being used in the internal and/or external sound generation operations and/or or the processing operations 15 can be selectively linked and used by the configuration file 7 . An audio device 1 according to the invention can also advantageously use the following types of sensors: - Joystick sensor for detecting joystick position, whereby the X/Y joystick orientations can be used as user input signals 2,

- Radar sensor for detecting hand movements in the immediate vicinity of the audio device 1, where the detected hand position can be used as a user input signal 2.

In particular, the audio device 1 also has a data transmission interface for wireless transmission of the data stream to external devices, in particular a Bluetooth interface.

Through the data transmission interface, the user can link detected user input signals 2 to external instruments and equipment based on the appropriate configuration in the configuration file 7 and send data. Thereby the audio device 1 can be used as a kind of remote control, whereby the user input signals 2 are converted accordingly and then transmitted to be further converted into sound signals and/or control signals in the external instruments and equipment.

9 shows an advantageous embodiment of said data transmission interface, in which case a bidirectional data connection F9b is set up between the audio device 1 and an external device F9a in order to exchange control signals between the devices.

The audio device 1 advantageously has at least one output unit 23, with the output unit 23 being able to convert at least one of the signals generated by the signal generation and processing functions into an optical or acoustic or mechanical feedback signal and to send this feedback signal as feedback functions to the submit user.

In particular, the audio device 1 has at least one speaker and at least one calibration source, with at least one of the feedback signals being routed to the speaker and the speaker representing the feedback signal routed to it as an acoustic reproduction of a feedback function, with at least one of the feedback signals being routed to the Light source is passed and the light source represents the feedback signal passed to her as an optical representation of a feedback function.

FIG. 10 shows an advantageous embodiment of a light source that can be used for said feedback function, it being advantageously proposed here that the light source is integrated in the finger pressure force sensor according to FIG. Similar to FIG. 7, FIG. 10 shows a cross-sectional image of the audio device 1, with a semi-transparent silicone pushbutton F10a protruding from a recess in the upper housing half F10b. The finger pressure force sensor FlOd is located under the silicone button FlOa and the light source FlOc is located in a recess in the middle of the finger pressure force sensor FlOd. The light source FlOc and the finger pressure force sensor FlOd are attached to a printed circuit board FlOe, which is placed between the upper housing half FlOb and the lower housing half FlOf. When the light source FlOc is activated, it emits light through the semi-transparent silicone push button FlOa, as a result of which said feedback function is formed. If the audio device 1 is configured by the user in such a way that the luminosity of the light source FlOc correlates with the finger pressure force exerted by the user on the finger pressure force sensor FlOd, this forms a particularly effective and advantageous feedback function which can be used in the present invention. The effect of the feedback function 1 becomes particularly strong when the light source in such an embodiment is able to emit different color tones. For example, the green-orange-red color tones emitted and their mixed colors can correlate with the force of the finger pressure.

Fig. 11 shows a top view of the audio device 1, the light source being located below the semi-transparent silicone button Fila and when activated, the light source radiates through the semi-transparent silicone button Fila.

FIG. 12 shows a top view of the audio device 1, a loudspeaker being integrated in the housing 8 and emitting acoustic signals through the housing recesses F12a.

The feedback function allows the user to return selected states and functions of the audio device 1 as feedback information. This feedback function is advantageously also linked by the configuration file 7 and configured accordingly. The user thus has the choice of which feedback functions in should be active in a configuration and for which states or which user input signals they should be activated.

The invention is not limited to the embodiments shown but is defined by the full scope of the claims. Individual aspects of the invention or the embodiments can also be taken up and combined with one another. Any reference signs in the claims are exemplary and only serve to make the claims easier to read, without limiting them.

Claims

25 CLAIMS

1. Portable digital audio device (1) for converting user interactions (3) that can be detected as user input signal(s) (2) into a digital data stream (4) for later conversion into an acoustic signal (5) and/or a control signal (6) for controlling a Acoustic signal (5), wherein the audio device (1) is set up to carry out the conversion taking into account specifications that can be configured in accordance with a configuration file (7), the audio device (1) comprising the following:

- a housing (8),

- A number of sensors (9) arranged in or on the housing (8), wherein

* at least one input sensor is provided for continuously detecting a user interaction (3) and for converting the detected user interaction (3) into a user input signal (2) with a resolution of at least 7 bits,

- a first internal data memory (10) arranged in the housing (8) which has stored a number of pre-stored sound generation and/or processing operations (15) which can be linked to one another,

- a processing unit (11) arranged in or on the housing (8) with a processor (12) and a processing memory (13), wherein the processing unit (11) has access to the internal data memory (10), on which an input event list (16th ) of potentially possible input events (17) is stored, it being possible for the user of the audio device (1) to define at least one link to at least one sound generation and/or operation (15) for each potentially possible input event (17), with each link corresponding a configuration file (7) stored on the audio device (1) is assigned a priority value (19), the processing unit (11) being set up using the processor (12) to execute an algorithm stored on the processing memory (13), according to which Algorithm to perform the following steps: a) Calling up a predefinable trigger generator (21) depending on the current configuration file (7), the trigger generator (21) for each sensor (9) assignment criteria (22) for assigning input events (17) to the sensors (9 ) detectable user input signals (2), wherein to reduce the energy consumption and the bandwidth requirement of the audio device (1), the assignment criteria (22) of the trigger generator (21) according to step a) limit values for the sensors (9) detected user input signals (2) include, so that only for those signals (2) input events (17) are switched to an active state that exceed limit values, b) comparison of the priority values of the recorded input events (17) that are active and which are also linked to a sound generation and/or processing operation (15), and determining the order in which the linked and simultaneously active input events (17) assigned sound generation and/or processing operations are to be processed according to the priority values, c) and executing the linked and simultaneously active input events (17) associated sound generation and/or processing operations (15) in the order specified according to the previous step b) to generate a digital data stream (4) for later conversion into an acoustic signal (5) and/or a control signal (6) for control an acoustic signal (5),

- Wherein the audio device (1) further comprises an energy source (14) for the electrical supply of the number of sensors (9), the internal data memory (10) and the processing unit (11).

2. Audio device (1) according to claim 1, wherein the priority values (19) correlate with the latency sensitivity of the sound generation or processing operation (15) assigned by the link (18).

3. Audio device (1) according to claim 2, wherein the priority values (19) are compared in such a way that those links (18) which are linked and active at the same time Input events (17) are assigned, are processed in terms of their latency sensitivity in descending order, so that the link (18) with the highest latency sensitivity is processed first.

4. Audio device (1) according to one of the preceding claims, wherein each priority value (19) is assigned a time function which is set up to increase priority values of links (18) of active input events (17) that have not yet been executed by forming a time integral is formed from the original priority value over the period of time until its linkage is processed, and the value calculated resulting from this integral calculation is used as a time-dynamic priority value in the calculation in the ranking according to step b) as a priority value.

5. Audio device (1) according to one of the preceding claims, wherein the sound generation operations (15) are internal sound generation operations and comprise at least one selection from the following list:

6. Audio device (1) according to one of the preceding claims, wherein the sound generation operations (15) comprise externally suppliable sound generation operations, these externally suppliable sound generation operations comprising at least one selection from the following list: 28

- Conversion of the signal value of the user input signals detected by the sensors (9) into a signal value that is permissible for later conversion into an acoustic signal and/or a control signal for controlling an acoustic signal and/or in a protocol format required for further data transmission,

- arithmetic and combinatorial processing of current and previous signals detected by the sensors (9) detected user input signals, capable of generating a modified signal suitable for subsequent conversion into an acoustic signal and/or a control signal for controlling an acoustic signal is usable.

7. Audio device (1) according to one of the preceding claims, wherein the processing operations (15) include at least one selection from the following list:

- a state machine, which is capable of reading the state of the modifying the state machine, each state of the state machine being capable of performing selectable sound generation and/or processing operations (15).

- An event generation that is able to trigger a selectable input event forcibly and to emulate this input event in order to process the assignment to the emulated input event associated with sound generation and processing operations in the predetermined prioritization list.

8. Audio device (1) according to one of the preceding claims, wherein the audio device (1) has at least one touch sensor for detecting manual user interactions (3) carried out on the audio device (1) by touching/pressing the touch sensor.

9. Audio device (1) according to one of the preceding claims, wherein the audio device (1) has at least one position sensor for detecting user interactions (3) carried out on the audio device (1) by moving and/or changing the direction and/or position of the position sensor . 29

10. Audio device (1) according to one of the preceding claims, wherein the audio device (1) also has a data transmission interface for wireless transmission of the data stream to external devices, in particular a Bluetooth interface.

11. Audio device (1) according to one of the preceding claims, wherein the audio device (1) has at least one output unit (23), the output unit (23) being able to generate at least one of the signals generated by the signal generation and processing functions is to be converted into an optical, acoustic or mechanical feedback signal and to transmit this feedback signal to the user as feedback functions.

12. Audio device (1) according to claim 11, wherein the audio device (1) has at least one loudspeaker and at least one light source (FlOc),

- wherein at least one of the feedback signals is routed to the loudspeaker and the loudspeaker represents the feedback signal routed to it as an acoustic reproduction of a feedback function,

- wherein at least one of the feedback signals is routed to the light source and the light source represents the feedback signal routed to it as an optical representation of a feedback function.