WO2022006958A1

WO2022006958A1 - Method and system for achieving haptics effect

Info

Publication number: WO2022006958A1
Application number: PCT/CN2020/102421
Authority: WO
Inventors: 秦英明; 张玉蕾; 邵笑杰; 王修越; 李建其
Original assignee: 瑞声声学科技(深圳)有限公司
Priority date: 2020-07-06
Filing date: 2020-07-16
Publication date: 2022-01-13
Also published as: CN111897524A; CN111897524B

Abstract

Provided are a method and system for achieving a haptics effect. The method for achieving a haptics effect comprises: receiving a vibration event that occurs at a certain time point, and defining an effect ID and describing the vibration event; generating a vibration parameter according to a description of the vibration event; encapsulating the vibration parameter by means of an integrated interface, so as to obtain encapsulated data; defining a haptics effect enumeration type list by means of the encapsulated data, so as to realize the mapping between a signal name and the effect ID; performing processing according to the haptics effect enumeration type list, so as to obtain a key hardware parameter for driving hardware to vibrate; adapting a driving haptics chip according to the key hardware parameter; and driving a motor to vibrate by means of the haptics chip, so as to create a haptics effect. By means of the technical solution provided in the present invention, the universality, scenario adaptability and consistency are improved; the implementation is simpler and more direct, and the power consumption is also reduced; and the vibration diversity is improved, and the experience in a haptics effect is good.

Description

Method and system for implementing Haptics haptic effects

technical field

The present invention relates to the technical field of tactile perception, and in particular, to a method and system for realizing Haptics tactile effects.

Background technique

With the rapid development of smart devices, various APPs provide a variety of applications, and motors, as vibrotactile devices, are getting more and more attention. The mobile phone application generates an excitation signal of the motor as required, and the excitation signal drives the motor to vibrate to generate a corresponding haptic experience. However, the tactile experience using motor vibration in the prior art has the following shortcomings: First, the excitation signal of the motor is mostly pre-stored, and the tactile experience effect generated by the pre-stored vibration is limited, lacks versatility and scene adaptability, and has poor consistency. The second is to generate the corresponding vibration signal data after using the input signal (such as audio data), decoding the tactile information and the associated vibration parameters, and then using the vibration signal data to drive the motor to generate vibration, the process is more complicated, and the vibration and the input signal such as Audio correlation, lack of flexibility and customization capabilities; third, in specific applications, multi-motors are used when complex vibration scenes are to be realized, and the design of multi-motors is complex, high cost and power consumption; fourth, existing haptics The data storage format is single, such as Haptic Elementary Stream (Haptic Elementary Stream, HES) data, the versatility is poor; fifth, most of the parameters of the driving motor use the motor resonant frequency, so that the haptic experience effect brought by the motor vibration is less, the effect calculation results are inaccurate, and the haptic effect experience is poor.

technical problem

The present invention provides a method and system for realizing Haptics haptic effect, aiming at solving the problems of lack of versatility and scene adaptability and poor consistency of the existing motor vibration, as well as the complicated realization process and high power consumption, which bring The problem of single haptic effect and poor user experience.

technical solutions

To achieve the above object, the present invention provides a method for realizing Haptics haptic effect, the method comprising:

Step S10: Receive a vibration event that occurs at a certain point in time and perform effect ID definition and vibration event description; the vibration event description includes an effect library, an envelope library, a combination effect, a third-party native vibration effect, a synchronization mechanism and any one or more of the looping mechanisms;

Step S20: generating vibration parameters according to the description of the vibration event;

Step S30: encapsulating the vibration parameters through an integrated interface to obtain encapsulated data; the integrated interface includes a JNI extension interface and a third-party native interface;

Step S40: defining the encapsulation data as a list of Haptics effect enumeration types to realize the mapping between signal names and effect IDs;

Step S50: process according to the Haptics effect enumeration type list to obtain key hardware parameters for driving hardware vibration;

Step S60: adapting and driving the Haptics chip according to the key hardware parameters;

Step S70: Drive the motor to vibrate through the Haptics chip to generate a haptic effect.

Further, the effect library includes a general effect library and a custom effect library; the general effect library is a set of preset short effects; the custom effect library is a combination of different effects performed by a user according to requirements.

Further, the general effect library includes vibration information and its corresponding haptic effect.

Further, the envelope library is a vibration waveform determined by specifying vibration parameters; the vibration parameters include any one or more of excitation signal duration, relative time of excitation signal envelope points, vibration intensity and vibration frequency.

Further, the combined effect is composed of one or more Haptics vibration events according to a certain time sequence, and each vibration event corresponds to one vibration.

Further, the synchronization mechanism is a time stamp-based audio, video and haptic synchronization management mechanism, and parameters and files of the ringtone follow-up vibration are automatically generated according to the design file.

Further, the loop mechanism dynamically adjusts the scaling amplitude and loop interval based on internal and external conditions.

Further, the round-robin mechanism includes priority setting.

Further, the third-party native vibration effect is an effect supported by a third-party native interface.

Further, the third-party native interface is a Google native vibration interface.

Further, the vibration event description is stored as haptic data, and the format of the haptic data includes any one of txt, act, HED and xml.

In addition, the present invention provides a system for realizing Haptics haptic effects, the system for realizing Haptics haptic effects includes a processor platform, a Haptics chip and a motor, the processor platform is connected with the Haptics chip, and the Haptics chip is connected with all the Haptics chips. described motor connections, where:

The processor platform is used for processing vibration events and converting the vibration events into key hardware parameters that drive hardware vibration;

The Haptics chip is used to drive the motor to work;

The motors are used to vibrate to create haptic effects.

Further, the processor platform includes an application layer, an architecture layer, a JNI layer, a HIDL layer, a HAL layer and a Kernel layer in order from top to bottom, wherein,

The application layer is used to receive vibration events that occur at a certain point in time and to define effect IDs and describe vibration events;

The architecture layer is configured to generate vibration parameters according to the description of the vibration event;

The JNI layer is used to encapsulate the vibration parameters through an integrated interface to obtain encapsulated data;

The HIDL layer is used to define the Haptics effect enumeration type list by the encapsulation data to realize the mapping of the signal name and the effect ID;

The HAL layer is used for processing according to the Haptics effect enumeration type list to obtain key hardware parameters for driving hardware vibration;

The Kernel layer is adapted to drive the Haptics chip according to the key hardware parameters.

Preferably, the JNI layer is a Java native interface layer, and the JNI layer includes a JNI extension interface and an Android native interface.

Preferably, the system for implementing Haptics haptic effects further includes a power supply, and the power supply provides power for the processor platform and the Haptics chip.

Preferably, the parameters of the driving motor include any one or more of desired excitation voltage signal, vibration acceleration signal, motor speed signal, motor displacement signal, maximum voltage limit and displacement protection.

Preferably, the maximum voltage limit is used to prevent exceeding the limit of the maximum output capability of the hardware.

Preferably, the displacement protection is to limit the electrical signal as a whole according to the safe displacement of the vibrator of the motor, so as to ensure that the electrical signal will not cause the vibrator to exceed the predicted safe displacement.

beneficial effect

The method and system for realizing Haptics haptic effect provided by the present invention, the benefits it brings include:

1. There are two ways of dynamic adaptation and automatic generation of the excitation signal of the pre-stored drive motor, which improves the versatility, scene adaptability and consistency;

2. Directly generate the excitation signal to drive the motor to vibrate by inputting the excitation signal and associated parameters, the process is simpler and more direct, and the vibration is not associated with the audio;

3. Using the technical solution of the present application to achieve the effect of multiple motors through a single motor, reducing power consumption;

4. The formats of haptic data are diversified, including txt, act, HED and xml formats are supported, which improves the versatility and diversity;

5. The parameters of the drive motor are rich, the effect calculation results are more accurate, and the haptic effect brought by the vibration is good.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, under the premise of no creative work, other drawings can also be obtained from these drawings, wherein:

1 is a schematic flowchart of a method for implementing a Haptics haptic effect according to an embodiment of the present invention;

2 is a schematic diagram of providing an envelope form according to an embodiment of the present invention;

3 is a schematic diagram of a combined effect form provided by an embodiment of the present invention;

4 is a schematic diagram of Google native vibration interface support provided by an embodiment of the present invention;

5 is a schematic structural diagram of a system for providing a Haptics haptic effect according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of the framework of the processor platform in FIG. 5 .

Embodiments of the present invention

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The user interface device for the target haptic effect experience is a mobile phone, and the target haptic output device inside the mobile phone includes a motor, and the motor includes a linear motor, a rotor motor, or a Piezo motor. Specifically, the applications of tactile experience include: various virtual keys, side keys, contextual change menus, and tactile interaction prompts edge touch, tap and slide operation and pressure touch, off-screen fingerprint recognition, and pressing operations, etc.; OS operation feedback ( For example: system-level reusable haptic controls, custom controls and custom feedback, combined with OEM design, animation, etc.); personalized notifications (for example: rhythm ringtone vibration, personalized notification, custom vibration pattern (vibration combination), etc.) ; Games, AR (for example: haptic enhancement of game elements, physical engine coordination, virtual space positioning, object selection, movement, distinguishing environmental elements and protagonist actions, etc.), etc. The present invention provides a method and system for realizing Haptics haptic effect, which can give users an immersive experience by accurately adapting the drive motor to different applications and scenarios.

Referring to FIG. 1, the present invention provides a method for realizing Haptics haptic effect, including:

Step S10: Receive a vibration event that occurs at a certain point in time and perform effect ID definition and vibration event description; the vibration event description includes an effect library, an envelope library, a combination effect, a third-party native vibration effect, a synchronization mechanism and Any one or more of the looping mechanisms.

Specifically, the effect ID is defined as an ID identifier of a preset short excitation signal, so as to distinguish different short excitation signals. The effect library includes a general effect library and a custom effect library; the general effect library is a set of preset short effects; specifically, the general effect library includes vibration information and its corresponding haptic effect, and the vibration information is a drive The excitation signal of the motor is an electrical signal that directly acts to drive the motor or an indirect signal of acceleration, momentum, displacement, speed, etc. The customized effect library is for the user to combine different effects according to the requirements; the user can perform the combination of different effects according to the requirements to realize the customized special touch.

The envelope library is a vibration waveform determined by specifying vibration parameters; the vibration parameters include any one or more of excitation signal duration, relative time of excitation signal envelope points, vibration intensity and vibration frequency. As shown in FIG. 2 , it is a schematic diagram of an envelope form. Specifically, in an embodiment of the present invention, a vibration including four parameter configurations is defined, wherein the four parameter configurations are: intensity, frequency, duration and F-MODE, F-MODE is the vibration type.

The combined effect is composed of one or more Haptics vibration events according to a certain time sequence, and each vibration event corresponds to one vibration. A Haptics vibration event can be a transient effect that changes the vibration intensity based on one of the vibration effects in the effects library. Steady transient effects and transient effects can be combined to form a combined effect. As shown in Figure 3, it is a schematic diagram of the combined effect form, in which each vibration is described by (a, b, freqn), a represents the start time, b represents the vibration intensity, and freqn represents the vibration frequency, where n is a natural number; Figure 3 , the initial vibration is (0, 0, freq0), the first vibration is (t1, Strength1, freq1), the second vibration is (t2, Strength2, freq2), and the third vibration is (t3, 0, freq3 ), combine the first vibration, the second vibration and the third vibration into a new vibration by combining: Act{(0, 0, freq0), (t1, Strength1, freq1), (t2, Strength2, freq2) , (t3, 0, freq3)}.

The synchronization mechanism is a time stamp-based audio, video and tactile synchronization management mechanism, and automatically generates parameters and files of ringtone follow-up vibrations according to design files.

The loop mechanism dynamically adjusts the zoom amplitude and loop interval based on internal and external conditions. The looping mechanism loops for combined effects, for example the following example is a practical application using the looping mechanism:

1. In the game, the vehicle is driving at a constant speed, and the driving process is relatively consistent, which can be achieved by playing the same combination effect in a loop;

2. When the vehicle is driving at a constant speed in the game, stepping on the accelerator will increase the overall amplitude, that is, increase the vibration intensity;

3. Continuous shooting in the game, the expression of each bullet is the same, which can be achieved by playing the same combination effect in a loop;

4. According to different guns in the game, the interval of bullets is different, and the interval of the vibration effect corresponding to each bullet can be adjusted;

Further, the looping mechanism includes priority setting. According to the priority presets of different playback streams, high-priority playback can interrupt low-priority playback, and when high-priority playback ends, the original playback content can be restored. An example of a practical application of the loop mechanism:

1. Support shooting while driving in the game;

2. Support system vibration (such as incoming calls) to interrupt the vibration of the current app.

The third-party native vibration effect is an effect supported by a third-party native interface; specifically, in an embodiment provided by the present invention, the third-party native interface is a Google native vibration interface, and for the Z-direction motor, the effect can be converted into Google The effect supported by the native interface, realize the basic vibration, and expand the Android platform application. As shown in Figure 4, input the design signal and hardware model information, the hardware model information is identified by the platform characteristics, and the design signal is the excitation signal for driving the vibration of the motor. , use the Android native VibratorService interface service directly. If it is not supported, use the Android native VibratorService interface service after using signal adjustment/interface conversion.

Step S20: Generate vibration parameters according to the description of the vibration event; the vibration parameters include a general effect library (also called built-in effect), a custom effect library (stored in the form of a file, also called a custom file or an ACT file) in the effect library. ), envelope library, combination effect, synchronization mechanism and loop mechanism, and, for third-party native vibration effects, such as Google's native vibration interface, use the extension interface to convert to vibration parameters.

Step S30: Encapsulate the vibration parameters through an integrated interface to obtain encapsulated data; the integrated interface includes a JNI extension interface and a third-party native interface.

Step S40: Define the encapsulation data as a list of Haptics effect enumeration types, so as to realize the mapping between signal names and effect IDs.

Step S50: perform processing according to the Haptics effect enumeration type list to obtain key hardware parameters used to drive hardware vibration; perform corresponding processing according to the parameters in the Haptics effect enumeration type list, including intensity scaling, envelope signal Generation, displacement protection, etc. The effect library is the pre-generated deterministic Haptics effect, and the hardware configuration information is the pre-configured key hardware parameters, including information such as the motor resonant frequency F0.

Step S60: Adapt and drive the Haptics chip according to the key hardware parameters.

Please refer to FIG. 5 , in order to realize the above method for realizing Haptics haptic effect, the present invention provides a system for realizing Haptics haptic effect, and the system for realizing Haptics haptic effect includes processor platform 10 , Haptics chip 20 , motor 30 and power supply 40 , the processor platform 10 is connected to the Haptics chip 20 , the Haptics chip 20 is connected to the motor 30 , and the power supply 40 provides power for the processor platform 10 and the Haptics chip 20 .

in:

The processor platform 10 is used to process vibration events and convert the vibration events into key hardware parameters that drive hardware vibration; the processor platform 10 is used as a system-level software carrier, and a customizable HAL interface can be used to process the vibration events. In various working scenarios in the mobile phone application, an interface with data transmission capability is used to communicate with the Haptics chip 20 .

The Haptics chip 20 is used to drive the motor to work; the Haptics chip 20 is a drive chip of the motor, and the Haptics chip 20 has a built-in power amplifier chip that can meet various motion modes of the motor. At the same time, the Haptics chip 20 also Including drive voltage, sampling rate and data interface, to achieve precise and flexible motor drive.

The motor 30 is used to vibrate to produce haptic effects. The motor 30 includes a linear motor, a rotor motor or a Piezo motor. The motor is positioned to match the end product and combined with the corresponding structural design. With the Haptics chip, the motor can respond quickly in various states and maintain low power consumption. , and excellent transient power response capability.

Referring to FIG. 6, the processor platform includes an application layer, an architecture layer, a JNI layer, a HIDL layer, a HAL layer, and a Kernel layer in sequence from top to bottom, wherein,

The application layer is used to receive a vibration event that occurs at a certain point in time and perform effect ID definition and vibration event description; the vibration event description includes an effect library, an envelope library, a combination effect, a third-party native vibration effect, Any one or more of synchronization mechanism and round-robin mechanism.

The architecture layer is used to generate vibration parameters according to the description of the vibration event; the vibration parameters include a general effect library (also called built-in effect) and a custom effect library (stored in the form of a file, also called a custom file) in the effect library. or ACT file), envelope library, combination effect, third-party native vibration effect support (such as Google support), for system applications, such as the system ringtone of the mobile phone system, the vibration design is realized through the json parsing and synchronization mechanism through the extension interface, and the Custom file containing vibration parameters.

The JNI layer is used to encapsulate the vibration parameters through the integrated interface to obtain encapsulated data; the JNI is a Java native interface (Java Native interface). Interface, JNI), provides an interface scheme integrating Java and C language native code; the JNI layer includes JNI extension interface and Android native interface.

The HIDL layer is used to define the encapsulation data as a list of Haptics effect enumeration types to realize the mapping between the signal name and the effect ID; the HIDL is an interface description language (HAL Interface) between the HAL layer and its users. Description Language, HIDL), the HIDL defines a list of Haptics effect enumeration types by matching with the HAL layer.

The HAL layer is used for processing according to the Haptics effect enumeration type list to obtain key hardware parameters for driving hardware vibration; the HAL is a hardware abstraction layer (Hardware Abstract Layer, HAL), the HAL processing the Haptics effect enumeration type list is to perform corresponding processing according to the parameter category, including intensity scaling, envelope signal generation, displacement protection, etc. The effect library is a pre-generated deterministic Haptics effect, and the hardware information table includes hardware configuration information, where the hardware configuration information is pre-configured key hardware parameters, including information such as the motor resonant frequency F0. In the realization of the system for realizing the Haptics haptic effect, there are two different processing paths according to different transfer parameters. If it is a pre-defined excitation voltage signal, it needs to directly output the voltage after scaling the digital amplitude according to the passed Scale parameter. If it is a defined vibration signal, it needs to be equalized to generate an electrical signal after scaling according to the Scale parameter; if it is other effects, a specific form of vibration signal will be generated according to the equalization algorithm formula according to the motor parameter information, etc. The parameters of the drive motor include any one or more of desired excitation voltage signal, vibration acceleration signal, motor speed signal, motor displacement signal, maximum voltage limit and displacement protection. By crest factor reduction (Crest Factor Reduction, CFR) process determines the maximum voltage limit that is used to prevent exceeding the limit of the maximum output capability of the hardware. Through the displacement protection, the electric signal of the motor's safe vibrator displacement is limited as a whole to ensure that the electric signal will not cause the vibrator to exceed the predicted safe displacement.

The Kernel layer is adapted to drive the Haptics chip according to the key hardware parameters. Due to the inconsistency of the performance of components in specific hardware circuits, such as driver chips and motors, the same effect may be different on the same type of electronic equipment. Therefore, after reading the parameter calibration information (such as resonant frequency, etc.), the drive signal input to the motor can be adjusted according to the preset algorithm, so as to achieve the consistency of the effect of different motors.

Compared with the prior art, the method and system for realizing Haptics haptic effect provided by the present invention, its benefits include:

The above are only the embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, improvements can be made without departing from the inventive concept of the present invention, but these belong to the present invention. scope of protection.

Claims

A method for realizing Haptics haptic effect, comprising:

Step S10: Receive a vibration event that occurs at a certain point in time and perform effect ID definition and vibration event description; the vibration event description includes an effect library, an envelope library, a combination effect, a third-party native vibration effect, a synchronization mechanism and any one or more of the looping mechanisms;

Step S20: generating vibration parameters according to the description of the vibration event;

Step S30: encapsulating the vibration parameters through an integrated interface to obtain encapsulated data; the integrated interface includes a JNI extension interface and a third-party native interface;

Step S40: defining the encapsulation data as a list of Haptics effect enumeration types to realize the mapping between signal names and effect IDs;

Step S50: process according to the Haptics effect enumeration type list to obtain key hardware parameters for driving hardware vibration;

Step S60: adapting and driving the Haptics chip according to the key hardware parameters;

Step S70: Drive the motor to vibrate through the Haptics chip to generate a haptic effect.
The method for implementing Haptics haptic effects according to claim 1, wherein the effect library includes a general effect library and a custom effect library; the general effect library is a set of preset short effects; the custom effect library For users to combine different effects according to their needs.
The method for implementing Haptics haptic effects according to claim 2, wherein the general effect library includes vibration information and its corresponding haptic effects.
The method for realizing Haptics haptic effect according to claim 1, wherein the envelope library is a vibration waveform determined by specifying vibration parameters; and the vibration parameters include the duration of the excitation signal and the relative time of the envelope points of the excitation signal. , any one or more of vibration intensity and vibration frequency.
The method for realizing a Haptics haptic effect according to claim 1, wherein the combined effect is composed of one or more Haptics vibration events according to a certain time sequence, and each vibration event corresponds to one vibration.
The method for realizing Haptics haptic effect according to claim 1, wherein the synchronization mechanism is a time stamp-based audio, video and haptic synchronization management mechanism, and parameters and files of ringtone follow-up vibration are automatically generated according to the design file.
The method for implementing Haptics haptic effects according to claim 1, wherein the loop mechanism dynamically adjusts the zoom amplitude and loop interval based on internal and external conditions.
The method of implementing a Haptics haptic effect of claim 7, wherein the round-robin mechanism includes priority setting.
The method for implementing a Haptics haptic effect according to claim 1, wherein the third-party native vibration effect is an effect supported by a third-party native interface.
The method for implementing Haptics haptic effects according to claim 9, wherein the third-party native interface is a Google native vibration interface.
The method for implementing a Haptics haptic effect according to claim 1, wherein the vibration event description is stored as haptic data, and the format of the haptic data includes any one of txt, act, HED and xml.
A system for realizing Haptics haptic effects, wherein the system for realizing Haptics haptic effects includes a processor platform, a Haptics chip and a motor, the processor platform is connected to the Haptics chip, and the Haptics chip is connected to the Haptics chip. Motor connection, where:

The processor platform is used for processing vibration events and converting the vibration events into key hardware parameters that drive hardware vibration;

The Haptics chip is used to drive the motor to work;

The motors are used to vibrate to create haptic effects.
The system for implementing Haptics haptic effects according to claim 12, wherein the processor platform includes an application layer, an architecture layer, a JNI layer, a HIDL layer, a HAL layer and a Kernel layer in order from top to bottom, wherein,

The application layer is used to receive vibration events that occur at a certain point in time and to define effect IDs and describe vibration events;

The architecture layer is configured to generate vibration parameters according to the description of the vibration event;

The JNI layer is used to encapsulate the vibration parameters through an integrated interface to obtain encapsulated data;

The HIDL layer is used to define the Haptics effect enumeration type list by the encapsulation data to realize the mapping of the signal name and the effect ID;

The HAL layer is used for processing according to the Haptics effect enumeration type list to obtain key hardware parameters for driving hardware vibration;

The Kernel layer is adapted to drive the Haptics chip according to the key hardware parameters.
The system for implementing Haptics haptic effects according to claim 13, wherein the JNI layer is a Java native interface layer, and the JNI layer includes a JNI extension interface and an Android native interface.
The system for implementing Haptics haptic effects according to claim 12, wherein the system for implementing Haptics haptic effects further comprises a power supply, and the power supply provides power for the processor platform and the Haptics chip.
The system for implementing Haptics haptic effects according to claim 12, wherein the parameters of the driving motor include desired excitation voltage signal, vibration acceleration signal, motor speed signal, motor displacement signal, maximum voltage limit and displacement protection any one or more of them.
The system of claim 16, wherein the maximum voltage limit is used to prevent exceeding the limit of the maximum output capability of the hardware.
The system for realizing Haptics haptic effect according to claim 16, wherein the displacement protection is to limit the electrical signal as a whole according to the safe displacement of the vibrator of the motor, so as to ensure that the electrical signal will not cause the vibrator to exceed the predicted safe displacement.