WO2022110259A1 - 振动生成方法、振动控制方法及其相关设备 - Google Patents

振动生成方法、振动控制方法及其相关设备 Download PDF

Info

Publication number
WO2022110259A1
WO2022110259A1 PCT/CN2020/133228 CN2020133228W WO2022110259A1 WO 2022110259 A1 WO2022110259 A1 WO 2022110259A1 CN 2020133228 W CN2020133228 W CN 2020133228W WO 2022110259 A1 WO2022110259 A1 WO 2022110259A1
Authority
WO
WIPO (PCT)
Prior art keywords
vibration
note
audio
energies
energy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/133228
Other languages
English (en)
French (fr)
Inventor
郑亚军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AAC Technologies Holdings Shenzhen Co Ltd
AAC Microtech Changzhou Co Ltd
Original Assignee
AAC Acoustic Technologies Shenzhen Co Ltd
AAC Microtech Changzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AAC Acoustic Technologies Shenzhen Co Ltd, AAC Microtech Changzhou Co Ltd filed Critical AAC Acoustic Technologies Shenzhen Co Ltd
Publication of WO2022110259A1 publication Critical patent/WO2022110259A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/36—Accompaniment arrangements
    • G10H1/40—Rhythm
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02P—CONTROL OR REGULATION OF ELECTRIC MOTORS, ELECTRIC GENERATORS OR DYNAMO-ELECTRIC CONVERTERS; CONTROLLING TRANSFORMERS, REACTORS OR CHOKE COILS
    • H02P25/00—Arrangements or methods for the control of AC motors characterised by the kind of AC motor or by structural details
    • H02P25/02—Arrangements or methods for the control of AC motors characterised by the kind of AC motor or by structural details characterised by the kind of motor
    • H02P25/032—Reciprocating, oscillating or vibrating motors
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02P—CONTROL OR REGULATION OF ELECTRIC MOTORS, ELECTRIC GENERATORS OR DYNAMO-ELECTRIC CONVERTERS; CONTROLLING TRANSFORMERS, REACTORS OR CHOKE COILS
    • H02P7/00—Arrangements for regulating or controlling the speed or torque of electric DC motors
    • H02P7/02—Arrangements for regulating or controlling the speed or torque of electric DC motors the DC motors being of the linear type

Definitions

  • the present invention relates to the technical field of vibration drive, and in particular, to a vibration generation method and device thereof, a vibration control method and system thereof, an electronic device and a storage medium.
  • a vibration motor is provided in the music playing device, and when the music is playing, the motor is controlled to vibrate so that the music playing device produces a haptic vibration effect.
  • the present invention provides a vibration generation method, the method includes:
  • a haptic vibration file corresponding to the audio signal is generated from a plurality of the normalized energies.
  • the acquiring a plurality of note energies of the audio signal according to the audio beat includes:
  • the step of obtaining the note energy of each of the notes includes:
  • the note energy of each of the notes is obtained by calculation.
  • performing normalization preprocessing on a plurality of the note energies respectively, and obtaining a plurality of normalized energies includes:
  • the normalized energy corresponding to each of the notes is obtained according to the maximum value of the energy of the multiple notes and the note energy of each of the notes.
  • the step of generating a haptic vibration file corresponding to the audio signal according to the normalized energy includes:
  • a haptic vibration file corresponding to the audio signal is generated.
  • the present invention provides a vibration control method, the vibration control method includes the steps of the vibration generation method of the present invention, and further includes:
  • the motor is driven to vibrate to generate a haptic vibration effect.
  • the present invention provides a vibration generating device, comprising:
  • an audio processing module for identifying audio beats in the audio signal
  • the calculation processing module is used for obtaining multiple note energies of the audio signal according to the audio beat; for performing normalization preprocessing on the multiple note energies respectively, and obtaining multiple normalized energies, wherein the The note energy is set in a one-to-one correspondence with the normalized energy; it is used for generating a haptic vibration file corresponding to the audio signal according to a plurality of the normalized energy.
  • the present invention provides a vibration control system, which includes:
  • the vibration generating device of the present invention is used for inputting a tactile vibration file
  • a tactile vibration identification module for extracting the tactile vibration points in the tactile vibration file, and also for generating a vibration driving voltage corresponding to the tactile vibration points according to each of the tactile vibration points;
  • a vibration unit configured to vibrate according to the vibration driving voltage to generate a haptic vibration effect.
  • the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vibration generating method of the present invention and/or the steps of the present invention are implemented.
  • the present invention provides a computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the vibration generating method of the present invention and/or the steps of the vibration generation method of the present invention are implemented. The steps of the vibration control method.
  • the above-mentioned vibration generation method and device thereof calculate and process the note energy of each note by a normalization optimization algorithm to obtain the normalized energy of each note, and generate a tactile vibration file corresponding to the audio signal according to the normalized energy , the tactile vibration effect and the beat of the audio signal are flexibly matched.
  • the normalized energy of each note is higher than the note energy of the note, the problem of insufficient local audio intensity is effectively avoided, and the haptic vibration effect and audio signal are improved.
  • the accuracy of matching the beats of each other, and the coordination between auditory perception and tactile perception is good, which provides conditions for improving the user's listening experience; in addition, the above vibration generation method is applied to the vibration control method and its system, When the audio signal is played, the above-mentioned vibration control method and system thereof can generate a tactile vibration effect according to the tactile vibration file, and the tactile vibration effect is highly matched with the beat of the audio signal, which improves the coordination between the user's auditory experience and tactile experience, Effectively improve the user's listening experience.
  • Fig. 1 is the schematic flow chart of the vibration generation method of the present invention
  • Fig. 2 is the schematic flow chart of step S24 of Fig. 1;
  • step S28 in FIG. 1 is a schematic flowchart of step S28 in FIG. 1;
  • Fig. 5 is the structural block diagram of the vibration generating device of the present invention.
  • FIG. 6 is a structural block diagram of the vibration control system of the present invention.
  • the present invention provides a vibration generation method, comprising the following steps:
  • Step 22 identifying the audio beat in the audio signal.
  • Step 24 Acquire multiple note energies of the audio signal according to the audio beat.
  • Step 26 Perform normalization preprocessing on a plurality of the note energies respectively to obtain a plurality of normalized energies; wherein the note energies and the normalized energies are set in a one-to-one correspondence.
  • Step 28 Generate a haptic vibration file corresponding to the audio signal according to the plurality of normalized energies.
  • the above vibration generation method calculates and processes the note energy of each note through a normalization optimization algorithm to obtain the normalized energy of each note, and generates a corresponding audio signal according to the normalized energy.
  • the haptic vibration file realizes the flexible matching of the haptic vibration effect and the beat of the audio signal.
  • the normalized energy of each note is higher than the note energy of the note, the problem of insufficient local audio intensity is effectively avoided, and the haptic vibration effect is improved.
  • the accuracy of matching with the beat of the audio signal, and the coordination between the auditory feeling and the tactile feeling are good, which is beneficial to improve the user's listening experience.
  • Step S22 identifying the audio beat in the audio signal.
  • a signal preprocessing process can be added according to actual needs, and the signal preprocessing process is: importing an audio file, acquiring the audio signal of the audio file, extracting the audio feature information of the audio signal, according to the The audio feature information identifies beat moments of a plurality of beat points, and obtains an audio beat sequence Bp from the audio signal according to the beat moments of the plurality of beat points, and the audio beat sequence Bp includes a plurality of audio beats.
  • Step S24 Acquire multiple note energies of the audio signal according to the audio beat.
  • the step S24 includes:
  • Step S241 Acquire multiple notes in the audio beat according to a preset time interval; wherein, the time interval between two adjacent notes is the preset time interval, denoted as T 0 .
  • the preset time interval T 0 can be specifically determined according to the actual application situation, and the notes in the same audio beat are represented as T(n), where n is the same audio frequency
  • the number of notes in a beat for example, in this embodiment, the preset time interval is specifically set to be a quarter of the duration of a beat, that is, it is equivalent to that there are four notes with the same time interval in each beat.
  • n represents the 1st, 2nd, 3rd, and 4th note numbers in the same audio beat
  • the first to fourth notes in the same audio beat are represented as T(1), T(2), T( 3), T(4).
  • Step S242 acquiring the note energy of each of the notes.
  • the note energy in the same audio beat is represented as P 0 (n), and in this embodiment, the note energy of the first to fourth notes in the same audio beat is represented as P 0 (1 ), P 0 (2), P 0 (3), P 0 (4).
  • step S242 within the same audio beat, the note energy of each of the notes is calculated and obtained according to the amplitude of the audio signal corresponding to each of the notes. Meanwhile, before step S242, the sampling rate of the audio signal needs to be detected in advance, which is denoted as f s .
  • X 0 (t n ) is the amplitude of the audio signal corresponding to the time t n of the nth note
  • X 0 (t n+k ) is the audio frequency corresponding to the time t n +k of the nth note.
  • the amplitude of the signal where time t n is the start time of the nth note, time tn +k is the end time of the nth note, k is the number of sampling points of the nth note, and T 0 is the adjacent two
  • the time interval of notes, f s is the sampling rate.
  • the second note T(2) of the third audio beat occurs at 3 seconds and lasts for 0.2 seconds (that is, the time interval T 0 is 0.2 seconds), and the signal sampling rate fs is 48000. in:
  • the step S242 may also select a specific calculation method of the note energy according to the actual situation.
  • Step S26 performing normalization preprocessing on a plurality of the note energies respectively to obtain a plurality of normalized energies; wherein the note energies and the normalized energies are set in a one-to-one correspondence.
  • the normalized energy in the same audio beat is represented as P(n), and in this embodiment, the normalized energy of the first to fourth notes in the same audio beat is represented as P in turn (1), P(2), P(3), P(4).
  • the normalized energy corresponding to each of the notes is obtained according to the maximum value of the energy of a plurality of the notes and the note energy of each of the notes.
  • P(n) represents the normalized energy corresponding to any note in the beat
  • max(P 0 (n)) represents the maximum note energy in the note energy sequence P 0 (n) in the same audio beat
  • max(P 0 (n)) is the maximum value among the note energies P 0 (1), P 0 (2), P 0 (3), and P 0 (4).
  • Step S28 generating a haptic vibration file corresponding to the audio signal according to a plurality of the normalized energies.
  • the step S28 includes:
  • Step S281 marking the note corresponding to the normalized energy higher than the preset energy threshold as a tactile vibration point.
  • the preset energy threshold value is expressed as Ph, and the value of the preset energy threshold value Ph can be specifically set according to the actual application.
  • the preset energy threshold Ph is set to 0.5 ⁇ max (P( n))
  • max(P(n)) represents the maximum normalized energy in the normalized energy sequence P(n) within the same audio beat, that is, the preset energy threshold Ph is set to be within the same audio beat One-half of the largest normalized energy in the normalized energy sequence P(n).
  • the distribution of the haptic vibration points on the audio signal can be made more uniform, and the haptic vibration can be evenly generated when the audio signal is played It avoids that the vibration frequency is too high and the vibration pain is caused to the user, and the vibration frequency is too low and the tactile vibration effect is not obvious, which effectively improves the user's tactile experience and makes the experience better.
  • Step S282 generating a haptic vibration file corresponding to the audio signal according to each of the haptic vibration points.
  • the present invention provides a vibration control method applying the above vibration generation method, and the vibration control method includes the following steps:
  • Step 302 extracting the haptic vibration points in the haptic vibration file.
  • Step 304 Generate a vibration driving voltage corresponding to each of the tactile vibration points according to each of the tactile vibration points.
  • Step 306 according to the vibration driving voltage, drive the motor to vibrate to generate a haptic vibration effect.
  • above-mentioned vibration control method when playing audio signal, above-mentioned vibration control method can produce tactile vibration effect according to tactile vibration file, this tactile vibration effect and the audio rhythm matching degree of audio signal are high, improve the user's auditory feeling and tactile feeling. It can effectively improve the user's listening experience.
  • steps in the flowcharts of FIGS. 1-4 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 1-4 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution of these steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or phases within the other steps.
  • the present invention provides a vibration generating device 100, which includes an audio processing module 101 and a calculation processing module 102, wherein:
  • the audio processing module 101 is used to identify the audio beat in the audio signal
  • the calculation processing module 102 is configured to obtain multiple note energies of the audio signal according to the audio beat; and is configured to perform normalization preprocessing on the multiple note energies to obtain multiple normalized energies, Wherein, the note energy and the normalized energy are set in a one-to-one correspondence; it is used for generating a haptic vibration file corresponding to the audio signal according to a plurality of the normalized energy.
  • the audio processing module 101 is further configured to acquire a plurality of notes in the audio beat according to a preset time interval, wherein the time interval between two adjacent said notes is the preset time interval;
  • the notes corresponding to the normalized energy above the preset energy threshold are marked as haptic vibration points.
  • the calculation processing module 102 is further configured to obtain a plurality of note energies of the audio signal according to the audio beat, specifically, to obtain the note energy of each of the notes; Perform normalization preprocessing respectively to obtain a plurality of normalized energies, wherein the note energy and the normalized energy are set in a one-to-one correspondence; it is used to generate a
  • the tactile vibration file corresponding to the audio signal specifically, used to identify the note corresponding to the normalized energy higher than the preset energy threshold is marked as a tactile vibration point, and is used to generate the corresponding tactile vibration point according to each of the tactile vibration points.
  • the haptic vibration file corresponding to the audio signal is further configured to obtain a plurality of note energies of the audio signal according to the audio beat, specifically, to obtain the note energy of each of the notes; Perform normalization preprocessing respectively to obtain a plurality of normalized energies, wherein the note energy and the normalized energy are set in a one-to-one correspondence; it is used to generate a
  • the present invention provides a vibration control system 200, which includes the above-mentioned vibration generating device 100, a tactile vibration recognition module 201, and a vibration unit 202, wherein:
  • the vibration generating apparatus 100 is used for inputting a haptic vibration file.
  • the tactile vibration identification module 201 is configured to extract the tactile vibration points in the tactile vibration file, and is also configured to generate a vibration driving voltage corresponding to the tactile vibration point according to each of the tactile vibration points.
  • the vibration unit 202 is configured to vibrate according to the vibration drive voltage to generate a tactile vibration effect; specifically, the structural form of the vibration unit 202 can be specifically selected according to the actual use situation, for example, in this embodiment,
  • the vibration unit 202 is a vibration motor.
  • each module in the above vibration generating device or vibration control system may be implemented in whole or in part by software, hardware and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vibration generating method of the present invention and/or the steps of the present invention are implemented.
  • the present invention provides a computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the vibration generating method of the present invention and/or the steps of the vibration generation method of the present invention are implemented. The steps of the vibration control method.
  • any reference to memory, storage, database or other media used in the various embodiments provided by the present invention may include at least one of non-volatile and volatile memory.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc.
  • Volatile memory may include random access memory (Random Access Memory) Access Memory, RAM) or external cache memory.
  • RAM may take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (Dynamic Random Access Memory). Access Memory, DRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Power Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种振动生成方法及其装置、振动控制方法及其系统、电子设备以及存储介质。振动生成方法包括：步骤S22，识别音频信号中的音频节拍；步骤S24，根据音频节拍获取音频信号的多个音符能量；步骤S26，对多个所音符能量分别进行归一化预处理，获取多个归一化能量；步骤S28，根据多个归一化能量，生成与音频信号对应的触觉振动文件。采用该方法能够使触觉振动效果与音频信号的节拍灵活匹配，且匹配准确性高。

Description

振动生成方法、振动控制方法及其相关设备 技术领域
本发明涉及振动驱动技术领域，特别是涉及一种振动生成方法及其装置、振动控制方法及其系统、电子设备以及存储介质。
 背景技术
随着生活水平的提高，人们对音乐的听感体验要求越来越高。人们不仅希望有良好的听觉感受，同时还希望有触觉感受。
相关技术中，在音乐播放设备设置了振动马达，当在播放音乐时，控制马达振动以使音乐播放设备产生触觉振动效果。
然而，目前的振动控制方法，由于马达产生的触觉振动效果与音乐节拍相互之间不匹配，在播放音乐时，听觉感受和触觉感受之间不协调，使得用户听感体验差。
 有益效果
基于此，有必要针对上述技术问题，提供一种能够使触觉振动效果与音频信号的节拍灵活匹配、且匹配准确性高的振动生成方法及其装置、振动控制方法及其系统、电子设备以及存储介质。
本发明提供一种振动生成方法，所述方法包括：
识别音频信号中的音频节拍；
根据所述音频节拍获取所述音频信号的多个音符能量；
对多个所述音符能量分别进行归一化预处理，获取多个归一化能量；其中，所述音符能量与所述归一化能量一一对应设置；
根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
优选的，所述根据所述音频节拍获取所述音频信号的多个音符能量包括：
按照预设时间间隔获取所述音频节拍内的多个音符；其中，相邻两个所述音符间的时间间隔为所述预设时间间隔；
获取每一所述音符的音符能量。
优选的，所述获取各所述音符的音符能量的步骤包括：
在同一所述音频节拍内，根据各所述音符所对应的音频信号的幅值，计算获得各所述音符的音符能量。
优选的，所述对多个所述音符能量分别进行归一化预处理，获取多个归一化能量包括：
在同一所述音频节拍内，根据多个所述音符能量的最大值和每一所述音符的音符能量，获得各所述音符所对应的归一化能量。
优选的，所述根据所述归一化能量，生成与所述音频信号对应的触觉振动文件的步骤包括：
将高于预设能量阈值的所述归一化能量所对应的音符标记为触觉振动点；
根据各所述触觉振动点，生成与所述音频信号对应的触觉振动文件。
本发明提供一种振动控制方法，所述振动控制方法包括本发明所述的振动生成方法的步骤，还包括：
提取触觉振动文件中的触觉振动点；
根据各所述触觉振动点，生成与所述触觉振动点对应的振动驱动电压；
根据所述振动驱动电压，驱动马达振动以产生触觉振动效果。
本发明提供一种振动生成装置，其包括：
音频处理模块，用于识别音频信号中的音频节拍；以及，
计算处理模块，用于根据所述音频节拍获取所述音频信号的多个音符能量；用于对多个所述音符能量分别进行归一化预处理，获取多个归一化能量，其中，所述音符能量与所述归一化能量一一对应设置；用于根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
本发明提供一种振动控制系统，其包括：
如本发明所述的振动生成装置，用于输入触觉振动文件；
触觉振动识别模块，用于提取触觉振动文件中的触觉振动点，还用于根据各所述触觉振动点，生成与所述触觉振动点对应的振动驱动电压；以及，
振动单元，用于根据所述振动驱动电压振动以产生触觉振动效果。
本发明提供一种电子设备，其包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现本发明所述的振动生成方法的步骤和/或本发明所述的振动控制方法的步骤。
本发明提供一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现本发明所述的振动生成方法的步骤和/或本发明所述的振动控制方法的步骤。
上述振动生成方法及其装置，通过归一化优化算法对各音符的音符能量进行计算处理，以获得各音符的归一化能量，根据归一化能量生成与所述音频信号对应的触觉振动文件，实现了触觉振动效果与音频信号的节拍灵活匹配，同时，由于各音符的归一化能量高于该音符的音符能量，有效地避免了局部音频强度不够的问题，提高触觉振动效果与音频信号的节拍相互之间匹配的准确性，听觉感受和触觉感受之间协调性好，为改善用户的听感体验提供了条件；另外，将上述的振动生成方法应用到振动控制方法及其系统中，当播放音频信号时，上述的振动控制方法及其系统可根据触觉振动文件产生触觉振动效果，该触觉振动效果和音频信号的节拍高度匹配，提高了用户的听觉感受和触觉感受之间协调性，有效改善了用户的听感体验。
 附图说明
图1为本发明的振动生成方法的流程示意图；
图2为图1步骤S24的流程示意图；
图3为图1步骤S28的流程示意图；
图4为本发明的振动控制方法的流程示意图；
图5为本发明的振动生成装置的结构框图；
图6为本发明的振动控制系统的结构框图。
 本发明的实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
请参阅图1所示，本发明提供了一种振动生成方法，包括以下步骤：
步骤22，识别音频信号中的音频节拍。
步骤24，根据所述音频节拍获取所述音频信号的多个音符能量。
步骤26，对多个所述音符能量分别进行归一化预处理，获取多个归一化能量；其中，所述音符能量与所述归一化能量一一对应设置。
步骤28，根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
上述振动生成方法， 在振动生成方法中，通过归一化优化算法对各音符的音符能量进行计算处理，以获得各音符的归一化能量，根据归一化能量生成与所述音频信号对应的触觉振动文件，实现了触觉振动效果与音频信号的节拍灵活匹配，同时，由于各音符的归一化能量高于该音符的音符能量，有效地避免了局部音频强度不够的问题，提高触觉振动效果与音频信号的节拍相互之间匹配的准确性，听觉感受和触觉感受之间协调性好，有利于改善用户的听感体验。
为了更进一步理解上述方法，请同时参阅图1-3所示，下面对上述方法的各步骤进行详细描述：
步骤S22，识别音频信号中的音频节拍。
具体的，在所述步骤S22之前可以根据实际需要增加信号预处理过程，所述信号预处理过程为：导入音频文件，并获取音频文件的音频信号，提取音频信号的音频特征信息，根据所述音频特征信息识别多个节拍点的节拍时刻，根据多个所述节拍点的节拍时刻将所述音频信号获得音频节拍数列Bp，所述音频节拍数列Bp包括多个音频节拍。
步骤S24，根据所述音频节拍获取所述音频信号的多个音符能量。
具体的，所述步骤S24包括：
步骤S241，按照预设时间间隔获取所述音频节拍内的多个音符；其中，相邻两个所述音符间的时间间隔为所述预设时间间隔，记为T 0。
具体的，在所述步骤S241中，所述预设时间间隔T 0可以根据实际应用的情况进行具体确定，将同一个音频节拍内的音符表示为T(n)，其中，n为同一个音频节拍内的音符编号，比如，在本实施方式中，预设时间间隔具体设置为一个节拍时长的四分之一，即，相当于将每一节拍内有四个时间间隔相同的的音符，在此，n表示同一音频节拍内的第1、2、3、4个音符编号，同一个音频节拍内的第一个至第四个音符依次表示为T(1)、T(2)、T(3)、T(4)。
步骤S242，获取每一所述音符的音符能量。
其中，将同一所述音频节拍内的音符能量表示为P 0(n)，在本实施方式中，同一个音频节拍内的第一个至第四个音符的音符能量依次表示为P 0(1)、P 0(2)、P 0(3)、P 0(4)。
具体的，在所述步骤S242中，在同一所述音频节拍内，根据各所述音符所对应的音频信号的幅值，计算获得各所述音符的音符能量。同时，在该步骤S242之前，还需要预先检测对音频信号的采样率，记为f s。
对音符能量P 0(n)的具体计算公式表示为：
P 0(n) = sqrt(X 0(t n) 2+ X 0(t n+1) 2 +…+X 0(t n+k) 2)/(k+1)，
K=T 0×f s，
式中，X 0(t n)为第n个音符的t n时刻所对应的音频信号的幅值，X 0(t n+k)为第n个音符的t n+k时刻所对应的音频信号的幅值，其中，t n时刻为第n个音符的起始时刻，t n+k时刻为第n个音符的终止时刻，k为第n个音符的采样点数，T 0为相邻两个音符的时间间隔，f s为采样率。
第3个音频节拍的第2个音符T(2)发生在3秒时刻，且持续了0.2秒（即时间间隔T 0为0.2秒），信号采样率fs为48000。其中:
第3个音频节拍的第2个音符T(2)采样点数k=0.2×48000=9600，
依次计算各采样点的对应的时刻，第1个采样点的时刻（即起始时刻）t n =3×48000=144000，第2个采样点的时刻t n+1 =3×48000+1=144001，……，第9600个采样点的时刻（即终止时刻）t n+k=3×48000+9600=144000+9600；
根据公式X 0 (t n)= sin(2π·100·t n /f s)计算各个采样点所对应的幅值，具体的，第一个采样点时刻t n所对应的幅值X 0 (144000)= sin(2π·100·144000 /9600)，第二个采样点时刻t n+1所对应的幅值X 0 (144001)= sin(2π·100·144001 /9600)，……，第9600个采样点时刻t n+k 所对应的幅值X 0 (144000+9600)= sin[2π·100·（144000+9600）/9600]。
根据上述数值，计算获得第3个音频节拍的第2个音符T(2)所对应的的归一化能量P 0 (2)= sqrt(X 0(144000) 2+ X 0(144001) 2+…+ X 0(144000+9600) 2)/(9600+1)。
需要说明的是，在其他实施方式中，所述步骤S242还可以根据实际情况选择音符能量的具体计算方式。
步骤S26，对多个所述音符能量分别进行归一化预处理，获取多个归一化能量；其中，所述音符能量与所述归一化能量一一对应设置。
其中，将同一所述音频节拍内的归一化能量表示为P(n)，在本实施方式中，同一个音频节拍内的第一个至第四个音符的归一化能量依次表示为P(1)、P(2)、P(3)、P(4)。
具体的，所述步骤S26，在同一所述音频节拍内，根据多个所述音符能量的最大值和每一所述音符的音符能量，获得各所述音符所对应的归一化能量。
在同一个节拍内，任意一个音符所对应的归一化能量P(n)的计算公式表示为：
P(n) = P 0(n)/max(P 0(n))，
式中，P(n)表示节拍内任意一个音符对应的归一化能量，max(P 0(n))表示同一个音频节拍内的音符能量数列P 0(n)中最大的音符能量，而在本实施方式中，max(P 0(n))即音符能量P 0(1)、P 0(2)、P 0(3)、P 0(4)当中的最大值。
步骤S28，根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
具体的，所述步骤S28中包括：
步骤S281，将高于预设能量阈值的所述归一化能量所对应的音符标记为触觉振动点。
其中，预设能量阈值表示为Ph，而预设能量阈值Ph的数值可以根据实际应用的情况进行具体的设置，将预设能量阈值Ph的数值设置越小，则音频信号能够匹配的触觉振动效果越多，反之，将预设能量阈值Ph的数值设置越大，则音频信号能够匹配的触觉振动效果就越少，而在本实施方式中，预设能量阈值Ph设置为0.5·max(P(n))，max(P(n))表示同一个音频节拍内的归一化能量数列P(n)中最大的归一化能量，即，将预设能量阈值Ph设置为同一个音频节拍内的归一化能量数列P(n)中最大的归一化能量的二分之一。
具体的，分别判断每一音频节拍内的四个归一化能量（P(1)、P(2)、P(3)、P(4)）是否高于能量阈值Ph（即0.5·max(P(n))），若是，则将四个归一化能量（P(1)、P(2)、P(3)、P(4)）中高于能量阈值Ph的归一化能量所对应的音符标记为触觉振动点。
需要说明的是，通过将预设能量阈值Ph设置为0.5·max(P(n))，能够使得触觉振动点在音频信号上的分布更加均匀，在播放音频信号时，能够均匀地产生触觉振动效果，避免了振动频率过高而给用户带来振动疼痛感、振动频率过低而导致触觉振动效果不明显，有效提高用户的触觉感受，使得体验感更佳。
步骤S282，根据各所述触觉振动点，生成与所述音频信号对应的触觉振动文件。
请参阅图4所示，本发明提供了应用上述振动生成方法的振动控制方法，振动控制方法包括以下步骤：
步骤302，提取触觉振动文件中的触觉振动点。
步骤304，根据各所述触觉振动点，生成与各所述触觉振动点对应的振动驱动电压。
步骤306，根据所述振动驱动电压，驱动马达振动以产生触觉振动效果。
上述振动控制方法，当播放音频信号时，上述的振动控制方法可根据触觉振动文件产生触觉振动效果，该触觉振动效果和音频信号的音频节拍匹配度高，提高了用户的听觉感受和触觉感受之间协调性，有效改善了用户的听感体验。
应该理解的是，虽然图1-4的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图1-4中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
请参阅图5所示，本发明提供一种振动生成装置100，其包括音频处理模块101以及计算处理模块102，其中：
所述音频处理模块101，用于识别音频信号中的音频节拍；
所述计算处理模块102，用于根据所述音频节拍获取所述音频信号的多个音符能量；用于对多个所述音符能量分别进行归一化预处理，获取多个归一化能量，其中，所述音符能量与所述归一化能量一一对应设置；用于根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
所述音频处理模块101，还用于按照预设时间间隔获取所述音频节拍内的多个音符，其中，相邻两个所述音符间的时间间隔为所述预设时间间隔；用于将高于预设能量阈值的归一化能量所对应的音符标记为触觉振动点。
所述计算处理模块102，还用于根据所述音频节拍获取所述音频信号的多个音符能量，具体的，用于获取每一所述音符的音符能量；用于对多个所述音符能量分别进行归一化预处理，获取多个归一化能量，其中，所述音符能量与所述归一化能量一一对应设置；用于根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件，具体的，用于识别高于预设能量阈值的所述归一化能量所对应的音符标记为触觉振动点，用于根据各所述触觉振动点，生成与所述音频信号对应的触觉振动文件。
请参阅图6所示，本发明提供一种振动控制系统200，其包括上述的振动生成装置100、触觉振动识别模块201以及振动单元202，其中：
所述振动生成装置100，用于输入触觉振动文件。
所述触觉振动识别模块201，用于提取触觉振动文件中的触觉振动点，还用于根据各所述触觉振动点，生成与所述触觉振动点对应的振动驱动电压。
所述振动单元202，用于根据所述振动驱动电压振动以产生触觉振动效果；具体的，所述振动单元202的结构形式可以根据实际使用的情况进行具体选择，比如，在本实施方式中，该振动单元202为振动马达。
关于振动生成装置的具体限定可以参见上文中对于振动生成方法的限定，关于振动控制系统的具体限定可以参见上文中对于振动控制方法的限定，在此不再赘述。上述振动生成装置或振动控制系统中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
本发明提供一种电子设备，其包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现本发明所述的振动生成方法的步骤和/或本发明所述的振动控制方法的步骤。需要说明的是，上述的振动生成装置及振动控制系统可以应用到所述电子设备上。
本发明提供一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现本发明所述的振动生成方法的步骤和/或本发明所述的振动控制方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本发明所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器（Read-Only Memory，ROM）、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器（Random Access Memory，RAM）或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器（Static Random Access Memory，SRAM）或动态随机存取存储器（Dynamic Random Access Memory，DRAM）等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。

Claims (10)

  1. 一种振动生成方法，其特征在于，所述方法包括：
    识别音频信号中的音频节拍；
    根据所述音频节拍获取所述音频信号的多个音符能量；
    对多个所述音符能量分别进行归一化预处理，获取多个归一化能量；其中，所述音符能量与所述归一化能量一一对应设置；
    根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
  2. 根据权利要求1所述的振动生成方法，其特征在于，所述根据所述音频节拍获取所述音频信号的多个音符能量包括：
    按照预设时间间隔获取所述音频节拍内的多个音符；其中，相邻两个所述音符间的时间间隔为所述预设时间间隔；
    获取每一所述音符的音符能量。
  3. 根据权利要求2所述的振动生成方法，其特征在于，所述获取各所述音符的音符能量的步骤包括：
    在同一所述音频节拍内，根据各所述音符所对应的音频信号的幅值，计算获得各所述音符的音符能量。
  4. 根据权利要求2所述的振动生成方法，其特征在于，所述对多个所述音符能量分别进行归一化预处理，获取多个归一化能量包括：
    在同一所述音频节拍内，根据多个所述音符能量的最大值和每一所述音符的音符能量，获得各所述音符所对应的归一化能量。
  5. 根据权利要求4所述的振动生成方法，其特征在于，所述根据所述归一化能量，生成与所述音频信号对应的触觉振动文件的步骤包括：
    将高于预设能量阈值的所述归一化能量所对应的音符标记为触觉振动点；
    根据各所述触觉振动点，生成与所述音频信号对应的触觉振动文件。
  6. 一种振动控制方法，其特征在于，所述振动控制方法包括上述权1-5任一项所述的振动生成方法的步骤，所述振动控制方法还包括：
    提取触觉振动文件中的触觉振动点；
    根据各所述触觉振动点，生成与所述触觉振动点对应的振动驱动电压；
    根据所述振动驱动电压，驱动马达振动以产生触觉振动效果。
  7. 一种振动生成装置，其特征在于，所述振动生成装置包括：
    音频处理模块，用于识别音频信号中的音频节拍；以及，
    计算处理模块，用于根据所述音频节拍获取所述音频信号的多个音符能量；用于对多个所述音符能量分别进行归一化预处理，获取多个归一化能量，其中，所述音符能量与所述归一化能量一一对应设置；用于根据多个所述归一化能量，生成与所述音频信号对应的触觉振动文件。
  8. 一种振动控制系统，其特征在于，所述振动控制系统包括：
    如权利要求7所述的振动生成装置，用于输入触觉振动文件；
    触觉振动识别模块，用于提取触觉振动文件中的触觉振动点，还用于根据各所述触觉振动点，生成与所述触觉振动点对应的振动驱动电压；以及，
    振动单元，用于根据所述振动驱动电压振动以产生触觉振动效果。
  9. 一种电子设备，其包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至5中任一项所述的振动生成方法的步骤和/或权利要求6所述的振动控制方法的步骤。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至5中任一项所述的振动生成方法的步骤和/或权利要求6所述的振动控制方法的步骤。
PCT/CN2020/133228 2020-11-24 2020-12-02 振动生成方法、振动控制方法及其相关设备 Ceased WO2022110259A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011326608.5 2020-11-24
CN202011326608.5A CN112466267B (zh) 2020-11-24 2020-11-24 振动生成方法、振动控制方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2022110259A1 true WO2022110259A1 (zh) 2022-06-02

Family

ID=74799354

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/133228 Ceased WO2022110259A1 (zh) 2020-11-24 2020-12-02 振动生成方法、振动控制方法及其相关设备

Country Status (2)

Country Link
CN (1) CN112466267B (zh)
WO (1) WO2022110259A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114299991B (zh) * 2021-11-25 2025-11-14 歌尔股份有限公司 基于音频信号的振动生成方法、装置、设备及存储介质
CN114327040A (zh) * 2021-11-25 2022-04-12 歌尔股份有限公司 振动信号生成方法、装置、电子设备及存储介质
CN114995638B (zh) * 2022-05-12 2024-08-23 北京有竹居网络技术有限公司 触觉信号生成方法、装置、可读介质及电子设备
CN116185165B (zh) * 2022-06-17 2024-04-02 武汉市聚芯微电子有限责任公司 一种触觉生成方法、系统、设备以及计算机存储介质
CN116185167B (zh) * 2022-10-20 2025-10-24 瑞声开泰声学科技(上海)有限公司 音乐分轨匹配振动的触觉反馈方法、系统及相关设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011048023A (ja) * 2009-08-25 2011-03-10 Pioneer Electronic Corp 体感振動生成装置および体感振動生成方法
CN108777747A (zh) * 2018-08-09 2018-11-09 瑞声科技(新加坡)有限公司 一种振动方法、移动终端和计算机可读存储介质
US20190005935A1 (en) * 2016-03-07 2019-01-03 Yamaha Corporation Sound signal processing method and sound signal processing apparatus
CN109670074A (zh) * 2018-12-12 2019-04-23 北京字节跳动网络技术有限公司 一种节奏点识别方法、装置、电子设备及存储介质
CN110010151A (zh) * 2018-12-31 2019-07-12 瑞声科技(新加坡)有限公司 一种音频信号处理方法及设备、存储介质
CN110215607A (zh) * 2019-05-31 2019-09-10 Oppo广东移动通信有限公司 基于电刺激的按摩方法和装置
CN111726684A (zh) * 2019-03-22 2020-09-29 腾讯科技（深圳）有限公司 一种音视频处理方法、装置及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011048023A (ja) * 2009-08-25 2011-03-10 Pioneer Electronic Corp 体感振動生成装置および体感振動生成方法
US20190005935A1 (en) * 2016-03-07 2019-01-03 Yamaha Corporation Sound signal processing method and sound signal processing apparatus
CN108777747A (zh) * 2018-08-09 2018-11-09 瑞声科技(新加坡)有限公司 一种振动方法、移动终端和计算机可读存储介质
CN109670074A (zh) * 2018-12-12 2019-04-23 北京字节跳动网络技术有限公司 一种节奏点识别方法、装置、电子设备及存储介质
CN110010151A (zh) * 2018-12-31 2019-07-12 瑞声科技(新加坡)有限公司 一种音频信号处理方法及设备、存储介质
CN111726684A (zh) * 2019-03-22 2020-09-29 腾讯科技（深圳）有限公司 一种音视频处理方法、装置及存储介质
CN110215607A (zh) * 2019-05-31 2019-09-10 Oppo广东移动通信有限公司 基于电刺激的按摩方法和装置

Also Published As

Publication number Publication date
CN112466267A (zh) 2021-03-09
CN112466267B (zh) 2024-04-02

Similar Documents

Publication Publication Date Title
CN112466267B (zh) 振动生成方法、振动控制方法及其相关设备
JP2023017710A (ja) オーディオ生成方法およびシステム
Lyon et al. Auditory representations of timbre and pitch
CN106537939A (zh) 优化助听器系统中的参数的方法和助听器系统
US20260065759A1 (en) Vibration control apparatus, vibration control program, and vibration control method
EP4120262A1 (en) Audio generation methods and systems
WO2014042718A2 (en) Methods, systems, and computer readable media for synthesizing sounds using estimated material parameters
CN114299991B (zh) 基于音频信号的振动生成方法、装置、设备及存储介质
EP4120259A1 (en) Audio generation methods and systems
US20120300941A1 (en) Apparatus and method for removing vocal signal
JP7055406B2 (ja) 振動制御装置，振動制御プログラム，振動制御方法及び振動制御プログラムを記録したコンピュータ読み取り可能な記録媒体
CN109841232B (zh) 音乐信号中音符位置的提取方法和装置及存储介质
US20250058077A1 (en) Audio infusion system and method
CN106844639B (zh) 音乐匹配运动的方法及系统
CN112201227B (zh) 语音样本生成方法及装置、存储介质、电子装置
JP7729607B2 (ja) オーディオ信号パターンを用いたハプティック信号発生方法および装置
US11601768B2 (en) Method of generating sounds for reducing an effect of tinnitus and tinnitus control instrument performing the same
Weger et al. Auditory perception of spatial extent in the horizontal and vertical plane
US20240173515A1 (en) Audio generation device and method for brain dynamics audio stimulation
CN111985643B (zh) 一种生成网络的训练方法、音频数据增强方法及相关装置
JP2022104960A (ja) 振動体感装置，方法，振動体感装置用プログラム及び振動体感装置用プログラムを記録したコンピュータ読み取り可能な記録媒体
US12165622B2 (en) Audio infusion system and method
CN111326162B (zh) 一种声纹特征的采集方法和装置以及设备
Krishnan et al. The Perception of Phase Intercept Distortion and its Application in Data Augmentation
CN116909403A (zh) 一种基于人耳感知的触觉反馈实时生成方法及装置、计算机可存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20963147

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20963147

Country of ref document: EP

Kind code of ref document: A1