TWI684477B - Method for processing customized operation of music game, non-transitory computer readable medium, computer program product and system of music game - Google Patents

Abstract

A method for processing a customized operation of a music game, a non-transitory computer readable medium, a computer program product and a system of music game are disclosed. The method performs a power summation procedure to generate the spectrum-based features with the salient frequencies determined by the spectrums corresponding to the user’s recordings having a plurality types of percussive or tapping sounds and the spectrums of the testing frames corresponding to a customized operation, and then performs an identification procedure with the mentioned spectrum-based features to simulate or classify the customized operation into a default operation.

Description

Processing method for customized operation of music games, non-transitory computer readable media, computer programs Products and Music Game System

The present invention relates to a processing method for computer game operation, non-transitory computer readable media, computer program products and game systems, in particular to a processing method for customized operation of music games, non-transitory computer readable media , Computer program products and music game systems.

Beat-type or percussive music games built on large game consoles, video game consoles, computer games, handheld electronic audio-visual devices or mobile phones are still very popular. However, the beat spectrum and operation interface used by such music games are often pre-set and configured by the game developer, so in addition to the inability to customize, automatically, and instantly detect or generate beats or music games, resulting in a lack of human-computer interaction In addition to reducing the fun of the game, in order to increase the portability of such music game carrying devices or systems, game developers often choose a lightweight or miniaturized design, which often limits the size and operability of the user interface. The operating position, which in turn affects the user's accuracy of control of the preset operating interface or buttons and other input modules, user experience and gameplay Willingness to play.

Furthermore, in the processing of audio data, non-frequency-domain or frame-based audio data has complex calculations related to beat detection and cannot be completed in real-time. For background music or sound Audio, configure noise reduction procedures to separate and output noise-free audio, but also help beat recognition and operation accuracy. However, not all application types, game design and processing have taken into account that some modules and computing work are concentrated based on reasons such as increasing computing speed, accelerating game response time, and reducing power consumption of handheld electronic audio-visual devices or mobile phones. Or decentralized problems and solutions. In view of this, it is necessary to provide a technical solution that can satisfy the user's customized operation, conversion or mapping of audio sources on any surface or object, that is, it is necessary to provide user-defined audio sources and input Ways, automatic judgment of the type of sounds to achieve the related technical inventions to generate beat spectrum or music games.

One of the main objectives of the present invention is to provide a processing method for customized operation of music games, to effectively improve the lack of human-computer interaction and reduce games caused by the inability to customize, automatically, and instantly detect or generate beats or music games Fun and other issues.

Another main object of the present invention is to provide a non-transitory computer-readable memory medium with a stored program, so as to effectively improve the lack of people caused by the inability to customize, automatically, or detect or generate beats or music games. Problems such as machine interaction and reducing game fun.

Another main object of the present invention is to provide a computer program product with built-in programs to effectively improve the lack of human-computer interaction and reduce the fun of games caused by the inability to customize, automatically, detect or generate beats or music games. And other issues.

Another main object of the present invention is to provide a music game system to effectively improve the problems of lack of human-computer interaction and reduction of game fun caused by the inability to customize, automatically, real-time detect or generate beats or music games.

In order to achieve the above purpose, the processing method of the customized operation of the music game provided by the present invention is used to automatically identify the customized operation of the user to simulate or categorize it as the default operation of the music game, including multiple users The complex significant frequency determined by the complex spectrum corresponding to the pre-recorded signal, and the complex to-be-measured spectrum corresponding to the user's customized operation, a step of summing the spectrum energy to generate a complex spectrum type eigenvalue, in which the user The pre-recorded signal has a plurality of types of beat or percussion sounds, and includes a recognition process based on spectrum-type characteristic values, and simulates or classifies the user's customized operation as the default operation step.

In an embodiment of the invention, the processing method of the customized operation of the music game has a significant frequency higher than a frequency of a predetermined ratio of all spectrum energy of the corresponding type of beat or percussion sound. In addition, in another embodiment of the present invention, the step of identifying a spectrum-type characteristic value further includes: using the spectrum-type characteristic value to match the sound box corresponding to the spectrum to be measured or the spectrum to be measured with an onset The sound frame corresponding to the measured spectrum, the sound frame is divided or classified into one type of beat or percussion sound, and the sound frame corresponding to the spectrum to be measured that cannot be classified is discarded or classified to the first other type than these types, Or classify the sound frame corresponding to the spectrum under test without the starting point of the utterance into the second type other than these types, or the identification process based on the complex spectrum threshold determined by the spectrum type characteristic value and the significant frequency And, based on the logical relationship between each spectrum-type feature value and the corresponding spectrum-type threshold value, the tone-by-tone frame determines the user's customized operation as a type of beat or percussion sound. If the logical relationship is the first spectral type eigenvalue and the second spectral type eigenvalue of the spectral type eigenvalue, respectively, the first spectrum that is greater than the corresponding spectral type threshold value at the same time Type threshold and second spectrum type threshold, and when the first spectrum type characteristic value is greater than the second spectrum type characteristic value, it is determined that the user-customized operation is the first type of these types of beats or percussion sounds. If the first spectral type characteristic value is less than the second spectral type characteristic value, it is determined that the user-customized operation is the second type of beat or percussion sound of these types. If the logical relationship is that only the first spectral type characteristic value is greater than the first spectral type threshold value, then it is determined that the user-customized operation is the first type of beat or percussion sound. If the logical relationship is that only the second spectral type characteristic value is greater than the second spectral type threshold value, it is determined that the user-customized operation is the second type of beat or percussion sound. If the logical relationship is the first spectral type characteristic value and the second spectral type characteristic value, which are respectively less than the corresponding first spectral type threshold value and the second spectral type threshold value, it is determined that the user-customized operations are these types The third type of beat or percussion sound, no action or non-beat or percussion sound.

In another embodiment of the present invention, if the user customized operation, the user pre-recorded message or the spectrum to be measured has background music or sound, it further includes: the user customized operation and the user pre-recorded message respectively Perform a noise reduction process to generate and output noise-free audio after background music or sound is subtracted, and the steps of performing a noise reduction process on user customized operations and user pre-recorded messages are pre-recorded messages on the user Or the user's customized operation of the individual previous content, the conversion function model program generates the conversion function, and the user pre-recorded message or user customized operation, conversion function and the corresponding user pre-recorded message or user guest The control operation individually deducts the remaining part of the previous content, performs background music or sound subtraction to obtain noise-free audio, and is used as a user pre-recorded message and user customized operation.

In other embodiments of the present invention, the frequency spectrum corresponding to the user pre-recorded message and the spectrum to be measured corresponding to the user customized operation are performed by the user pre-recorded message and the user customized operation frame by frame Generated by the frequency domain conversion program. User customized operation Objects are generated in the game settings and operation interface, and customized operations are included and temporarily stored in the mobile phone and/or the file access location of the near-end, remote or cloud computer system with network communication capabilities and their combination The game setting and operation interface are also available for users to select or operate the beat spectrum of the displayed music game.

To achieve the above purpose, the non-transitory computer with a stored program provided by the present invention can read a memory medium, and when the computer loads the program and executes it, it can complete the aforementioned processing method of the customized operation of the music game.

In order to achieve the above purpose, the computer program product with built-in program provided by the present invention can complete the aforementioned processing method of the customized operation of the music game after the computer loads the program and executes it.

In order to achieve the above purpose, the music game system provided by the present invention can automatically recognize the user's customized operation to simulate or categorize it as the default operation of the music game, and at least includes an audio processing program module for receiving Or access to multiple user pre-recorded messages and user-customized operations, and the complex significant frequency determined by the complex frequency spectrum corresponding to the user pre-recorded signals, and the complex to-be-measured spectrum corresponding to user customized operations , Perform a spectrum energy summation procedure to generate a complex spectrum-type feature value, and use the spectrum-type feature value to perform an identification procedure, and simulate or classify a user-customized operation as the default operation.

In an embodiment of the present invention, the audio processing program module further performs the identification process using a complex spectral threshold value determined by the spectral characteristic value and the significant frequency. In another embodiment of the present invention, if the user pre-recorded message or the user's customized operation has background music or sound, the audio processing program module further includes a noise reduction program module, and the noise reduction program module is It is used to perform noise reduction procedures on user customized operation and user pre-recorded signals to generate and output reduction No noise audio except for background music or sound. Among them, the noise reduction program is to pre-record the user's pre-recorded message or the user's customized operation, the conversion function model program generates the conversion function, and the user's pre-recorded message or the user's customized operation and conversion function And the corresponding user pre-recorded message or user customized operation separately deducts the remaining part of the previous content, and performs background music or sound subtraction to obtain noise-free audio, which is individually used as user pre-recorded message and user customized化操作。 Chemical operation.

In other embodiments of the present invention, the music game system further includes: a game setting and display interface for users to operate with objects to generate user-customized operations; and an audio storage module for Store and provide audio processing modules to access user-customized operations, user pre-recorded messages or noise-free audio, and audio storage modules are configured on mobile phones and/or near-end and remote-end with network communication capabilities Or a cloud computer system and its combination; and a video processing program module, which is used to display at least the beat spectrum in the game setting and display interface when the music game is executed.

In summary, the processing method of music game customization operation provided by the present invention, non-transitory computer readable media, computer program products and music game system can effectively improve the non-customization, automatic and real-time detection Or generating beats or music games leads to problems such as lack of human-computer interaction and reduced game fun.

S11~S13, S21~S25‧Steps

U‧‧‧ user

3‧‧‧Music game system

31‧‧‧Game settings and display interface

311‧‧‧Operation options for uploading audio

32‧‧‧Audio storage module

33‧‧‧Audio processing module

34‧‧‧Video processing module

331‧‧‧ Frequency domain conversion program

332‧‧‧ Significant frequency determination procedure

333‧‧‧Spectrum energy summation procedure

334‧‧‧ threshold decision procedure

335‧‧‧Identification procedure

336‧‧‧ Noise reduction program module

41‧‧‧ Transfer function model program

42‧‧‧Noise reduction program

S‧‧‧Customized operation

TF‧‧‧ spectrum to be tested

RA, RB‧‧‧User pre-recorded messages

SA, SB‧‧‧ Spectrum

SFA, SFB‧‧‧Significant frequency

FA, FB‧‧‧Spectral characteristic value

TA, TB‧‧‧ Spectrum threshold

TR‧‧‧Conversion function

S1A, S1B‧‧‧‧Pre-recorded message content of user in 1st second

RS1A, RS1B ‧‧‧ remaining user pre-recorded message content

NS, NCA, NCB‧‧‧no noise audio

R‧‧‧Customized operation processing results

FIG. 1 is a flowchart of an embodiment of a processing method for a customized operation of a music game according to the present invention.

FIG. 2 is a flowchart of an embodiment of a recognition procedure of a processing method of a customized operation of a music game according to the present invention.

3 is a block diagram of a system architecture according to an embodiment of a music game system of the present invention.

4 is a block diagram of a system architecture according to another embodiment of the music game system of the present invention.

Please refer to FIG. 1, which is a flowchart of an embodiment of a processing method for a customized operation of a music game according to the present invention. As shown in FIG. 1, in this embodiment, in step S11, the significant frequency determined by the user's pre-recorded signal spectrum after the noise reduction process and the spectrum to be measured corresponding to the user's customized operation are performed. The overall procedure to generate spectral-type eigenvalues. Among them, in order to improve the accuracy and breadth of recognition and comparison, the user pre-recorded messages have more than two types of beats or percussion sounds. The user pre-recorded messages and, for example, the user directly or pre-recorded on the surface of any object with fingers, The audio generated by the pen or other objects being knocked or generated can be frame-by-frame (frame-by-frame), for example, a fast Fourier transform (FFT) program to perform the frequency domain conversion process to generate its spectrum and the spectrum to be tested respectively, and the user pre-records the message The frequency spectrum formed by each sound frame can further determine a significant frequency from which, for example, the frequency can be determined by selecting a frequency higher than a predetermined ratio of all spectrum energy of the corresponding type of beat or percussion sound.

According to the above embodiment, if the user customizes the operation, the user pre-recorded message or the spectrum to be measured has background music or sound, in order to carry out the subsequent identification process and improve the accuracy, a noise reduction process may be performed to generate and output a reduction In addition to the noise-free audio after the background music or sound, the relevant steps are: pre-recording the user or customizing the individual previous content of the user, such as the audio content within the first second of the beginning with 60 seconds of audio, The conversion function is generated by the conversion function model program, and the user pre-recorded message or the user's customized operation, conversion function and corresponding use The pre-recorded message or user-customized operation individually deducts the remaining 59 seconds of audio content from the first 1 second of the content, and performs background music or sound subtraction together to obtain noise-free audio, which is used as a user pre-recorded message And the user-customized operation, and may further provide the significant frequency determined by its spectrum in step S11 and the user-customized operation corresponding to the spectrum to be measured, and perform a spectrum energy summation procedure to generate a spectrum-type characteristic value.

Again, according to the embodiment, in step S12, each spectrum to be measured with a vocalization starting point is classified or classified into a type of beat or percussion sound by spectrum-type feature values, and discarded or classified in step S13 cannot be classified The sound frame corresponding to the spectrum to be measured with or without a sound origin is the first or second other type of beat or percussion sound.

It is mentioned here that in another embodiment, the aforementioned step S12 may also be performed by a spectral type threshold value determined by a spectral type characteristic value and a significant frequency, and each spectral type characteristic value and corresponding spectrum The logical relationship of the type thresholds, the tone-by-tone box determines whether the user's customized operation is a certain type of beat or percussion sound. The number of the aforementioned logical relationships depends on the number of types of user pre-recorded messages. If there are two types, it will be a function of the binary relationship greater than or less than the number of types, such as simultaneously greater than, one of which is greater than, and one of which is less than At the same time, it is less than, and according to the interval or boundary conditions, other judgments that have a union relationship with the existing logical relationship (ie, "AND" operation) can also be performed. Therefore, user-customized operations can be automatically identified, simulated, categorized, or corresponded to as preset operations by music game developers, realizing the technical concepts and effects of artificial intelligence or machine learning.

In other embodiments, user-customized operations are generated by users using objects in game settings and operation interfaces, and customized operations are temporarily stored in mobile phones and/or near-ends with network communication capabilities after being included , Remote or cloud computer systems and their combined file access locations for later Continue to access, upload and process, and the game settings and operation interface can also be used for users to select or operate the beat spectrum of the displayed music game. As mentioned above, the object may be, for example, a user's finger, pen, or other object. Therefore, the music game beat spectrum displayed in the game setting and operation interface can be referred to by the user's finger, pen or other objects in the non-game setting and operation interface when the music game is executed. The effect is similar to that of the user. The actual operation game developer presets the buttons or operation options in the game settings and operation interface. In other words, the user-customized operation is automatically recognized and simulated or classified as the default operation of the music game with the beat spectrum. The user's operation in the non-game setting and operation interface is specifically mapped and interpreted as the game. The preset operation of the setting and operation interface, so the user or gamer can be free from the limitation of the operation interface and operation mode.

Therefore, according to the embodiment of the processing method of the customized operation of the music game of the present invention, the present invention is mainly to automatically identify the frequency spectrum corresponding to each sound frame of the user's customized operation with the user pre-recorded message for automatic identification , By simulation or classified as the default operation of music games, in addition to customized, automatic, real-time detection or generation of beats or music games, it can also increase human-computer interaction and game fun, users or gamers can not be It is limited to the control accuracy derived from the operable area of the user interface of the music game such as a mobile phone or the smaller user interface of the system and the default operation position, and can be processed by the audio to be measured in the frequency domain. The operations related to beat detection are less complicated and can be completed in real time to speed up the game reaction time. At the same time, some modules and computing work can be centralized or distributed to reduce the power consumption of handheld electronic audio-visual devices or mobile phones.

Please refer to FIG. 1 and FIG. 2 together. FIG. 2 is a flowchart of another embodiment of the identification procedure of the processing method of the customized operation of the music game according to the present invention. As shown in Figure 2, in this embodiment In step S21, the identification process is performed based on the logical relationship between the spectral type characteristic value and the spectral type threshold value determined by the significant frequency, and the frame-by-sound frame determines the user's customized operation as a specific type of beat or percussion sound among various types . In steps S22~25, the aforementioned logical relationship between each spectrum-type feature value and the corresponding spectrum-type threshold value determines the user-customized operation as a type of beat or percussion sound, and the following specific steps may be included : In step S22, if the logical relationship is the first spectrum type characteristic value and the second spectrum type characteristic value of the spectrum type characteristic value, respectively, the first spectrum type threshold value and the second spectrum type value that are greater than the corresponding spectrum type threshold value respectively Threshold value, and the first spectral type characteristic value is greater than the second spectral type characteristic value, it is determined that the user-customized operation is the first type of beat or percussion sound of the type, and if the first spectral type characteristic value is less than When the second spectrum type eigenvalue is determined, the user's customized operation is the second type of beat or percussion sound of the type. If the aforementioned logical relationship is not available, it is further determined in step S23: if the logical relationship is that only the first spectral type characteristic value is greater than the first spectral type threshold value, it is determined that the user-customized operation is the first type of beat or percussion sound Or, it is further determined in step S24: if the logical relationship is that only the second spectral type characteristic value is greater than the second spectral type threshold value, it is determined that the user-customized operation is the second type of beat or percussion sound. If the logical relationship is that the first spectral type characteristic value and the second spectral type characteristic value are respectively less than the corresponding first spectral type threshold value and the second spectral type threshold value, it is determined that the user customized operation is the multiple The third type of beat or percussion sound in the category, non-action or non-beat or percussion sound. Or, if none of the aforementioned logical relationships are available, it is further determined in step S25: If the logical relationship is that the first and second spectral-type feature values are less than the first and second spectral-type thresholds, for example, a player or user The user's customized operation corresponds to the beat or percussion sound in this sound frame that does not belong to the aforementioned first or second category, or there is no movement, no beat or percussion sound in the sound frame, or a singing sound instead of Belongs to a certain plane The beat or striking sound generated on the surface of the object, the treatment of such exceptions is to determine the user's customized operation as a third type of beat or striking sound, no action or non-beat or striking sound.

Please refer to FIG. 3, which is a block diagram of a system architecture according to an embodiment of a music game system of the present invention. In this embodiment, the music game system 3 includes: a game setting and display interface 31, an audio storage module 32, an audio processing program module 33, and a video processing program module 34. Among them, the audio processing program module 33 is used to receive or access the user's customized operation S, for example, the user U operates the upload audio operation option 311 or the user U taps any desktop with a finger to generate, Temporarily stored in the audio storage module 32, the measured frequency spectrum TF obtained after being accessed and converted by the frequency domain conversion program 331, and the two user pre-recorded signals RA, RB are respectively subjected to the frequency domain conversion program 331, respectively. The user pre-recorded signals RA, RB generate corresponding spectrum SA, SB at the salient frequency determined by the salient frequency determination process 332, SFA, SFB, and the user generated by the customized operation. Measure the spectrum TF, perform the spectrum energy summation process 333 to generate spectrum-based feature values FA, FB, and use the spectrum-type feature values FA, FB and significant frequencies SFA, SFB at the threshold determined by the threshold determination process 334 The identification thresholds TA and TB are used for the identification process 335, and the customized operation of the simulated or classified user is used as the preset operation to generate the customized operation processing result R. For example, the default operation is preset by the game developer on the user's actual operation game setting and operation interface, which can be used by the user or player to select, tap or otherwise operate the physical or virtual interface or button of the music game beat spectrum.

In the above embodiment, the audio processing program module 33 further includes a noise reduction program module 336, which performs a noise reduction process on the user's customized operation S, the user pre-recorded signal RA, and RB to generate background music or sound reduction. The noise-free audio NS, NCA, NCB after the removal is customized as the user Operation S, user pre-recorded messages RA, RB. In addition, here, the audio storage module 32 can be configured in mobile phones and/or near-end, remote or cloud computer systems with network communication capabilities and combinations thereof. Therefore, the customized operation S, user pre- The recorded messages RA and RB can be temporarily stored in file access locations of mobile phones and/or near-end, remote or cloud computer systems with network communication capabilities and combinations thereof.

Please also refer to FIGS. 3 and 4, which is a block diagram of a system architecture according to another embodiment of the music game system of the present invention. In this embodiment, the audio processing program module 33 of the music game system 3 shown in FIG. 3, which includes the noise reduction program module 336, performs the noise reduction process. As far as the user pre-recorded signals RA and RB are concerned, The transfer function TR may be generated by the transfer function model program 41 by, for example, pre-recording the contents of the RA and RB of the user with a total length of 60 seconds, such as the contents S1A and S1B of the user of the first second. , And the remaining user pre-recorded message content RS1A, RS1B (in this embodiment, starting from the second second) is deducted from the user pre-recorded message RA, RB, the conversion function TR and the user pre-recorded message minus the content of the first second To the 59th second), in the noise reduction program 42 to perform background music or sound reduction, output noise-free audio NCA, NCB, as user pre-recorded messages RA, RB.

The above-mentioned embodiments are only examples, and are not intended to limit the scope of implementation of the present invention; any simple or equivalent changes and modifications made without departing from the spirit and scope of the present invention shall still fall within the scope of the patent of the present invention. .

S11~S13‧‧‧Step

Claims (18)

A processing method for customized operation of a music game is used to automatically identify a user's customized operation to simulate or classify as a preset operation of a music game, including: corresponding to a plurality of user pre-recorded messages The complex significant frequency determined by the complex spectrum, and the complex to-be-measured spectrum corresponding to the user's customized operation, a spectrum energy summation process is performed to generate complex spectrum-type eigenvalues, wherein the user pre-recorded signals have complex numbers One of the types of beats or percussion sounds; and performing an identification procedure with the spectral type feature values to simulate or classify the user-customized operation as the preset operation. For example, the processing method of the customized operation of the music game according to item 1 of the patent application, wherein the significant frequencies are higher than the frequency of the preset ratio of all the spectral energy of the beat or percussion sound of the corresponding one of the types. For example, the processing method of the customized operation of the music game according to item 1 of the patent application, wherein the step of performing the identification procedure with the spectrum-type feature values further includes: using the spectrum-type feature values to correspond the spectrums to be measured One of the sound frames or the sound frames corresponding to the spectrums to be measured in the spectrums to be tested has the starting point of utterance, and the frame-by-frames are distinguished or classified as the types of the beat or percussion sounds. For example, the processing method of the customized operation of the music game in item 3 of the patent application scope further includes: discarding or classifying the sound frame corresponding to the spectrums to be tested that cannot be classified to one of the types other than the first other types , Or classify the sound frame corresponding to the spectrums to be tested that do not have a starting point of sound in the spectrums to be tested to a second other type than the types. For example, the processing method of the customized operation of the music game in item 3 of the patent application scope further includes: performing the identification process with the complex spectrum thresholds determined by the spectral feature values and the significant frequencies; and Based on a logical relationship between each of the spectrum-type feature values and the corresponding spectrum-type threshold values, the frame-by-sound frame determines the user-customized operation as the types of the beat or percussion sounds. For example, the processing method of the customized operation of the music game of the fifth patent application further includes: if the logical relationship is a first spectral type characteristic value and a second spectral type characteristic value of the spectral type characteristic values, respectively When the first spectral type threshold value and the second spectral type threshold value are greater than the corresponding spectral type threshold values, and the first spectral type characteristic value is greater than the second spectral type characteristic value, the user is determined The customized operation is the beat or percussion sound of the first type of the types, and if the first spectral type characteristic value is less than the second spectral type characteristic value, the user customized operation is determined to be the The beat or percussion sound of a second type of these types; if the logical relationship is that only the first spectral type characteristic value is greater than the first spectral type threshold value, then the user customized operation is determined to be the first The type of the beat or percussion sound; if the logical relationship is that only the second spectral type characteristic value is greater than the second spectral type threshold value, it is determined that the user-customized operation is the second type of the beat or percussion Sound; and if the logical relationship is that the first spectral type characteristic value and the second spectral type characteristic value are respectively less than the corresponding first spectral type threshold value and the second spectral type threshold value, the decision is made The user-customized operation is one of the third types of the beat or percussion sound, the non-action or non-beat or percussion sound. For example, the processing method of the customized operation of the music game in item 1, if the user customized operation, the user pre-recorded messages or the spectrum to be tested have a background music or sound, Including: a noise reduction process for the user's customized operation and the user's pre-recorded messages Sequence to generate and output a plurality of noise-free audio after subtracting the background music or sound. For example, the processing method of the customized operation of the music game according to item 7 of the patent scope, in which the step of performing the noise reduction process on the customized operation of the user and the pre-recorded messages of the users is a pre-processing of the users A pre-recorded content of the audio message or the user customized operation, a conversion function model program is used to generate a conversion function, and the user pre-recorded messages or the user customized operation, the conversion function and the corresponding The user pre-recorded messages or the user customized operation individually deducts the remaining part of the previous content, performs the subtraction of the background music or sound to obtain the noise-free audio, and individually acts as the users Pre-recorded messages and customized operation by the user. For example, the processing method of the customized operation of the music game according to item 1 of the patent scope, in which the frequency spectra corresponding to the user pre-recorded messages and the frequency spectra to be measured corresponding to the user customized operation are The user pre-recorded messages and the user customized operation frame by frame are respectively generated by a frequency domain conversion process. For example, the processing method of the customized operation of the music game in item 1, wherein the user customized operation is generated by a user with an object on a game setting and operation interface, and the customized operation is After being included, it is temporarily stored in a mobile phone and/or a file access location of a near-end, remote or cloud computer system with network communication capabilities and a combination thereof. For example, the processing method of the customized operation of the music game according to item 10 of the patent scope, wherein the game setting and operation interface are further available for the user to select or operate one of the beat spectrum of the music game displayed. A non-transitory computer with a stored program that can read the memory medium. After the computer loads the program and executes it, it can complete the customized operation of the music game as described in any one of the patent application items 1 to 11. Approach. A computer program product with a stored program. When the computer loads the program and executes it, it can complete The method for processing the customized operation of the music game as described in any one of the items 1 to 11 of the patent scope. A music game system, which can automatically identify a user's customized operation to simulate or classify as a preset operation of a music game, at least includes: an audio processing program module for receiving or accessing a plurality of users Pre-recorded messages and the user's customized operation, and the complex significant frequency determined by the complex frequency spectrum corresponding to the user's pre-recorded messages, and the complex number of the frequency spectrum to be measured corresponding to the user's customized operation A spectral energy summation procedure is used to generate a complex spectral characteristic value, and an identification procedure is performed on the spectral characteristic value to simulate or classify the user-customized operation as the preset operation. For example, in the music game system of claim 14, the audio processing program module further performs the identification process based on the complex spectrum threshold values determined by the spectral feature values and the significant frequencies. For example, the music game system according to item 14 of the patent application, where if the user pre-recorded messages or the user's customized operation has a background music or sound, the audio processing program module further includes a noise reduction program module Group, and the noise reduction program module is used to perform a noise reduction process on the user's customized operation and the user's pre-recorded signals, respectively, to generate and output the complex number after subtracting the background music or sound Noise audio. For example, the music game system of claim 16 in which the noise reduction program is a pre-recorded message of the user or the user's customized operation of a previous section, a conversion function model program is used to generate a conversion function , And with the user pre-recorded messages or the user customized operation, the conversion function and the corresponding user pre-recorded messages or the user customized operation, individually deduct the remaining part of the previous content , The background music or sound is subtracted to obtain the noise-free audio, which are individually used as the user's pre-recorded messages and the user's customized operation. For example, the music game system of claim 16 of the patent scope further includes: a game setting and display interface for a user to operate with an object to generate the user's customized operation; an audio storage module, It is used to store and provide the audio processing module access to the user's customized operation, the user pre-recorded messages or the noise-free audio, and the audio storage module is configured on a mobile phone and/or A near-end, remote or cloud computer system with network communication capabilities and combinations thereof; and a video processing program module for displaying at least one beat spectrum when the music game is executed on the game setting and display interface.

Images