WO2023126011A1

WO2023126011A1 - Vehicle sound playback method and apparatus, storage medium and device

Info

Publication number: WO2023126011A1
Application number: PCT/CN2023/070018
Authority: WO
Inventors: 李龙晨; 许震洪; 沈东生; 马桂林; 陶然; 雷琴辉; 刘俊峰; 颜祺峰
Original assignee: 科大讯飞(苏州)科技有限公司
Priority date: 2021-12-31
Filing date: 2023-01-03
Publication date: 2023-07-06
Also published as: CN114339546A; CN114339546B

Abstract

Disclosed in the present application are a vehicle sound playback method and apparatus, a storage medium and device. The method comprises: firstly obtaining a target sound; then preprocessing the target sound to obtain a preprocessed target sound; obtaining a rotating speed of a target vehicle, and according to the rotating speed and a preset corresponding relationship between the rotating speed and a playback frequency, determining the preprocessed target sound and the playback frequency; and then playing back the preprocessed target sound in the target vehicle according to the playback frequency. According to the present application, a sound simulation effect of a whole working condition of a vehicle is realized by utilizing collected target sounds under different constant working conditions, and when a vehicle speed is at a certain constant working condition and a certain section of a sound sample needs to be repeatedly played back, no sound explosion will be generated, so that the sound sounds more natural to a user, thereby improving the driving experience of the user.

Description

A vehicle sound playback method, device, storage medium and equipment

This disclosure claims the priority of the Chinese patent application submitted to the China Patent Office on December 31, 2021, with the application number 202111679086.1, and the title of the invention is "A Method, Device, Storage Medium, and Equipment for Playing Vehicle Sound", the entire content of which is passed References are incorporated in this disclosure.

technical field

The present application relates to the field of vehicle technology, and in particular to a method, device, storage medium and equipment for playing vehicle sound.

Background technique

With the improvement of people's living standards and the rapid development of social economy, the utilization rate of cars has gradually increased, and more and more cars have entered people's lives, bringing great convenience to all aspects of people's lives. However, with the continuous development of automobile technology and manufacturing process, the noise level in the car has been reduced year by year. However, the quiet driving environment also makes the car lack the sense of power brought by the sound of the engine, which reduces the driving pleasure of some consumers. In this regard, the method of simulated playback of vehicle sound effects can be used to actively emit sound through the speakers in the car, thereby improving the sound quality in the car and enhancing driving pleasure.

At present, there are usually two methods for vehicle sound simulation and playback: one is the formula method, although the design of this method is simple, but it can only generate a limited number of single-frequency signals, and it is impossible to achieve full-band sound simulation; the other is the particle method. Synthesis algorithm, although the algorithm can realize full-band sound simulation, but the design is complicated, and if the design is not good, it will produce "popping sound" when switching different sound segments, and because the duration of the segmented sound segments is very short, when the vehicle operating condition In a certain constant working condition, playing a certain short section of sound repeatedly will sound unnatural, which reduces the user's driving experience.

Therefore, how to realize the intelligent simulated playback of vehicle sound to make the user's sense of hearing more natural and to improve the user's driving experience is a technical problem to be solved urgently at present.

Contents of the invention

The main purpose of the embodiments of the present application is to provide a vehicle sound playback method, device, storage medium and equipment, which can realize intelligent simulation playback of vehicle sounds, make the user's sense of hearing more natural, and further improve the user's driving experience.

An embodiment of the present application provides a method for playing vehicle sound, including:

Get target sound;

Preprocessing the target sound to obtain the preprocessed target sound;

Acquiring the rotational speed of the target vehicle, and determining the preprocessed target sound and the playing frequency according to the rotational speed of the target vehicle and the correspondence between the preset rotational speed and the playing frequency;

The preprocessed target sound is played in the target vehicle according to the playback frequency.

In a possible implementation manner, the target sound includes multiple pieces of preprocessed sound data, and the multiple pieces of sound data are the interior sounds collected separately when the vehicle of the preset model type runs to multiple different constant working conditions data.

In a possible implementation manner, the acquiring the rotational speed of the target vehicle, and determining the preprocessed target sound and the playback frequency according to the rotational speed of the target vehicle and the preset correspondence between the rotational speed and the playback frequency include: :

selecting at least one piece of preprocessed sound data from the plurality of pieces of preprocessed sound data according to the rotational speed of the target vehicle;

The playback frequency of the at least one piece of preprocessed sound data is determined according to the rotation speed of the target vehicle and the preset correspondence between the rotation speed and the playback frequency.

In a possible implementation manner, the multiple different constant working conditions include an idling working condition.

In a possible implementation manner, the preprocessing the target sound to obtain the preprocessed target sound, so as to eliminate the pop sound generated when playing the target sound, includes:

Dividing the target sound into a first sound segment and a second sound segment in sequence; the sound time lengths of the first sound segment and the second sound segment are both greater than or equal to a first preset duration;

The sound of the first second preset duration of the first sound segment is set to fade-in mode to obtain an updated first sound segment; and the sound of the second preset duration of the second sound segment is set to fade out mode to obtain the updated second sound segment;

The sequence of the updated first sound segment and the updated second sound segment are exchanged, and an overlap of a second preset duration is set between the two to obtain a pre-processed target sound to eliminate the need to play the Popping sound when the target sounds.

In a possible implementation manner, the second preset duration is one-third or one-half of the first preset duration, or the second preset duration is not longer than the first preset duration half of the duration.

In a possible implementation manner, after the acquisition of the rotational speed of the target vehicle, the method further includes:

judging whether the speed of the target vehicle is in a speed overlapping interval;

If so, the preprocessed target sound corresponding to the previous speed interval and the preprocessed target sound corresponding to the next speed interval of the overlapping speed interval are simultaneously played.

In a possible implementation manner, the simultaneously playing the preprocessed target sound corresponding to the previous speed interval and the preprocessed target sound corresponding to the next speed interval in the speed overlapping interval includes:

The preprocessed target sound corresponding to the previous speed interval of the speed overlapping interval is played in fade-out mode, and the preprocessed target sound corresponding to the subsequent speed interval of the speed overlapping interval is played in fade-in mode.

In a possible implementation, the method further includes:

When the vehicle speed of the target vehicle is in a preset stable working condition, within the range of a third preset duration, the volume of the preprocessed target sound played in the target vehicle is reduced to a preset volume threshold.

In a possible implementation, the method further includes:

Obtaining a first preset number of accelerator pedal strokes of the target vehicle;

According to the first preset number of accelerator pedal strokes, it is determined whether the target vehicle is in an acceleration or deceleration state; if yes, the preprocessed target sound is played in the target vehicle.

In a possible implementation manner, after acquiring the first preset number of accelerator pedal strokes of the target vehicle, the method includes:

By sequentially shifting the first preset number of accelerator pedal strokes of the target vehicle one bit forward, the shifted first preset number of accelerator pedal strokes are obtained;

The determining whether the target vehicle is in an acceleration or deceleration condition according to the first preset number of accelerator pedal strokes; if so, playing the preprocessed target sound in the target vehicle includes:

calculating the difference between the newest accelerator pedal stroke and the most delayed accelerator pedal stroke among the shifted first preset number of accelerator pedal strokes; if the difference is smaller than a first preset threshold value, then determining the target vehicle It is in a deceleration state, or if the difference is greater than a second preset threshold, it is determined that the target vehicle is in an acceleration state, and the preprocessed target sound is played in the target vehicle.

In a possible implementation, the method further includes:

Obtaining a second preset number of instantaneous vehicle speeds of the target vehicle;

According to the second preset number of instantaneous vehicle speeds, it is determined whether the target vehicle is in an acceleration or deceleration state; if yes, the preprocessed target sound is played in the target vehicle.

In a possible implementation manner, after acquiring the second preset number of instantaneous vehicle speeds of the target vehicle, the method includes:

By shifting the second preset number of instantaneous vehicle speeds one bit forward sequentially, the shifted second preset number of instantaneous vehicle speeds is obtained, and the average of the shifted second preset number of instantaneous vehicle speeds is calculated value; and further calculate the average value of the average value of the instantaneous vehicle speed after the shift of the third preset number; the average value of the instantaneous vehicle speed after the shift of the third preset number is compared to the The mean is lagged;

The determining whether the target vehicle is in an acceleration or deceleration condition according to the second preset number of instantaneous vehicle speeds; if so, playing the preprocessed target sound in the target vehicle includes:

calculating the difference between the average value of the second preset number of shifted instantaneous vehicle speeds and the average value of the third preset number of shifted instantaneous vehicle speeds; if the difference is greater than the third preset If the threshold is set, then it is determined that the target vehicle is in an acceleration condition, or, if the difference is less than the fourth preset threshold, it is determined that the target vehicle is in a deceleration condition, and the pre-processing is played in the target vehicle after the target sound.

The embodiment of the present application also provides a vehicle sound playback device, the device comprising:

an acquisition unit, configured to acquire the target sound;

a preprocessing unit, configured to preprocess the target sound to obtain the preprocessed target sound;

The first determining unit is configured to acquire the rotational speed of the target vehicle, and determine the preprocessed target sound and the playback frequency according to the rotational speed of the target vehicle and the preset correspondence between the rotational speed and the playback frequency;

A playback unit, configured to play the preprocessed target sound in the target vehicle according to the playback frequency.

In a possible implementation manner, the first determination unit includes:

A selection subunit, configured to select at least one piece of preprocessed sound data from the plurality of pieces of preprocessed sound data according to the rotational speed of the target vehicle;

The determination subunit is configured to determine the playback frequency of the at least one piece of preprocessed sound data according to the rotation speed of the target vehicle and the preset correspondence between the rotation speed and playback frequency.

In a possible implementation manner, the preprocessing unit includes:

Dividing subunits, used to sequentially divide the target sound into a first sound segment and a second sound segment; the sound time lengths of the first sound segment and the second sound segment are both greater than or equal to a first preset duration;

A subunit is set, which is used to set the sound of the first second preset duration of the first sound segment to fade-in mode to obtain an updated first sound segment; and set the rear second preset of the second sound segment The duration sound is set to fade-out mode, and the updated second sound segment is obtained;

The exchange subunit is used to exchange the order of the updated first sound segment and the updated second sound segment, and set a second preset duration of overlap between the two to obtain the preprocessed target sound, to eliminate popping sounds when the target sound is played.

In a possible implementation manner, the device further includes:

A judging unit, configured to judge whether the speed of the target vehicle is located in a speed overlapping interval after the speed of the target vehicle is acquired;

A switching unit, configured to simultaneously play the preprocessed target sound corresponding to the previous speed interval of the speed overlapping interval and the preprocessed sound corresponding to the subsequent speed interval if it is determined that the speed of the target vehicle is located in the speed overlapping interval. The processed target sound.

In a possible implementation manner, the switching unit is specifically configured to:

In a possible implementation manner, the device further includes:

A reducing unit, configured to reduce the volume of the preprocessed target sound played in the target vehicle to within the range of a third preset duration when the vehicle speed of the target vehicle is in a preset stable working condition Preset volume threshold.

In a possible implementation manner, the device further includes:

a first acquiring unit, configured to acquire a first preset number of accelerator pedal strokes of the target vehicle;

The second determining unit is configured to determine whether the target vehicle is in an acceleration or deceleration state according to the first preset number of accelerator pedal strokes; if so, play the preprocessed target in the target vehicle sound.

In a possible implementation manner, the device further includes:

The first shifting unit is configured to sequentially shift the first preset number of accelerator pedal strokes of the target vehicle forward one bit after the acquisition of the first preset number of accelerator pedal strokes of the target vehicle After that, the first preset amount of accelerator pedal travel after shifting is obtained;

The second determining unit is specifically used for:

In a possible implementation manner, the device further includes:

a second acquiring unit, configured to acquire a second preset number of instantaneous vehicle speeds of the target vehicle;

A third determining unit, configured to determine whether the target vehicle is in an acceleration or deceleration state according to the second preset number of instantaneous vehicle speeds; if so, play the preprocessed target sound in the target vehicle .

In a possible implementation manner, the device further includes:

The second shifting unit is used to obtain the shifted second preset number of instantaneous vehicle speeds by sequentially shifting the second preset number of instantaneous vehicle speeds one bit forward, and calculate the second preset number The average value of the instantaneous vehicle speed after the shift; and further calculate the average value of the average value of the instantaneous vehicle speed after the shift of the third preset number; the average value of the instantaneous vehicle speed after the shift of the third preset number is compared The average value of the instantaneous vehicle speed after the shift is lagged;

The third determining unit is specifically used for:

The embodiment of the present application also provides a vehicle sound playback device, including: a processor, a memory, and a system bus;

The processor and the memory are connected through the system bus;

The memory is used to store one or more programs, and the one or more programs include instructions, and the instructions, when executed by the processor, cause the processor to execute any one of the above-mentioned vehicle sound playing methods. Way.

An embodiment of the present application also provides a vehicle, the vehicle includes the above-mentioned vehicle sound playback device.

The embodiment of the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are run on the terminal device, the terminal device is made to perform the above-mentioned vehicle sound playing method. any implementation of the .

The embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any one of the above methods for playing vehicle sound.

A vehicle sound playback method, device, storage medium, and equipment provided in the embodiments of the present application, first obtain the target sound; then perform preprocessing on the target sound to obtain the preprocessed target sound; then, obtain the speed of the target vehicle, and According to the rotational speed of the target vehicle and the corresponding relationship between the preset rotational speed and the playback frequency, the preprocessed target sound and the playback frequency are determined; furthermore, the preprocessed target sound can be played in the target vehicle according to the playback frequency. It can be seen that this application not only realizes the sound simulation effect of the vehicle's full working condition by using the collected target sounds of the preset model vehicle running under different constant working conditions, but also needs to repeatedly play a certain sound sample when the vehicle speed is in a certain constant working condition At the same time, there will be no popping sound, which makes the user's sense of hearing more natural, thereby improving the user's driving experience.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present application, those of ordinary skill in the art can also obtain other drawings based on these drawings without creative effort.

Fig. 1 is a schematic flow chart of a vehicle sound playing method provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of playing target sounds in different speed ranges provided by the embodiment of the present application;

Fig. 3 is a schematic composition diagram of a vehicle sound playback device provided by an embodiment of the present application.

Detailed ways

With the continuous development of automobile technology and manufacturing process, the noise level in the car has been reduced year by year. However, the quiet driving environment also makes the car lack the sense of power brought by the sound of the engine, reducing the driving pleasure of some consumers. In this regard, the method of simulated playback of vehicle sound effects can be used to actively emit sound through the speakers in the car, thereby improving the sound quality in the car and enhancing driving pleasure.

In this regard, the existing vehicle sound effect sound simulation playback methods generally include the following two types:

The first is the formula method, such as deriving the order frequency, amplitude, and phase to be enhanced according to the engine speed signal, and then calculating the output value based on the sine wave formula, but this method can only generate a limited number of single-frequency signals, which cannot be realized The full-band sound simulation results in a poor sound simulation playback effect in the vehicle.

The second is the particle synthesis algorithm. The sound of all working conditions is pre-recorded, and then the sound is cut into multiple sound segments and pre-processed. Each sound segment seems to be a sound of a certain constant working condition. According to different working conditions Play different sound segments. Although the algorithm can realize full-band sound simulation, the design is complicated. If the design is not good, it will produce "popping sound" when switching different sound segments. When the vehicle operating condition is in a certain constant operating condition, playing a certain short section of sound repeatedly will sound unnatural, which reduces the user's driving experience.

Therefore, how to realize the intelligent simulated playback of vehicle sound to make the user's sense of hearing more natural, so as to improve the user's driving experience is a technical problem to be solved urgently.

In order to solve the above-mentioned defects, the present application provides a method for playing vehicle sound. First, the target sound to be played is obtained; then the target sound is preprocessed to obtain the preprocessed target sound; The rotational speed of the target vehicle and the corresponding relationship between the preset rotational speed and the playback frequency determine the playback frequency of the target sound; furthermore, the target sound can be played in the target vehicle according to the playback frequency. It can be seen that this application not only realizes the sound simulation effect of the vehicle's full working condition by using the collected target sounds of the preset model vehicle running under different constant working conditions, but also needs to repeatedly play a certain sound sample when the vehicle speed is in a certain constant working condition At the same time, there will be no popping sound, which makes the user's sense of hearing more natural, thereby improving the user's driving experience.

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

first embodiment

Referring to Fig. 1, it is a schematic flow chart of a method for playing vehicle sound provided by this embodiment, the method includes the following steps:

S101: Obtain a target sound to be played.

In this embodiment, any sound that is simulated and played in the car is defined as the target sound to be played, and the vehicle that plays the target sound is defined as the target vehicle. In some embodiments, the target sound may include multiple pieces of preprocessed sound data, and the multiple pieces of sound data are in-vehicle sound data collected respectively when the vehicle of a preset model type is running in multiple different constant working conditions. Preset models of vehicles can be selected arbitrarily. For example, when the vehicle of the preset model type is running to a different constant working condition, the noise data inside the vehicle is collected as the target sound to be played to perform the subsequent step S102. In some embodiments, the target sound generally includes multiple.

For example, it is possible to collect the target sound when the vehicle of the preset model is running to the idling condition, constant low-speed condition, constant medium-speed condition, and constant high-speed condition, as shown in Table 1 below:

Table 1

Among them, RecordRate1, RecordRate2, RecordRate3, and RecordRate4 are sampling rates, and the unit is sample/s, which is used to indicate that different sampling rates can be used to collect the target sound under the above four working conditions. Alternatively, in some embodiments, these sampling rates may also be the same, or any of them may be the same.

It should be noted that, in this embodiment, except for the idling condition, the number of constant-speed conditions is not limited to the three in Table 1 above, and the speed and rotation speed of different conditions should be significantly different. Moreover, the collection time is not limited to the recommended time of 3-5 seconds, and the specific value can be set according to the actual situation, which is not set in this application.

S102: Preprocessing the target sound to obtain the preprocessed target sound.

In this embodiment, after the target sound to be played is acquired through step S101, in order to eliminate the popping sound that may be generated when the target sound is played, so as to make the user's sense of hearing more natural, the target sound can be preprocessed to obtain the preprocessed The final target sound is used to execute the subsequent step S103.

Specifically, an optional implementation is that first, the target sound can be divided into a first sound segment and a second sound segment in sequence; wherein, the sound time lengths of the first sound segment and the second sound segment are both greater than or It is equal to the first preset duration, and the specific value of the first preset duration can be set according to actual conditions and experience values, which is not limited in this application, for example, the first preset duration can be set to 1 second. At this time, assuming that the target sound is 3 seconds long, the intercepted first sound segment and the second sound segment may be 1.5 seconds respectively. Then, the sound of the first second preset duration of the first sound segment can be set to fade-in mode to obtain the updated first sound segment; and the sound of the second preset duration of the second sound segment can be set to fade-out mode , to obtain the updated second sound segment, wherein the specific value of the second preset duration can be set according to the actual situation and experience value, which is not limited in this application, for example, the second preset duration can be the first preset One-third or one-half of the duration, or not more than one-half, etc. For example, when the first preset duration is set to 1 second, the second preset duration can be set to 0.5 seconds. Then, the order of the updated first sound segment and the updated second sound segment can be exchanged, and a second preset duration (such as 0.5 seconds) of overlapping can be set between the two to obtain the preprocessed target sound, to eliminate popping sounds when the target sound is played. It can be seen that the preprocessed target sound is shorter than the original template sound by the second preset duration (for example, 0.5 seconds).

S103: Obtain the rotation speed of the target vehicle, and determine the preprocessed target sound and the playback frequency according to the rotation speed of the target vehicle and the preset corresponding relationship between the rotation speed and the playback frequency.

Wherein, the content of the rotational speed of the target vehicle is determined according to the type of the target vehicle. For example, when the target vehicle is a petrol vehicle, the rotational speed of the target vehicle can be the rotational speed of the engine, and the acquisition method is not limited, for example, it can be obtained by using an on-board sensor; Alternatively, when the target vehicle is an electric car, the speed of the target vehicle may be the virtual speed of the electric car, and the specific acquisition method is not limited, for example, it may be provided by the system of the target vehicle.

When the target sound includes multiple pieces of preprocessed sound data, and the multiple pieces of sound data are in-vehicle sound data collected separately when the vehicle of the preset model type runs to multiple different constant working conditions, an optional The implementation method is that the specific implementation process of this step S103 is: first, according to the speed of the target vehicle, select at least one piece of pre-processed sound data from a plurality of pieces of pre-processed sound data; and then according to the speed of the target vehicle and preset The corresponding relationship between the rotation speed and the playback frequency determines the playback frequency of at least one piece of preprocessed sound data for subsequent sound playback.

The method for determining the preprocessed target sound to be played and its playing frequency according to the rotational speed of the target vehicle is specifically described as follows.

S104: Play the preprocessed target sound in the target vehicle according to the playback frequency.

It should be noted that since the time length of the target sound collected in this application is much longer than the sound segments segmented by the current particle synthesis algorithm, it can improve the ability of the target vehicle to repeatedly play a certain sound segment under a certain constant working condition. realism. When the working condition of the target vehicle changes, the rate of the preprocessed target sound played (that is, the playing frequency) can be changed in real time according to the rotational speed signal of the target vehicle.

Specifically, before executing this step S103, it is necessary to configure the hardware in advance, and the DAC pin of the control chip is connected to the signal input pin of the analog power amplifier chip. In this way, in terms of software, the timer interrupt function can be used to , and transmit the preprocessed target sound data to the DAC pin one by one in real time and output to the analog power amplifier chip. At the same time, after obtaining the speed of the target vehicle, the time interval of timer interruption is controlled by the real-time change of the speed, so that the preprocessed target sound at idle speed or constant speed can be played with a variable rate of hearing. That is, the corresponding relationship between the preset rotation speed and the playback frequency at each level is used to control the time interval of the timer interruption, that is, to determine the playback frequency of the pre-processed target sound. In this way, each time the timer interruption function is triggered, the Pass a preprocessed target sound data to the DAC pin, and then transmit it to the analog power amplifier chip through the DAC pin, so that the speaker in the target connected to the output end of the analog power amplifier chip can sound and play the preprocessed target sound. Therefore, according to the real-time change of the speed signal of the target vehicle, the trigger time interval of the timer interrupt function can also be used to realize the effect of changing the speed of the preprocessed target sound at idle speed or constant speed.

Among them, the corresponding relationship between the preset rotation speed and the playback frequency can be set according to the actual situation. For example, the playback frequency corresponding to 600 rotation speed is 0.01 seconds, the playback frequency corresponding to 700 rotation speed is 0.009 seconds, and the playback frequency corresponding to 800 rotation speed is 0.001 seconds. Seconds, etc., so that when the rotational speed of the target vehicle is 600 rpm, the timer interrupt function can be triggered every 0.01 second, that is, the preprocessed target sound is played in the target vehicle every 0.01 second.

It can be seen that the higher the speed of the target vehicle, the shorter the timer interruption interval should be, so that the DAC output signal will be faster, and the speaker at the output end of the analog power amplifier chip will play the preprocessed target sound at a variable rate.

In some embodiments, the multiple different constant operating conditions include at least four operating conditions, and the specific implementation process of "obtaining the target sound to be played" in the above step S101 may include: when the vehicle of the preset model type is in the idle state, Under constant low-speed working conditions, constant medium-speed working conditions, and constant high-speed working conditions, respectively collect the first target sound, the second target sound, the third target sound, and the fourth target sound to be played, and record the corresponding first speed, The second rotation speed, the third rotation speed, and the fourth rotation speed are shown in the above table 1; wherein, when the preset vehicle gear is fixed, when it is in a constant low-speed working condition, a constant medium-speed working condition, or a constant high-speed working condition , it means that the speed of the vehicle is constant, which means that the speed of the vehicle is also stable. Therefore, when the vehicle of the preset model is in the idling condition, the constant low-speed condition, the constant medium-speed condition, and the constant high-speed condition, not only Different target sounds can be collected correspondingly, and four fixed rotational speeds of the vehicle can be recorded and defined as the first rotational speed, the second rotational speed, the third rotational speed, and the fourth rotational speed respectively. Further, after obtaining the rotational speed of the target vehicle, it is necessary to determine whether the rotational speed of the target vehicle is in the rotational speed overlapping interval, and if so, it is necessary to simultaneously play the preprocessed target sound corresponding to the previous rotational speed interval of the rotational speed overlapping interval and the subsequent rotational speed interval The corresponding preprocessed target sound. Moreover, in the process of simultaneous playback, the preprocessed target sound corresponding to the previous speed interval of the speed overlapping interval can be played in a fade-out mode, and the preprocessed target sound corresponding to the subsequent speed interval of the speed overlapping interval can be played in a fade-out mode. Playback in fade-in mode for better playback.

Wherein, it should be noted that the rotational speed overlapping interval may be an overlapping portion of the playing rotational speed ranges of any two preprocessed target sounds. For example: assume that the preset model vehicle is vehicle A, and the target vehicle is vehicle B. In the above embodiment, the first rotation speed is 600 rpm (idling mode), the second rotation speed is 1500 rpm, and the third rotation speed is 2500 rpm. Four revs at 3,500 rpm. When obtaining the speed of car B, it is necessary to determine the preprocessed target sound to be played. In some embodiments, it can be determined directly according to the rotational speed. For example, when the acquired rotational speed of vehicle B is within a certain range around 1500 rpm, the preprocessed second target sound is selected to be played. When the acquired vehicle B When the rotating speed is within a certain range around 2500 rpm, the preprocessed third target sound is selected to be played. In other embodiments, it can be determined proportionally according to the speed range. For example, the speed range of vehicle B is from 800 rpm (during idle speed) to 8000 rpm, and when the speed of vehicle B is 3200, 5600, 8000, the speed of vehicle A corresponds to 1500. 2500, 3500, or after deducting part of the upper limit (for example, minus 1200) proportionally, when the speed of car B is 2800, 4800, 6800, the corresponding speed of car A is 1500, 2500, 3500, when the speed of car B is obtained within a certain range around 2800 When it is inside, the preprocessed second target sound is selected to be played, and when the acquired speed of vehicle B is within a certain range around 4800 rpm, the preprocessed third target sound is selected to be played. For any preprocessed target sound, there is a playback speed range, for example, it can be centered on the corresponding speed (for example, for the convenience of description, take the determination of the target sound directly according to the speed as an example, the corresponding speed can be preprocessed Acquisition speed of the target sound after that), several speed ranges before and after it (for example, plus or minus 600), that is, when the speed of the acquired target vehicle B is at any value between 900-2100, it is selected to collect when the speed of vehicle A is 1500 The preprocessed second target sound is used as the target sound to be played. The preset corresponding relationship between the rotational speed and the playback frequency means that when the rotational speed of the second car is 1500, the preprocessed target sound is played normally; , the more the frequency decreases. When the speed of car B is greater than 1500, then increase its frequency to play, and the higher the speed, the more the frequency increases.

Similarly, when the rotational speed of the second car is 1900-3100, the preprocessed third target sound is selected for playback. At this time, the rotation speed overlap interval is 1900 to 3100 rotation speeds. Within this rotation speed range, the second pre-processed target sound corresponding to the previous speed range and the pre-processed third target sound corresponding to the next speed range will be played at the same time. At the same time, a fade-in and fade-out control, that is, a gain control, will be added. The third target sound gain (ie volume) will gradually increase, and the preprocessed second target sound gain will gradually decrease until it is 0. The rotational speed overlapping interval is the overlapping portion of the playing rotational speed ranges of any two preprocessed target sounds. It can be understood that in practical applications, the setting of the speed range must also ensure that there are overlapping intervals.

As another example, when vehicle A is at speeds of 1500, 2000, and 2500, the preprocessed target sounds to be played are collected respectively as sound A, sound B, and sound C. Assuming that the speed range of the preprocessed target sound played by vehicle B is plus or minus 400 rpm, the playback speed range of sound A is from 1100 to 1900 rpm, and the playback speed range of sound B is from 1600 to 2400 rpm. The playback speed range of C is from 2100 to 2900 rpm, and the overlapping speed range includes 1600-1900 rpm and 2100-2400 rpm. Then, when the rotational speed of vehicle B reaches 1100 rpm, sound A starts to be played, and as the rotational speed of vehicle B increases, the playing frequency of sound A gradually increases, and the volume of sound A gradually increases until the rotational speed of vehicle B reaches At 1500 rpm, the playback frequency of sound A reaches normal. When the speed of vehicle B gradually increases from 1500 to 1900 rpm, its playback frequency continues to increase. The volume is also gradually reduced until the speed of vehicle B reaches 1900 rpm, the lowest volume of sound A is 0, that is, the playback of sound A is stopped, and playback in fade-out mode is realized. Among them, the interval between 1600 rpm and 1900 rpm is the speed overlap interval, and sound A and sound B need to be played overlappingly. When the speed of vehicle B reaches 1600 rpm, the sound B starts to be played in a manner of gradually increasing frequency, and the volume of sound B is gradually increased until the speed of vehicle B reaches 2000 rpm, and the frequency of sound B is normal. And when the rotational speed of vehicle B gradually increases from 2000 to 2400 rpm, its playing frequency continues to increase, and when entering the next overlapping interval of rotational speed (that is, starting at 2100 rpm), the volume of playing sound B also gradually decreases until the vehicle B's When the rotation speed reaches 2400 rpm, the lowest volume of sound B reaches 0, that is, sound B stops playing, and sound B is played in fade-out mode.

Similarly, the interval from 2100 rpm to 2400 rpm is also an overlapping interval of rotational speeds, and sound B and sound C need to be played overlappingly.

In this implementation, the switching between adjacent processed target sounds needs to achieve a smooth and natural transition. Based on the above statement, it can be seen that the time interval of the timer interruption is linearly inversely proportional to the current speed information of the target vehicle, that is, the higher the speed, the shorter the time interval of the timer interruption, the faster the DAC output signal, and the faster the sound playback rate. faster.

On this basis, as shown in Figure 2, the idling condition, constant low-speed condition, constant medium-speed condition, and constant high-speed condition respectively correspond to the preprocessed first target sound, second target sound, and third target sound. The sound, the fourth target sound, and the speed corresponding to the idling condition are between PRM1 and RPM2, the speed corresponding to the constant low speed condition is between RPM3 and RPM4, and the speed corresponding to the constant medium speed condition is between RPM5 and RPM6 , the speed corresponding to the constant high-speed condition is between RPM7 and RPM8 as an example, preferably the median value of each interval, and the engine speed of the target vehicle satisfies: RPM1<RPM2<RPM3<RPM4<RPM6<RPM7<RPM8. Then the playback target sound in this speed range is as follows:

1) When the engine speed of the target vehicle is between RPM1 and RPM2, only the preprocessed first target sound corresponding to the idling condition is played.

2) When the engine speed of the target vehicle is between RPM2 and RPM3, simultaneously play the preprocessed first target sound corresponding to the idling condition and the preprocessed second target sound corresponding to the constant low speed condition.

3) When the engine speed of the target vehicle is between RPM3 and RPM4, only the preprocessed second target sound corresponding to the constant low speed condition is played.

4) When the engine speed of the target vehicle is between RPM4 and RPM5, play the preprocessed second target sound corresponding to the constant speed constant low speed condition and the preprocessed third target sound corresponding to the constant medium speed condition sound.

5) When the engine speed of the target vehicle is between RPM5 and RPM6, only the preprocessed third target sound corresponding to the constant medium speed condition is played.

6) When the engine speed of the target vehicle is between RPM6 and RPM7, simultaneously play the preprocessed third target sound corresponding to the constant medium speed condition and the preprocessed fourth target sound corresponding to the constant high speed condition.

7) When the engine speed of the target vehicle is between RPM7 and RPM8, only the preprocessed fourth target sound corresponding to the constant high speed condition is played.

Among them, when playing two pre-processed target sounds at the same time, the former pre-processed target sound should add the fade-out algorithm, and the latter pre-processed target sound should add the fade-in algorithm. The values of parameters RPM1, RPM2, RPM3, RPM4, RPM5, RPM6, RPM7, and RPM8 can be set according to actual conditions, and this embodiment of the present application does not limit this. RPM8 should be greater than the maximum speed of the vehicle engine to meet the target sound and can be produced under all working conditions of the vehicle. For example: Assuming that the engine speed range of a certain model vehicle is between 700 and 6000 rpm, then RPM1=400, RPM2=1200, RPM3=1500, RPM4=2500, RPM5=3000, RPM6=3500, RPM7= 5000, RPM8=7000, etc.

It should be noted that due to the existing vehicle sound simulation playback method, there is only the function of turning on or off the sound effect, lacking more customization functions, and the user's driving experience is poor. Therefore, in this embodiment, in order to further improve the driving experience of the driver, the playback of the pre-processed target sound can be started when the target vehicle is in an acceleration or deceleration condition, so as to avoid the driver driving at a constant speed for a long time (such as in driving on the highway) caused hearing fatigue. For the specific playback process, please refer to the above steps S101-S104. Next, this application will introduce in detail the process of how to determine when to start the playback of the preprocessed target sound:

An optional implementation is to first determine whether the speed of the target vehicle is in a relatively stable state, that is, to determine whether the driver of the target vehicle is driving at a constant speed for a long time (such as driving on a highway), and if so, Then, in order to avoid hearing fatigue, the volume of the preprocessed target sound that has been played in the target vehicle can be reduced to the preset volume threshold within the third preset duration, and wait until the subsequent target vehicle is accelerating or accelerating again. When decelerating, turn on the playback of the target sound.

Wherein, the specific value of the range of the third preset duration can be set according to the actual situation and experience value, which is not limited in this application, for example, the range of the third preset duration can be set to 5 seconds to 10 seconds, etc. Moreover, the specific value of the preset volume threshold can also be set according to actual conditions and experience values, and the present application does not limit it. For example, the preset volume threshold can be set to 0, etc., such as within 5 seconds to 10 seconds. In the target vehicle, the volume of the pre-processed target sound that has been played in the target vehicle is reduced to 0, even if the virtual sound wave disappears.

On this basis, when the target vehicle is again in the acceleration, deceleration, or throttle change condition, the preprocessed target sound can be played again when the preset condition is met. Specifically, an optional implementation method is to obtain the first preset number of the target vehicle (here, it is defined as P _num , the specific value is not limited, and can be set according to the actual situation, for example, The value is 10) of the accelerator pedal stroke (here it is defined as P), and then according to the accelerator pedal stroke P of the first preset quantity P _num , it is determined whether the target vehicle is in an acceleration or deceleration condition; if so, in the target The pre-processed target sound is played in the vehicle.

In this implementation, after obtaining the P _num accelerator pedal stroke of the target vehicle, define the accelerator pedal stroke history data array as P _array [P _num -1], and then move it forward one bit in turn to obtain the shift Accelerator pedal stroke of P _num after digit, that is, P _array [0]=P _array [1], P _array [1]=P _array [2],..., P _array [P _num -2]=P _array [P _num -1], then P _array [P _num -1]=P _new , wherein the size of P _array is P _num . Furthermore, the difference between the latest accelerator pedal stroke and the most lagging accelerator pedal stroke among the first preset number of accelerator pedal strokes after shifting can be calculated, that is, calculating P _array [P _num -1]-P _array [0] If the value is less than the first preset threshold (here it is defined as P1 _Difference , the specific value is not limited, it can be set according to the actual situation, for example, the value can be -20, etc.), then determine The target vehicle is in a deceleration condition, and the preprocessed target sound can be played in the target vehicle, and the countdown variable for turning off the sound effect is set to a preset time (here, it is defined as TimerCount, and the specific value is not limited, and can be determined according to Actual situation setting, for example, can take a value of 10 seconds, etc.), or, if the value of P _array [P _num -1]-P _array [0] is greater than the second preset threshold (here it is defined as P2 _Difference , the specific value is not limited, it can be set according to the actual situation, for example, the value can be 20, etc.), then it is determined that the target vehicle is in the acceleration condition, the preprocessed target sound can be played in the target vehicle, and the sound effect sound The closing countdown variable is set to the preset time TimerCount, and when the preset time is reached, the control target sound gradually fades out until it is turned off.

Another optional implementation method is to firstly obtain the second preset number of the target vehicle (here, it is defined as V1 _num , the specific value is not limited, and can be set according to the actual situation, for example, the value can be 15, etc.), then determine whether the target vehicle is in an acceleration or deceleration condition according to the instantaneous vehicle speed V of the second preset quantity V1 _num ; if so, then play in the target vehicle The preprocessed target sound.

In this implementation, after obtaining the instantaneous vehicle speed of V1 _num of the target vehicle, define the historical data array of instantaneous vehicle speed of the target vehicle as V1 _array [V1 _num -1], and then move the values in the data array forward sequentially After one bit, get the instantaneous vehicle speed of V1 _num after shifting, that is, V1 _array [0] = V1 _array [1], V1 _array [1] = V1 _array [2], ..., V1 _array [V1 _num -2 ]=V1 _array [V1 _num -1], then, V1 _array [V1 _num -1]=V _new can be defined, and the value of calculating the average V1 _ave is as follows:

Wherein, V1 _ave represents the average value of the instantaneous vehicle speed after the shift of the second preset number V1num; the dimension of the V1array array is the second preset number V1num.

Further, it is possible to calculate the average of the shifted instantaneous vehicle speed of the third preset quantity (here, it is defined as V2 _num , the specific value is not limited, and can be set according to the actual situation, for example, the value can be 20, etc.) The average value of the value, specifically, the average vehicle speed historical data array of the target vehicle can be defined as V2 _array [V2 _num -1], and then the values in the data array are moved forward one by one to obtain V2 _array [0 ] = V2 _array [1], V2 _array [1] = V2 _array [2], ..., V2 _array [V1 _num -2] = V2 _array [V1 _num -1], then V2 _array [V2 _num -1] =V1 _ave , and calculate the value of the average value V2 _ave as follows:

Wherein, V2 _ave represents the average value of the average value of the instantaneous vehicle speed after the shift of the third preset number V2 _num ; the dimension of the V2 _array is the second preset number V2 _num .

Furthermore, the difference between the average value of the second preset number of shifted instantaneous vehicle speeds and the third preset number of shifted instantaneous vehicle speeds can be calculated, that is, the value of V1 _ave -V2 _ave can be calculated. value,

If the value is greater than the third preset threshold (here, it is defined as V1 _Difference , the specific value is not limited, it can be set according to the actual situation, for example, the value can be 5, etc.), then it is determined that the target vehicle is in the acceleration mode. In this case, the preprocessed target sound can be played in the target vehicle, and the countdown variable for turning off the sound effect is set to the preset time TimerCount (such as 10 seconds, etc.), or, if the value of V1 _ave -V2 _ave is less than the fourth preset Set the threshold (here it is defined as V2Difference, the specific value is not limited, it can be set according to the actual situation, for example, the value can be -5, etc.), then it is determined that the target vehicle is in a deceleration condition and can be played in the target vehicle The target sound is preprocessed, and the countdown variable for closing the sound effect is set to the preset time TimerCount. When the preset time is reached, the target sound is gradually faded out until it is turned off.

To sum up, the vehicle sound playing method provided by this embodiment first obtains the target sound; then preprocesses the target sound to obtain the preprocessed target sound; then obtains the speed of the target vehicle, and And the corresponding relationship between the preset rotational speed and the playback frequency determines the pre-processed target sound and the playback frequency; and then the pre-processed target sound can be played in the target vehicle according to the playback frequency. It can be seen that, since the embodiment of the present application not only utilizes the collected target sounds of vehicles of preset models running under different constant working conditions to realize the sound simulation effect of the full working condition of the vehicle, but also needs to play a certain section repeatedly when the vehicle speed is in a certain constant working condition. When sampling sound samples, there will be no popping sound, which makes the user's sense of hearing more natural, thereby improving the user's driving experience.

second embodiment

This embodiment will introduce a vehicle sound playback device, and for relevant content, please refer to the foregoing method embodiments.

Referring to FIG. 3 , it is a schematic diagram of the composition of a vehicle sound playback device provided in this embodiment. The device 300 includes:

An acquisition unit 301, configured to acquire a target sound;

A preprocessing unit 302, configured to preprocess the target sound to obtain the preprocessed target sound;

The first determination unit 303 is configured to obtain the rotational speed of the target vehicle, and determine the preprocessed target sound and the playback frequency according to the rotational speed of the target vehicle and the preset correspondence between the rotational speed and the playback frequency;

The playing unit 304 is configured to play the preprocessed target sound in the target vehicle according to the playing frequency.

In an implementation manner of this embodiment, the target sound includes multiple pieces of preprocessed sound data, and the multiple pieces of sound data are respectively collected when the vehicle of a preset model type runs to multiple different constant working conditions In-vehicle sound data.

In an implementation manner of this embodiment, the first determining unit 303 includes:

In an implementation manner of this embodiment, the multiple different constant working conditions include an idling working condition.

In an implementation manner of this embodiment, the preprocessing unit 302 includes:

In an implementation manner of this embodiment, the second preset duration is one-third or one-half of the first preset duration, or the second preset duration is not longer than the first preset duration. One-half of a preset duration.

In an implementation manner of this embodiment, the device further includes:

In an implementation manner of this embodiment, the switching unit is specifically configured to:

In an implementation manner of this embodiment, the device further includes:

The second determining unit is specifically used for:

In an implementation manner of this embodiment, the device further includes:

The third determining unit is specifically used for:

Further, the embodiment of the present application also provides a vehicle sound playback device, including: a processor, a memory, and a system bus;

The processor and the memory are connected through the system bus;

The memory is used to store one or more programs, and the one or more programs include instructions, and the instructions, when executed by the processor, cause the processor to perform any implementation method of the above-mentioned vehicle sound playing method .

Further, an embodiment of the present application also provides a vehicle, the vehicle includes the above-mentioned vehicle sound playback device.

Further, the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores instructions, and when the instructions are run on the terminal device, the terminal device executes the above-mentioned vehicle sound Any implementation of the play method.

Further, an embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any implementation method of the above-mentioned vehicle sound playing method.

From the above description of the implementation manners, it can be seen that those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in storage media, such as ROM/RAM, disk , optical disc, etc., including several instructions to make a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) execute the various embodiments or some parts of the embodiments of the present application. method.

It should be noted that each embodiment in this specification is described in a progressive manner, each embodiment focuses on the differences from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for relevant details, please refer to the description of the method part.

It should also be noted that in this article, relational terms such as first and second etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations Any such actual relationship or order exists between. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

A method for playing vehicle sound, comprising:

Get target sound;

Preprocessing the target sound to obtain the preprocessed target sound;

Acquiring the rotational speed of the target vehicle, and determining the preprocessed target sound and the playing frequency according to the rotational speed of the target vehicle and the correspondence between the preset rotational speed and the playing frequency;

The preprocessed target sound is played in the target vehicle according to the playback frequency.
The method according to claim 1, wherein the target sound comprises a plurality of pieces of preprocessed sound data, and the plurality of pieces of sound data are respectively generated when the vehicle of a preset model type operates under a plurality of different constant working conditions. Acquired in-vehicle sound data.
The method according to claim 2, characterized in that the acquisition of the rotational speed of the target vehicle, and determining the preprocessed target sound according to the rotational speed of the target vehicle and the corresponding relationship between the preset rotational speed and the playback frequency and play frequency, including:

selecting at least one piece of preprocessed sound data from the plurality of pieces of preprocessed sound data according to the rotational speed of the target vehicle;

The playback frequency of the at least one piece of preprocessed sound data is determined according to the rotation speed of the target vehicle and the preset correspondence between the rotation speed and the playback frequency.
The method of claim 2, wherein the plurality of different constant operating conditions includes an idle operating condition.
The method according to claim 1, wherein the preprocessing the target sound to obtain the preprocessed target sound comprises:

Dividing the target sound into a first sound segment and a second sound segment in sequence; the sound time lengths of the first sound segment and the second sound segment are both greater than or equal to a first preset duration;

The sound of the first second preset duration of the first sound segment is set to fade-in mode to obtain an updated first sound segment; and the sound of the second preset duration of the second sound segment is set to fade out mode to obtain the updated second sound segment;

The order of the updated first sound segment and the updated second sound segment is reversed, and a second preset duration of overlap is set between the two to obtain the preprocessed target sound.
The method according to claim 5, wherein the second preset duration is one-third or one-half of the first preset duration, or the second preset duration is not greater than the half of the first preset duration.
The method according to claim 1, characterized in that, after acquiring the rotational speed of the target vehicle, the method further comprises:

judging whether the speed of the target vehicle is in a speed overlapping interval;

If so, the preprocessed target sound corresponding to the previous speed interval and the preprocessed target sound corresponding to the next speed interval of the overlapping speed interval are simultaneously played.
The method according to claim 7, wherein the simultaneously playing the preprocessed target sound corresponding to the previous speed interval of the speed overlapping interval and the preprocessed target sound corresponding to the next speed interval includes: :

The pre-processed target sound corresponding to the previous speed interval of the speed overlapping interval is played in fade-out mode, and the pre-processed target sound corresponding to the next speed interval of the speed overlapping interval is played in fade-in mode.
The method according to claim 1, further comprising:

When the vehicle speed of the target vehicle is in a preset stable working condition, within the range of a third preset duration, the volume of the preprocessed target sound played in the target vehicle is reduced to a preset volume threshold.
The method according to claim 9, characterized in that the method further comprises:

Obtaining a first preset number of accelerator pedal strokes of the target vehicle;

According to the first preset number of accelerator pedal strokes, it is determined whether the target vehicle is in an acceleration or deceleration state; if yes, the preprocessed target sound is played in the target vehicle.
The method according to claim 10, characterized in that, after said acquisition of the first preset number of accelerator pedal strokes of said target vehicle, said method comprises:

By sequentially shifting the first preset number of accelerator pedal strokes of the target vehicle one bit forward, the shifted first preset number of accelerator pedal strokes are obtained;

The determining whether the target vehicle is in an acceleration or deceleration condition according to the first preset number of accelerator pedal strokes; if so, playing the preprocessed target sound in the target vehicle includes:

calculating the difference between the newest accelerator pedal stroke and the most delayed accelerator pedal stroke among the shifted first preset number of accelerator pedal strokes; if the difference is smaller than a first preset threshold value, then determining the target vehicle It is in a deceleration state, or if the difference is greater than a second preset threshold, it is determined that the target vehicle is in an acceleration state, and the preprocessed target sound is played in the target vehicle.
A vehicle sound player, characterized in that it comprises:

an acquisition unit, configured to acquire the target sound;

a preprocessing unit, configured to preprocess the target sound to obtain the preprocessed target sound;

The first determining unit is configured to acquire the rotational speed of the target vehicle, and determine the preprocessed target sound and the playback frequency according to the rotational speed of the target vehicle and the preset correspondence between the rotational speed and the playback frequency;

A playback unit, configured to play the preprocessed target sound in the target vehicle according to the playback frequency.
A vehicle sound playback device, characterized in that it includes: a processor, a memory, and a system bus;

The processor and the memory are connected through the system bus;

The memory is used to store one or more programs, and the one or more programs include instructions that, when executed by the processor, cause the processor to perform the method described in any one of claims 1-11. method.
A vehicle, characterized in that the vehicle comprises the vehicle sound playback device as claimed in claim 13 .
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the instructions are run on a terminal device, the terminal device is made to perform the operation described in any one of claims 1-11. described method.