JP2016057570A - Acoustic analysis device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To positively estimate a subjective impression.SOLUTION: A correlation formula setting part 40 sets a correlation formula Fm for expressing a relation between an impression index Ym of an auditory impression in a correspondence relation regulated by a relation descriptive data DC using a plurality of reference data r making the impression index ym showing the auditory impression of a reference sound mutually correspond to an acoustic characteristic of the reference sound and the relation descriptive data DC for regulating the correspondence relation between the auditory impression and a plurality of acoustic characteristics.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for analyzing sound.

  Techniques for evaluating song singing have been proposed. For example, Patent Literature 1 discloses a technique for evaluating a song in consideration of song expressions such as vibrato and intonation in addition to the pitch of the song voice. Patent Document 2 discloses a technique for evaluating a song according to the pitch (basic frequency) and volume of a singing voice.

JP 2005-107337 A JP2013-020265A

  However, the techniques of Patent Document 1 and Patent Document 2 merely evaluate an objective skill of a singing that focuses only on the difference between a reference value indicating an exemplary singing and a feature value of the singing voice to be evaluated. There is a problem that subjective viewpoints such as impressions perceived by the listener of the singing voice are not properly evaluated. For example, individual or characteristic singing can actually give the listener a skilled impression, but as a result of deviating from the exemplary singing, the techniques of Patent Document 1 and Patent Document 2 are not rated highly. Probability is high. In the above description, the evaluation of the singing voice has been exemplified. However, when evaluating various sounds such as the performance sound of the musical instrument and the reproduction sound of the acoustic device, the subjective impression that the listener senses is appropriately set. There is a similar situation that cannot be evaluated. In view of the above circumstances, an object of the present invention is to appropriately evaluate a subjective impression of sound.

  In order to solve the above problems, the acoustic analysis device of the present invention includes a plurality of reference data in which an impression index indicating an auditory impression of a reference sound and a feature index indicating an acoustic feature of the reference sound are associated with each other, and Using the relationship description data that defines the correspondence between the auditory impression and multiple types of acoustic features, the impression index of the auditory impression and the feature index of each acoustic feature in the correspondence specified by the relationship description data A relational expression setting unit for setting a relational expression expressing the relationship is provided. In the above configuration, a relational expression that expresses the relationship between the impression index of the auditory impression and the feature index of each acoustic feature is set. Therefore, by using the relational expression set by the relational expression setting means, it is possible to appropriately evaluate the subjective impression of sound.

  By the way, in the configuration in which the relational expression is set only by the statistical analysis of the reference data, the pseudo-correlation (as if the specific feature index does not actually correlate with the specific auditory impression but correlates with the wrinkles due to potential factors. There is a possibility that a relational expression in which a feature index that does not actually correlate with the auditory impression has an influence on the auditory impression is derived. In the present invention, in addition to a plurality of reference data in which an impression index and a feature index correspond to each other, a relational expression is set using relationship description data that defines a correspondence relationship between an auditory impression and a plurality of types of acoustic features. Is done. Therefore, compared to a configuration in which a relational expression is set using only reference data, a relational expression that appropriately reflects the actual correlation between the impression index and a plurality of feature indices (that is, a relational expression that can appropriately evaluate an auditory impression) ) Can be set.

  In a preferred aspect of the present invention, the relationship description data defines a correspondence relationship between the auditory impression and a plurality of types of acoustic features via a plurality of intermediate elements included in the auditory impression. In the above aspect, since the correspondence relationship between the auditory impression and a plurality of types of acoustic features via a plurality of intermediate elements included in the auditory impression is defined by the relationship description data, the auditory impression and each acoustic feature The above-described effect that the relational expression that appropriately reflects the actual correlation between the auditory impression and each acoustic feature can be set is particularly remarkable as compared with the case where the above is directly correlated.

  In a preferred aspect of the present invention, the relational expression setting means sets a relational expression for each of a plurality of types of auditory impressions. In the above aspect, since the relational expression is set for each of a plurality of types of auditory impressions, there is an advantage that the auditory impression can be appropriately evaluated from various viewpoints. For example, the relational expressions for evaluating the auditory impression of singing voices are: young (adult / childish), light / dark (bright / dark), and turbidity (clean and transparent / cloudy) A configuration in which a relational expression is set for each of a plurality of types of auditory impressions including is particularly suitable.

  In a preferred aspect of the present invention, the relational expression setting unit obtains reference data and updates a predetermined relational expression using the reference data. In the above aspect, since the relational expression is updated using the reference data acquired after setting the relational expression, it is possible to set the relational expression that appropriately reflects the actual correlation between the auditory impression and each acoustic feature. The effect of is particularly remarkable.

  An acoustic analysis apparatus according to another aspect of the present invention is an apparatus that analyzes an auditory impression of a sound to be analyzed using the relational expressions generated in each of the above aspects, and a feature that extracts a feature index of the sound to be analyzed A plurality of reference data in which the extraction means, the impression index indicating the auditory impression of the reference sound and the feature index indicating the acoustic feature of the reference sound are associated with each other, and the correspondence relationship between the auditory impression and the plurality of types of acoustic features A relational expression that calculates the relationship between the impression index of auditory impression and the characteristic index of multiple types of acoustic features in the correspondence relationship defined by the relationship description data. Applying the feature index extracted by the extraction unit includes an impression specifying unit that calculates an impression index of the analysis target sound. In the above aspect, the auditory impression of the analysis target sound is obtained by using the relational expression that appropriately reflects the actual correlation between the impression index and the plurality of feature indices by using the plurality of reference data and the relationship description data. Can be evaluated appropriately.

  The acoustic analysis device according to each aspect described above is realized by a dedicated electronic circuit, or by cooperation of a general-purpose arithmetic processing device such as a CPU (Central Processing Unit) and a program. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (acoustic analysis method) of the acoustic analysis device according to each of the above aspects.

1 is a configuration diagram of an acoustic analysis device according to a first embodiment of the present invention. It is a schematic diagram of an analysis result image. It is a flowchart of the operation | movement which analyzes the auditory impression of a song voice. It is explanatory drawing of the correspondence of the auditory impression prescribed | regulated with relationship description data, and an acoustic feature. It is a block diagram of the acoustic analyzer which concerns on 2nd Embodiment. It is a block diagram of the acoustic analyzer which concerns on 3rd Embodiment. It is explanatory drawing of the modification of 3rd Embodiment. It is a block diagram of the acoustic analyzer which concerns on 4th Embodiment. It is explanatory drawing of the property estimation in 4th Embodiment. It is explanatory drawing of the property estimation in 4th Embodiment. It is a block diagram of the acoustic analyzer which concerns on 5th Embodiment. It is explanatory drawing of the operation screen in 5th Embodiment. It is a block diagram of the acoustic analyzer which concerns on 6th Embodiment. It is a block diagram of the acoustic analyzer which concerns on a modification.

<First Embodiment>
FIG. 1 is a configuration diagram of an acoustic analysis device 100A according to the first embodiment of the present invention. The acoustic analysis device 100A according to the first embodiment is realized by a computer system including an arithmetic processing device 10, a storage device 12, an input device 14, a sound collection device 16, and a display device 18. For example, a portable information processing device such as a mobile phone or a smartphone, or a portable or stationary information processing device such as a personal computer can be used as the acoustic analysis device 100A.

  The sound collection device 16 is a device (microphone) that collects ambient sounds. The sound collection device 16 of the first embodiment collects a singing voice V in which a user sang a song. The acoustic analysis device 100A can also be used as a karaoke device that mixes and reproduces the accompaniment sound of the music and the singing voice V. In addition, illustration of the A / D converter which converts the signal of the singing voice V picked up by the sound pickup device 16 from analog to digital is omitted for convenience.

  The display device 18 (for example, a liquid crystal display panel) displays an image instructed from the arithmetic processing device 10. The input device 14 is an operating device operated by the user for various instructions to the acoustic analysis device 100A, and includes a plurality of operators operated by the user, for example. A touch panel configured integrally with the display device 18 can also be used as the input device 14. The storage device 12 stores a program executed by the arithmetic processing device 10 and various data used by the arithmetic processing device 10. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12.

  The acoustic analysis device 100A of the first embodiment is a signal processing device that analyzes the singing voice V collected by the sound collection device 16. The arithmetic processing device 10 executes a program stored in the storage device 12 to thereby analyze a plurality of functions for analyzing the singing voice V (a feature extraction unit 22, an impression identification unit 24, a presentation processing unit 26, and a related expression setting). Part 40). A configuration in which each function of the arithmetic processing device 10 is distributed to a plurality of devices or a configuration in which a dedicated electronic circuit realizes a part of the function of the arithmetic processing device 10 may be employed.

  The feature extraction unit 22 analyzes the singing voice V collected by the sound collection device 16 to extract a plurality (N) of feature indexes X1 to XN indicating different types of acoustic features (N is a natural number). . The acoustic feature means an acoustic feature of the singing voice V that influences an auditory impression (hereinafter referred to as “auditory impression”) sensed by the listener of the singing voice V. Specifically, the feature index Xn (n = 1 to N) quantifying each of various acoustic features such as pitch (pitch) stability, vibrato depth (pitch amplitude), frequency characteristics, etc. is sung. Extracted from voice V. The numerical value ranges of the N feature indexes X1 to XN extracted by the feature extraction unit 22 of the first embodiment are common. As understood from the above description, the auditory impression means a subjective or sensory feature (impression) that is perceived by the listener of the singing voice V, and the acoustic feature is extracted by analysis of the singing voice V. Means an objective or physical feature.

  The impression specifying unit 24 specifies the auditory impression of the singing voice V using the N feature indexes X1 to XN extracted by the feature extracting unit 22. The impression specifying unit 24 of the first embodiment calculates a plurality (M) of impression indexes Y1 to YM indicating different auditory impressions of the singing voice V (M is a natural number). The arbitrary one impression index Ym (m = 1 to M) in the first embodiment is an index obtained by quantifying the degree of two types of impressions that oppose each other. Specifically, each of various auditory impressions such as young children (adult / childish), light / dark (bright / dark), and turbidity (clear and transparent / slow and muddy) were quantified. An impression index Ym is specified. For example, the larger an impression index Ym related to a young child is in a positive number range, the more adult-like sound is meant, and the smaller the impression index Ym is in a negative number range, the more child-like sound is meant.

For the calculation of the impression index Ym (Y1 to YM) corresponding to the N feature indices X1 to XN, an arithmetic expression (hereinafter referred to as “related expression”) Fm set in advance for each impression index Ym is used. An arbitrary relational expression Fm is an arithmetic expression that expresses the relationship between the impression index Ym and the N feature indices X1 to XN. The relational expression Fm of the first embodiment represents each impression index Ym as a linear expression of N feature indices X1 to XN, as exemplified below.

  The coefficient anm (a11 to aNM) of the relational expression Fm exemplified above is a constant (gradient of the impression index Ym with respect to the feature index Xn) according to the degree of correlation between the feature index Xn and the impression index Ym, and the coefficient bm ( b1 to bM) are predetermined constants (intercepts). The coefficient anm can also be restated as the contribution (weighted value) of the feature index Xn to the impression index Ym. The impression specifying unit 24 applies the N feature indexes X1 to XN extracted by the feature extraction unit 22 to each of the related expressions F1 to FM, so that M impression indexes Y1 to YM corresponding to different auditory impressions are applied. Is calculated. The impression specifying unit 24 of the first embodiment generates singing style information S corresponding to the M impression indices Y1 to YM calculated from the feature indices Xn. Specifically, an M-dimensional vector having M impression indexes Y1 to YM as elements is generated as singing style information S. As understood from the above description, the singing style information S comprehensively represents M types of auditory impressions (subjective singing styles perceived by the listener) of the singing voice V. Although the linear system is exemplified in the first embodiment as described above, a nonlinear system such as a hidden Markov model or a neural network (multilayer perceptron) can be used for calculating the impression index Ym (Y1 to YM). .

  The presentation processing unit 26 in FIG. 1 displays various images on the display device 18. Specifically, the presentation processing unit 26 of the first embodiment displays an analysis result image 70 that expresses the M impression indices Y1 to YM (singing style information S) of the singing voice V specified by the impression specifying unit 24. It is displayed on the device 18.

  FIG. 2 shows one impression index Y1 related to a young child (adult / childish) among M kinds of impression indexes Y1 to YM and one related to turbidity (clean and transparent / slowly turbid) This is a display example of an analysis result image 70 representing the impression index Y2. As understood from FIG. 2, the analysis result image 70 includes a coordinate plane in which a first axis 71 indicating the numerical value of the impression index Y1 and a second axis 72 indicating the numerical value of the impression index Y2 are set. The singing voice V is located at a coordinate position corresponding to the numerical value of the impression index Y1 calculated by the impression specifying unit 24 in the first axis 71 and the numerical value of the impression index Y2 calculated in the second axis 72 by the impression specifying unit 24. An image (icon) 74 meaning an auditory impression is arranged. As understood from the above description, the analysis result image 70 is an image representing an auditory impression of the singing voice V (an image representing a singing style including young children and turbidity). The user can visually and intuitively grasp the auditory impression of the singing voice V by visually recognizing the analysis result image 70 displayed on the display device 18.

  FIG. 3 is a flowchart of the operation of analyzing the auditory impression of the singing voice V. For example, the processing of FIG. 3 is started when an operation (instruction to start analysis) from the user with respect to the input device 14 is triggered. When the processing of FIG. 3 is started, the feature extraction unit 22 acquires the singing voice V picked up by the sound pickup device 16 (S1), and N feature indices X1 indicating the acoustic features of the analysis section of the singing voice V. .About.XN are extracted (S2). The analysis section is a section of the singing voice V that is an analysis target of an auditory impression, and is, for example, the entire section or a part of the singing voice V (for example, a chorus section). The impression specifying unit 24 calculates M impression indexes Y1 to YM by applying the N feature indexes X1 to XN extracted by the feature extraction unit 22 to each related expression Fm (S3). The presentation processing unit 26 causes the display device 18 to display the analysis result image 70 of FIG. 2 representing the analysis result by the impression specifying unit 24 (S4).

  The relational expression setting unit 40 in FIG. 1 sets relational expressions Fm (F1 to FM) used for calculating the impression index Ym of each auditory impression. As illustrated in FIG. 1, the storage device 12 of the first embodiment stores a reference data group DR and relationship description data DC. The related expression setting unit 40 sets M related expressions F1 to FM using the reference data group DR and the relationship description data DC.

  The reference data group DR is a set (database) of a plurality of reference data r. The plurality of reference data r included in the reference data group DR is generated in advance by using a sound (hereinafter referred to as “reference sound”) generated by an unspecified number of speakers. For example, the sound of an arbitrary singer singing an arbitrary piece of music is recorded as a reference sound and used to generate reference data r. As illustrated in FIG. 1, any one piece of reference data r corresponds to each impression index ym (y1 to yM) of the reference sound and the feature index xn (x1 to xN) of the reference sound. It is data. The impression index ym is set to a numerical value corresponding to the auditory impression actually sensed by the listener of the reference sound, and the characteristic index xn is a numerical value of the acoustic feature extracted from the reference sound in the same process as the feature extracting unit 22. Set to That is, each reference data r corresponds to material (learning data) in which the relationship between the impression index ym and the feature index xn is actually observed.

  The relationship description data DC defines a correspondence relationship (correlation) between an auditory impression and a plurality of acoustic features. FIG. 4 is an explanatory view illustrating the correspondence defined by the relationship description data DC of the first embodiment. As illustrated in FIG. 4, the relationship description data DC of the first embodiment affects the auditory impression EYm for each of M types of auditory impressions EY (EY1 to EYM) corresponding to different impression indices Ym. A correspondence λm (λ1 to λM) with a plurality of types of acoustic features EX is defined. FIG. 4 illustrates correspondence relationships λ1 to λ3 with a plurality of types of acoustic features EX for each of the three types of auditory impressions EY1 to EY3, which are childhood, clearness, and light and dark.

Specific contents of the acoustic feature EX correlated with each auditory impression EYm are as follows. The numerical value of each acoustic feature EX exemplified below corresponds to the above-described feature index Xn.
・ Pitch stability: Degree of minute change (fluctuation) in time ・ Rise speed: Degree of increase in volume immediately after pronunciation ・ Fall: Decrease the pitch from the reference value (note pitch) Degree of singing expression (eg number of times)
・ Scribbling: the degree of singing expression that raises the pitch over time from the reference value (for example, the number of times)
・ Vibrato depth: the degree of pitch change in vibrato (eg amplitude and frequency)
Contour: degree of sound clarity. For example, the volume ratio of the high frequency component to the low frequency component is suitable.
-Tongue: The degree of temporal change in acoustic characteristics. For example, the degree of temporal change (typically the time change rate) of the frequency characteristics (for example, formant frequency or fundamental frequency) is suitable.
・ Attack: Volume immediately after sound generation ・ Crescend: Degree of increase in volume over time ・ Frequency characteristics: Shape of frequency spectrum ・ Higher harmonics: Intensity of higher harmonic components

  As illustrated in FIG. 4, the correspondence relationship λm that the relationship description data DC of the first embodiment defines for any one type of auditory impression EYm is used to identify multiple types of intermediate elements EZ related to the auditory impression EYm. This is a hierarchical relationship (hierarchical structure) interposed between the impression EYm and each acoustic feature EX. A plurality of types of intermediate elements EZ related to one type of auditory impression EYm correspond to an impression that causes the listener to perceive the auditory impression EYm and an impression obtained by subdividing the auditory impression EYm into a plurality of parts. Any one intermediate element EZ is associated with a plurality of types of acoustic features EX that affect the intermediate element EZ.

  Each correspondence λm defined in the relationship description data DC is, for example, a survey (interviews, singer, etc.) for experts who have a lot of specialized knowledge about music and voice (singing). Constructed by analyzing the correlation between each acoustic impression EYm and each acoustic feature EX (what kind of acoustic impression EYm the listener tends to perceive from the sound of the acoustic feature EX) by questionnaire) Is done. For the construction of the correspondence relationship λm, a known investigation technique represented by an evaluation grid method or the like can be arbitrarily employed.

  The relationship description data DC described above defines only the mutual relationship (connection) of each element (acoustic feature EX, intermediate element EZ, auditory impression EYm) included in the correspondence relationship λm, and between the elements. The degree of correlation is not specified. From the above viewpoint, each correspondence λm defined by the relationship description data DC is an actual correlation between the acoustic feature EX and the auditory impression EYm observed from the reference sounds collected from a large number of unspecified speakers. (I.e., the actual relationship between each impression index ym and each feature index xn statistically observed from the reference data group DR reflecting the tendency of the actual reference sound) It can be said that there is.

  The reference data group DR and the relationship description data DC described above are created in advance and stored in the storage device 12. The relational expression setting unit 40 in FIG. 1 sets M relational expressions F1 to FM using the reference data group DR and the relationship description data DC stored in the storage device 12. That is, the relational expression setting unit 40 represents the relational expression Fm that expresses the relationship between the impression index Ym of the auditory impression EYm and the characteristic index Xn of the acoustic feature EX under the corresponding relations λm defined by the relationship description data DC. Is set for each of the M impression indices Y1 to YM. Specifically, the relationship in which the degree of correlation between the impression index ym and the feature index xn in the plurality of reference data r in the reference data group DR is reflected in the correspondence relationship λm of the relationship description data DC is expressed by the relational expression Fm. As described above, the relational expression setting unit 40 sets N coefficients a1m to aNm and one coefficient bm for each relational expression Fm. For the setting of each relational expression Fm by the relational expression setting unit 40, for example, known statistical processing such as structural equation modeling (SEM: Structural Equation Modeling) or multivariate analysis (for example, multiple regression analysis) can be arbitrarily employed. As understood from the example of FIG. 4, the type and total number of acoustic features EX that are correlated with the auditory impression EYm based on the correspondence λm expressed by the relationship description data DC are actually Although different for each auditory impression EYm, the type and the total number of feature indexes Xn included in each of the related formulas Fm described above are common to M related formulas F1 to FM. The coefficient anm corresponding to the feature index Xn of the acoustic feature EX whose correlation with the auditory impression EYm is not defined under the correspondence relationship λm is set to zero in the related expression Fm (that is, the feature index Xn is Does not affect the impression index Ym).

  The M relational expressions (for example, structural equations and multiple regression equations) F1 to FM set by the relational expression setting unit 40 in the above procedure are stored in the storage device 12. Specifically, N coefficients a1m to aNm and one coefficient bm are stored in the storage device 12 for each of the M related expressions F1 to FM. As described above, the impression specifying unit 24 applies the M feature indexes X1 to XN to each of the M related formulas F1 to FM set by the related formula setting unit 40, so that the M types of impression indexes Y1 to YM are applied. Is calculated.

  As described above, in the first embodiment, the singing is performed using the relational expression Fm that defines the relationship between each feature index Xn extracted from the singing voice V and the impression index Ym indicating the auditory impression of the singing voice V. Auditory impressions (impression indices Y1 to YM) of the voice V are specified. Therefore, for example, the singing voice V is compared with the techniques of Patent Document 1 and Patent Document 2 that evaluate the skill of the singing by focusing only on the difference between the reference value indicating the exemplary singing and the characteristic index Xn of the singing voice V. It is possible to appropriately evaluate the subjective impression that the listener actually takes.

  By the way, by analyzing only a plurality of reference data r of the reference data group DR, the tendency of the correlation between the impression index ym and the feature index xn is statistically analyzed to set the related expression Fm (hereinafter referred to as “proportional”). Can also be envisaged. That is, in the proportionality, the relationship description data DC is not used for setting the relational expression Fm. In contrast, however, a specific acoustic feature EX that does not actually correlate with the auditory impression EYm is recognized as if it is correlated with the auditory impression EYm due to a potential factor (pseudo-correlation). ), There is a possibility that a relational expression Fm is derived in which the characteristic index Xn that does not actually correlate with the impression index Ym has a dominant influence on the impression index Ym. On the other hand, in the first embodiment, the relationship description data DC defining the hypothetical correspondence λm between each auditory impression EYm and each acoustic feature EX is used together with the reference data group DR for setting the relational expression Fm. The influence of the pseudo correlation between the auditory impression EYm and the acoustic feature EX is reduced (ideally excluded). Therefore, there is an advantage that the relational expression Fm appropriately expressing the actual correlation between the auditory impression EYm and each acoustic feature EX can be set. In the first embodiment, since the correspondence relationship λm between the auditory impression EYm and each acoustic feature EX via a plurality of intermediate elements EZ related to the auditory impression EYm is defined by the relationship description data DC, the auditory impression EYm and each Compared with the configuration in which the acoustic feature EX is directly correlated (the configuration in which the correspondence λm includes only the auditory impression EYm and the acoustic feature EX), the actual correlation between the auditory impression EYm and each acoustic feature EX is related The above-described effect of being able to be appropriately expressed by the formula Fm is particularly remarkable.

<Modification of First Embodiment>
In the above description, a case where a plurality of reference data r is stored in advance in the storage device 12 is exemplified. However, as exemplified below, a new singing voice V collected by the sound collection device 16 is used as a reference sound. It is also possible to update each relational expression Fm using the reference data r.

  A user (speaker or listener of the singing voice V) designates an auditory impression of the singing voice V by appropriately operating the input device 14 after the music is finished. For example, for each of the M types of auditory impressions, a plurality of options (multiple levels of evaluation) of the impression index Ym are displayed on the display device 18, and the user specifies one desired option for each auditory impression.

  1, the relational expression setting unit 40 extracts each feature extracted by the feature extraction unit 22 for the impression index ym (y1 to ym) and the singing voice V of each auditory impression specified by the user. Reference data r including the index xn (x1 to xN) is acquired and stored in the storage device 12. Then, the relational expression setting unit 40 uses the reference data group DR including the new reference data r according to the singing voice V to obtain the relational expression Fm (F1 to FM) in the same manner as in the first embodiment. Set and remember. That is, the predetermined relational expression Fm (F1 to FM) is updated to reflect the relationship between the auditory impression (impression index ym) and the acoustic feature (feature index xn) of the singing voice V collected by the sound collection device 16. The According to the above configuration, there is an advantage that the relational expressions F1 to FM can be updated to contents reflecting the relationship between the auditory impression of the actual singing voice V and the acoustic features. The timing for setting (updating) the relational expression Fm using the reference data group DR is arbitrary. For example, a configuration in which the related formula Fm is updated each time the reference data r corresponding to the singing voice V is acquired, or a configuration in which the related formula Fm is updated when a predetermined number of new reference data r is accumulated can be adopted. Moreover, the modification illustrated above can be similarly applied to each embodiment illustrated below.

Second Embodiment
A second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are the same as that of 1st Embodiment in each form illustrated below, the reference | standard referred by description of 1st Embodiment is diverted, and each detailed description is abbreviate | omitted suitably.

  FIG. 5 is a configuration diagram of the acoustic analysis device 100B of the second embodiment. As illustrated in FIG. 5, the acoustic analysis device 100 </ b> B according to the second embodiment includes the same elements as the first embodiment (the feature extraction unit 22, the impression identification unit 24, the presentation processing unit 26, and the related expression setting unit 40). The information generation unit 32 is added. The information generation unit 32 generates presentation data QA corresponding to the auditory impressions (M impression indexes Y1 to YM) specified by the impression specification unit 24 in the same manner as in the first embodiment. In other words, the information generation unit 32 can be rephrased as an element that converts the M impression indexes Y1 to YM into the presentation data QA. The presentation processing unit 26 of the second embodiment presents the presentation data QA generated by the information generation unit 32 to the user. Specifically, the presentation processing unit 26 causes the display device 18 to display the contents of the presentation data QA. The extraction of N feature indices X1 to XN by the feature extraction unit 22 and the setting of M relational expressions F1 to FM by the relational expression setting unit 40 are the same as in the first embodiment. Therefore, the same effects as those of the first embodiment are realized in the second embodiment.

  The information generation unit 32 of the second embodiment generates music related data dA corresponding to the M impression indexes Y1 to YM (singing style information S) specified by the impression specifying unit 24 as the presentation data QA. Specifically, the information generation unit 32 searches a plurality of candidates for music corresponding to the M impression indexes Y1 to YM, and acquires related data dA of the music. The related data dA is information related to music. For example, in addition to music identification information (for example, music number), attribute information such as music name, singer name, and genre is included in the related data dA.

  The search data WA stored in the storage device 12 is used for searching for music by the information generating unit 32 (generating related data dA). The search data WA defines the relationship between the singing style information S (M impression indices Y1 to YM) and music. Specifically, the search data WA of the second embodiment is related to music related data dA (dA1, dA2,...) For each of a plurality of classes CL (CL1, CL2,...) Corresponding to different singing styles. ) Is specified.

  Specifically, a large number of singing style information S generated from the singing voice V of an arbitrary piece of music is classified into a plurality of classes CL, and the singing voice of each singing style information S classified into an arbitrary one class CL. In V, for example, the related data dA of one piece of music having the highest number of singing is designated as the class CL in the search data WA. That is, for the class CL corresponding to any one kind of singing style, the related data dA of the music that many singers tend to sing in the singing style is designated. For the classification of the singing style information S, known statistical processing (clustering) is arbitrarily adopted, and the plurality of classes CL are expressed by, for example, a mixed normal distribution that approximates the distribution of the singing style information S belonging to each class CL. .

  The information generating unit 32 specifies one class CL to which the singing style information S (M impression indexes Y1 to YM) generated by the impression specifying unit 24 among a plurality of classes CL registered in the search data WA belongs. Then, the related data dA of the music designated as the class CL in the search data WA is selected as the presentation data QA. The presentation processing unit 26 causes the display device 18 to display the presentation data QA (related data dA) generated by the information generation unit 32. That is, music identification information and attribute information are displayed on the display device 18. As will be understood from the above description, a song that tends to be sung by many singers in the same singing style as the singing voice V (that is, a tune that is easy to sing in the same singing style as the singing voice V) is given to the user. Presented.

  Further, the information generation unit 32 of the second embodiment can update the search data WA using the singing voice V collected by the sound collection device 16. Specifically, the information generation unit 32 sequentially accumulates the singing style information S generated from the singing voice V of any one piece of music together with the related data dA of the piece of music in the storage device 12 and stores it in the storage device 12. The search data WA is updated by known machine learning, for example, so that the relationship between the accumulated singing style information S and the related data dA is reflected. As will be understood from the above description, in the second embodiment, music is not searched based on the characteristics (musical tone, etc.) of the music itself, but based on singing styles sung by many singers in the past. Is searched. For example, if the auditory impression of the singing voice V is “passionate and bright singing”, similarly, songs sung by many singers in the past with the singing style of “passionate and bright singing” are searched.

  By the way, a technique for searching for music that matches a user's request has been proposed. For example, Japanese Patent Application Laid-Open No. 2011-197345 discloses a technique of searching for music corresponding to a keyword designated by the user and presenting it to the user. However, the above technique only searches for music that is formally related to the keyword specified by the user. In the second embodiment, music corresponding to the auditory impression (singing style) of the singing voice V specified by the impression specifying unit 24 is presented to the user, so that the user who uttered the singing voice V uses his / her singing style. There is an advantage that it is possible to recognize compatible music (music that is easy to sing in its own singing style).

<Modification of Second Embodiment>
(1) In the above description, the configuration in which the search data WA specifies one piece of music for each of the plurality of classes CL of the singing style information S has been exemplified. However, for any one class CL, the class CL It is also possible to specify related data dA of a plurality of music pieces sung by each singing voice V of the singing style classified as “1”. The information generation unit 32 generates, as the presentation data QA, related data dA of a plurality of pieces of music specified in one class CL to which the singing style information S specified by the impression specifying unit 24 belongs. That is, a plurality of music pieces that tend to be sung in the same singing style as the singing voice V are presented to the user.

(2) It is also possible to selectively present related data dA of one piece of music among a plurality of pieces of music designated in one class CL to which the singing style information S belongs. The condition for selecting one piece of music from a plurality of pieces designated for one class CL is arbitrary. For example, a piece of music having the largest number of times of singing with each singing voice V classified into the class CL is selected. According to the configuration to be selected and instructions from the user (for example, selection conditions such as “90s” specified by the user and attribute information such as the user's age) among the plurality of songs specified in the class CL A configuration in which one piece of music is selected is preferable.

(3) The search data WA can be updated at any time so that the relationship between the singing style information S of the singing voice V and the music of the singing voice V is reflected. However, if the singing style information S of the singing voice V whose hearing impression is not appropriate is reflected in the search data WA, an appropriate search for music may be hindered. Therefore, it is preferable to select the singing style information S to be reflected in the search data WA. For example, the user (speaker or listener) specifies whether or not the actual auditory impression of the singing voice V is appropriate by operating the input device 14, and the singing style information S and the singing voice V for which the auditory impression is determined to be appropriate. The relationship with the music is reflected in the search data WA, and the singing style information S of the singing voice V for which the auditory impression is determined to be inappropriate is not reflected in the search data WA. According to the above structure, there exists an advantage that the search data WA reflecting the singing style of many singers can be generated.

<Third Embodiment>
FIG. 6 is a configuration diagram of an acoustic analysis device 100C according to the third embodiment. As illustrated in FIG. 6, the acoustic analysis device 100 </ b> C of the third embodiment is similar to the acoustic analysis device 100 </ b> B (FIG. 5) of the second embodiment in that M impression indexes Y <b> 1 to Y specified by the impression specification unit 24. In this configuration, an information generation unit 32 that generates presentation data QA corresponding to YM is added to the first embodiment. The extraction of N feature indices X1 to XN by the feature extraction unit 22 and the setting of M relational expressions F1 to FM by the relational expression setting unit 40 are the same as in the first embodiment. Therefore, the third embodiment can achieve the same effect as the first embodiment.

  The storage device 12 according to the third embodiment stores a plurality of image data dB representing an auditory impression (M impression indices Y1 to YM). Each image data dB represents an image (including symbols and characters) representing the auditory impression figuratively or schematically. For example, an image of a character (animal or the like) or a celebrity suitable for the auditory impression specified by the impression specifying unit 24 is suitable as the image data dB. The information generating unit 32 of the second embodiment displays image data dB representing the auditory impression (M impression indices Y1 to YM) specified by the impression specifying unit 24 among the plurality of image data dB stored in the storage device 12. Is selected as the presentation data QA. For the selection of the image data dB by the information generation unit 32, the conversion data WB stored in the storage device 12 is used.

  The conversion data WB defines the relationship between the M impression indexes Y1 to YM (singing style information S) and the image data dB. Specifically, the conversion data WB expresses a structural equation that defines the correlation between the M impression indexes Y1 to YM and the image data dB. For the setting of each structural equation, for example, structural equation modeling (SEM) is preferably used as in the setting of M related equations F1 to FM exemplified in the first embodiment. That is, for example, a plurality of learning data (learning data) in which M impression indices Y1 to YM and image data dB are associated with each other, and a relationship that defines a correspondence relationship between M types of auditory impressions and image data dB. In the structural equation modeling using the sex description data, the structural equation defining the relationship between the M impression indexes Y1 to YM and the image data dB is set in advance and stored in the storage device 12 as the conversion data WB. . For example, when the impression index Ym regarding a young child indicates a child-like song, image data dB indicating an image of a child-like character is generated, and when the impression index Ym regarding light and dark indicates a bright song, a character with a bright expression is generated. Conversion data WB is set and stored so that image data dB representing an image is generated.

  The information generation unit 32 specifies the image data dB by applying the M impression indices Y1 to YM specified by the impression specification unit 24 to the structural equation of the conversion data WB, and the image stored in the storage device 12 Data dB is acquired as presentation data QA. The presentation processing unit 26 causes the display device 18 to display the presentation data QA generated by the information generation unit 32. As understood from the above description, an image representing the auditory impression (singing style) of the singing voice V figuratively or schematically is displayed on the display device 18. The user can visually and intuitively grasp the auditory impression of the singing voice V by visually recognizing the image displayed on the display device 18.

  By the way, for example, Japanese Patent Application Laid-Open No. 2002-041063 discloses a technique for displaying an image of a character according to information such as a song name, the number of singings, and a scoring result. However, the above technique only displays an image unrelated to the auditory impression of the singing voice. In the third embodiment, an image (an image representing the singing style) corresponding to the auditory impression of the singing voice V specified by the impression specifying unit 24 is presented, so that the user intuitively views the auditory impression of the singing voice V. There are advantages of being able to grasp and providing interest to users.

  If the image data dB corresponding to the characteristics of the singing voice V is simply selected, a direct relationship between each feature index Xn of the singing voice V and the image data dB is determined in advance, and a feature extracting unit is selected. A configuration in which image data dB corresponding to each feature index Xn extracted by 22 can be assumed. However, since it is difficult to grasp which auditory impression corresponds to a specific numerical value of each feature index Xn, image data dB representing the auditory impression of the singing voice V is appropriately associated with the feature index Xn. It is actually difficult to do. In the third embodiment, since the relationship between the auditory impression (M impression indices Y1 to YM) and the image data dB is defined by the conversion data WB, the image data dB suitable for each auditory impression is used as the conversion data WB. It is possible to correspond to each impression index Ym. There is also an advantage that the relationship between the auditory impression and the image data dB can be changed independently of the relational expressions F1 to FM.

<Modification of Third Embodiment>
(1) It is also possible to generate a plurality of image data dB corresponding to the auditory impression of the singing voice V as the presentation data QA. Specifically, a plurality of image data dB stored in the storage device 12 is classified into a plurality (K) groups (categories), and presentation data QA including K image data dB selected from different groups is displayed. Generated by the information generator 32. As illustrated in FIG. 7, each group corresponds to each element constituting a specific article. For example, assuming “cake” having “topping”, “cream” and “base” as elements (K = 3), a plurality of image data dB indicating each image of “topping”, and “cream” A plurality of image data dB indicating each of the images and a plurality of image data dB indicating each “base” image are stored in the storage device 12.

  The extraction of the N feature indexes X1 to XN by the feature extraction unit 22 and the specification of the M impression indexes Y1 to YM by the impression specification unit 24 are the K unit sections obtained by dividing the singing voice V on the time axis. Each is executed sequentially. Although the method of dividing the singing voice V into a plurality of unit sections is arbitrary, for example, as illustrated in FIG. 7, the singing voice V is divided into a plurality of unit sections (A to C melos, It is possible to classify into rust 1 and rust 2). Each of the K unit intervals corresponds to one group of image data dB. For each of the K unit sections of the singing voice V, the information generating unit 32 selects 1 corresponding to M impression indexes Y1 to YM of the unit section among the plurality of image data dB of the group corresponding to the unit section. Pieces of image data dB are selected. That is, one image data dB is selected for each unit section of the singing voice V, and finally, presentation data QA including K image data dB corresponding to different unit sections is generated. Specifically, in the singing voice V, one piece of image data dB (see FIG. 7) from the “topping” group according to the M impression indexes Y1 to YM specified from the unit section of “A to C melody”. In the example, “strawberry” image) is selected, and one piece of image data dB from the “cream” group (“in the example of FIG. 7,“ in the illustration of FIG. 7 ”) according to the M impression indexes Y1 to YM in the unit section of“ rust 1 ”. "Whipped cream" image) is selected, and one piece of image data dB ("disc-like" in the example of FIG. 7) from the "base" group according to the M impression indices Y1 to YM of the unit section of "Rust 2" "Sponge" image) is selected.

  The presentation processing unit 26 causes the display device 18 to display an image obtained by combining the K pieces of image data dB included in the presentation data QA after the song singing is completed. Specifically, as illustrated in FIG. 7, an image of “cake”, which is a combination of “topping” image data dB, “cream” image data dB, and “base” image data dB, is displayed on the display device 18. Is displayed. Since the image data dB of each unit section is selected according to the auditory impression of the unit section, the content of the image of the article displayed according to the presentation data QA (the mode of each element constituting the article) is the unit section. It changes according to the auditory impression. Therefore, it is possible to provide interest by diversifying the images presented to the user. Note that the content of the image displayed by the combination of the plurality of image data dB is not limited to the above example (cake). For example, in a configuration in which a character such as an avatar representing the user is displayed, an image of each element constituting the character (for example, each element such as clothes or hairstyle, each element such as eyes and mouth constituting the face) is shown. The image data dB is selected for each unit section of the singing voice V.

  In the above example, the image data dB is selected for each unit section obtained by dividing the singing voice V on the time axis. However, it is also possible to select the image data dB for each element other than the unit section. For example, a configuration in which the information generation unit 32 selects the image data dB corresponding to the impression index Ym of the auditory impression for each of M types of auditory impressions (that is, for each auditory impression) may be employed.

(2) A plurality of image data dB prepared in advance is classified into a plurality of groups (categories), and M impression indexes Y1 to YM are selected from one group selected under a predetermined condition among the plurality of groups. The corresponding image data dB can be selected by the information generation unit 32 as the presentation data QA. The condition for selecting one group is arbitrary. For example, a configuration in which the image data dB is selected from a group designated by an operation on the input device 14 by a user among a plurality of groups, or a user among a plurality of groups. A configuration in which the image data dB is selected from a group selected according to the attribute information (for example, age, sex, etc.) is suitable. It is also possible to select a group of image data dB according to the attribute information of a plurality of users.

(3) In the above description, the conversion data WB that represents the structural equation that defines the correlation between each impression index Ym and the image data dB has been exemplified. However, the impression index Ym and the image data dB are associated with each other. It is also possible to use the data table as the conversion data WB.

<Fourth embodiment>
FIG. 8 is a configuration diagram of an acoustic analysis device 100D of the fourth embodiment. As illustrated in FIG. 8, the acoustic analysis device 100D of the fourth embodiment is similar to the acoustic analysis device 100B (FIG. 5) of the second embodiment, and the presentation data QA corresponding to the M impression indexes Y1 to YM. This is a configuration in which an information generation unit 32 for generating the information is added to the first embodiment. The extraction of N feature indices X1 to XN by the feature extraction unit 22 and the setting of M relational expressions F1 to FM by the relational expression setting unit 40 are the same as in the first embodiment. Therefore, the same effect as that of the first embodiment is realized in the fourth embodiment.

  In 4th Embodiment, the log | history data H which shows the log | history of the hearing impression of the singing voice V are memorize | stored in the memory | storage device 12 for every user. As illustrated in FIG. 8, the history data H includes user information hA and impression history hB. The user information hA includes identification information and attribute information (for example, age and sex) of the user who uttered the singing voice V. The impression history hB is a time series of each impression index Ym specified by the impression specifying unit 24 in the past from the user's singing voice V. When M impression indexes Y1 to YM (singing style information S) of the singing voice V are specified, the impression specifying unit 24 adds the impression index Y1 to the impression history hB of the history data H of the user who uttered the singing voice V. Add ~ YM. As understood from the above description, the history data H can be rephrased as time-series data expressing temporal transition of each user's singing style.

  The storage device 12 of the fourth embodiment stores a plurality of property data dC that represents the properties of the user. Specifically, the property data dC represents a character string that represents the user's property. A user's property is a user's property (temperament, character) and state (for example, mental or physical situation). For example, a plurality of personalities defined by known personality classifications (for example, Kretschmer temperament classification, Jung classification, Enneagram classification) are expressed by the property data dC.

  The information generation part 32 of 4th Embodiment estimates the property of the said user according to the auditory impression which the impression specific | specification part 24 specified about the user's song voice V in the past. Specifically, the information generation unit 32 selects, as the presentation data QA, the property data dC corresponding to the auditory impression history indicated by the user history data H among the plurality of property data dC stored in the storage device 12. . The conversion data WC stored in the storage device 12 is used for the selection of the property data dC (estimation of the user's property) by the information generator 32.

  The conversion data WC defines the relationship between the impression history hB (auditory impression time series) and the property data dC. Specifically, the conversion data WC of the fourth embodiment is a data table in which impression history hB (hB1, hB2,...) And property data dC (dC1, dC2,...) Are associated with each other. . For example, as illustrated in FIG. 9, the characteristic data dC of “circular temperament” in the Kretschmer temperament classification corresponds to the impression history hB in which light and dark (bright / dark) appear alternately in the time series of the impression index Ym related to light and dark. To do. In addition, as illustrated in FIG. 10, the impression history hB that changes from a strong (severe) voice to a quiet voice in the time series of the impression index Ym related to activity (strong / quiet) is “I am tired today. Corresponding to property data dC indicating a state such as “?”.

  The user sings the music after designating his / her identification information through an operation on the input device 14. The information generating unit 32 sets the property data associated with the impression history hB of the user history data H specified by the identification information among the plurality of property data dC stored in the storage device 12 by the conversion data WC. dC is specified as the presentation data QA. The presentation processing unit 26 causes the display device 18 to display the presentation data QA generated by the information generation unit 32. As understood from the above description, in the fourth embodiment, the result (property data dC) of estimating the user's property with reference to the auditory impression of the singing voice V is displayed on the display device 18. The user can confirm the estimation result of his / her property by visually recognizing the image displayed on the display device 18. In the fourth embodiment, since the character of the speaker is estimated using the time series (impression history hB) of each impression index Ym of the singing voice V, it is appropriate to take into account the temporal change of the auditory impression of the singing voice V. There is an advantage that a proper property can be estimated.

  In addition, in the technique of patent document 1 or patent document 2, the objective skill of the singing which paid attention only to the difference of the feature-value between model singing voice and the singing voice of evaluation object is only evaluated. According to 4th Embodiment, since a user's property is estimated and shown according to the auditory impression of the singing voice V, it is possible to give a production effect and interest property to a user. In addition, the result of estimating the user's properties from the singing voice V in the fourth embodiment is used for managing the mental / physical state of the user (for example, psychological counseling, health management, therapy, self-development). It is also possible to do. It is also possible to learn a singing style that can give a desired impression to others by adjusting the singing style so that his / her properties presented on the display device 18 approach the target.

<Modification of Fourth Embodiment>
(1) A plurality of property data dC prepared in advance is classified into a plurality of groups (categories), and history data H (impression history hB) is selected from one group selected under a predetermined condition among the plurality of groups. It is also possible for the information generating unit 32 to specify the corresponding property data dC. The condition for selecting one group is arbitrary. For example, the configuration in which the property data dC is selected from the group specified by the operation of the input device 14 by the user among a plurality of groups, or the user among the plurality of groups. A configuration in which the property data dC is selected from a group selected according to the attribute information (for example, age and sex) is suitable. It is also possible to select a group of property data dC according to the attribute information of a plurality of users.

(2) The content of the impression history hB of the history data H is not limited to the above example (time series of impression index Ym). For example, it is also possible to use the frequency and variation rate (change amount within unit time) for each numerical value of the impression index Ym as the impression history hB. In addition, the history data H is generated by using the time series of the impression index Ym of a specific section (for example, rust) in the music as the impression history hB, or for each specific period (for example, every day, every week, every month). Alternatively, a configuration for generating the history data H may be employed. Further, considering that the way of singing may differ depending on the tune of the music, a configuration in which the history data H is generated for each music (or for each genre of music) is also suitable.

(3) In the above description, the property data dC indicating the character string indicating the user's property is illustrated, but image data of an image (for example, a portrait or character) representing the user's property is used as the property data dC. It is also possible to do. In addition, a configuration in which a user or a celebrity who shares the property data dC is presented, or a configuration in which a property opposite to the property indicated by the property data dC is proposed to the user may be employed.

<Fifth Embodiment>
FIG. 11 is a configuration diagram of an acoustic analysis device 100E according to the fifth embodiment. As illustrated in FIG. 11, the acoustic analysis device 100E according to the fifth embodiment includes elements similar to those in the first embodiment (the feature extraction unit 22, the impression identification unit 24, the presentation processing unit 26, and the related expression setting unit 40). In this configuration, a target setting unit 42 and an analysis processing unit 44 are added. Extraction of N feature indexes X1 to XN by the feature extraction unit 22, specification of M impression indexes Y1 to YM by the impression specification unit 24, and setting of M related equations F1 to FM by the related equation setting unit 40 Is the same as in the first embodiment. Therefore, the same effect as that of the first embodiment is also realized in the fifth embodiment.

  The target setting unit 42 in FIG. 11 sets a target value Am (A1 to AM) for each of the M impression indices Y1 to YM. Specifically, the target setting unit 42 variably sets each target value Am in accordance with an instruction from the user to the input device 14.

  For example, the presentation processing unit 26 of the fifth embodiment causes the display device 18 to display the operation screen 80 of FIG. 12 that accepts an instruction of the target value Am of each impression index Ym. The operation screen 80 includes an operator image 82 corresponding to each of the M impression indices Y1 to YM (M = 3 in the illustration of FIG. 12). Each operation element image 82 is an image of a slider-type operation element that moves in response to an instruction from the user to the input device 14 and accepts an instruction of a target value Am by the user. The target setting unit 42 sets a target value Am for each impression index Ym according to the position of each operator image 82. Note that each of the plurality of operation element images 82 on the operation screen 80 can be moved individually, but each operation element image 82 can also be moved in conjunction with each other.

  The analysis processing unit 44 in FIG. 11 specifies the acoustic feature (feature index Xn) to be changed in order to bring each impression index Ym specified for the singing voice V by the impression specifying unit 24 close to the target value Am. The analysis processing unit 44 according to the fifth embodiment generates analysis data QB that designates acoustic features that should be changed to bring each impression index Ym closer to the target value Am and the direction (increase / decrease) of the change. The presentation processing unit 26 causes the display device 18 to display the contents of the analysis data QB generated by the analysis processing unit 44 (acoustic features to be changed and change directions). Therefore, the user can grasp an improvement point for bringing his / her song close to the target auditory impression. As understood from the above description, the presentation of the analysis data QB corresponds to singing instruction for realizing a target auditory impression.

  The analysis processing unit 44 of the fifth embodiment minimizes a numerical value (hereinafter referred to as “total difference”) δ obtained by summing the absolute value | Ym−Am | of the difference between the impression index Ym and the target value Am for M auditory impressions. The acoustic features that should be changed in order to be converted are identified from the N types of acoustic features. Specifically, the analysis processing unit 44 calculates the total difference δ when it is assumed that the feature index Xn of any one of N types of acoustic features is changed by a predetermined change amount p, as the acoustic feature to be changed. Calculate multiple cases with different values and compare them with each other to generate analysis data QB that specifies the acoustic feature to be changed and the direction (increase / phenomenon) of the change when the total difference δ is minimized To do.

数式(A)のうち変化量ｐと係数ａnmとの乗算値の減算は、特徴指標Ｘnを変化量ｐだけ変化させる処理に相当する。解析処理部４４は、変化量ｐの正負を反転させた２通りの場合（ｐ＝±１）について、特徴指標Ｘnを変化量ｐだけ変化させた数式(A)の合計差分δを算定し、合計差分δが最小化された場合の変化対象の音響特徴と変化の方向（変化量ｐの正負）とを特定する。 The total difference δ when any one feature index Xn is changed by the change amount p is expressed by the following formula (A).

The subtraction of the multiplication value of the change amount p and the coefficient anm in the formula (A) corresponds to a process of changing the feature index Xn by the change amount p. The analysis processing unit 44 calculates the total difference δ of the mathematical formula (A) in which the characteristic index Xn is changed by the change amount p in two cases where the sign of the change amount p is reversed (p = ± 1), The acoustic feature to be changed and the direction of change (positive or negative of the change amount p) when the total difference δ is minimized are specified.

For example, paying attention to the impression index Y1 for young children and the impression index Y2 for clearness, the feature index X1 indicating the vibrato depth and the feature index X2 indicating the accuracy of the pitch (M = N = 2), the relational expression F1 and The relational expression F2 is assumed as follows (a11 = 0.7, a21 = 0.3, a12 = −0.4, a22 = 0.7).

  Assume that the target value A1 is 4 for the impression index Y1 being 5 and the target value A2 is 6 for the impression index Y2 being 4 ((Y1, Y2) = (5,4), (A1, A2) = (4,6)). That is, the feature index Xn to be changed is searched in order to realize “child-like and clean singing” (Y1: 5 → 4, Y2: 4 → 6) as compared with the evaluated singing voice V.

[1] p = 1 (assuming an increase in the feature index Xn)
・ Condition 1a: Assuming a change in the characteristic index X1 (when increasing the vibrato depth)
δ = | A1-Y1-p.a11 | + | A2-Y2-p.a12 |
= | 4-5-1 · 0.7 | + | 6-4-1 · (−0.4) |
= 1.7 + 2.4 = 4.1
・ Condition 1b: Assuming a change in the characteristic index X2 (when increasing pitch accuracy)
δ = | A1-Y1-p.a21 | + | A2-Y2-p.a22 |
= | 4-5-1 · 0.3 | + | 6-4-1 · 0.7 |
= 1.3 + 1.3 = 2.6
[2] p = −1 (assuming a decrease in the feature index Xn)
・ Condition 2a: Assuming a change in the feature index X1 (when reducing the vibrato depth)
δ = | A1-Y1-p.a11 | + | A2-Y2-p.a12 |
= | 4-5-(-1) · 0.7 | + | 6-4-(-1) · (−0.4) |
= 0.3 + 1.6 = 1.9
・ Condition 2b: Assuming a change in the characteristic index X2 (when reducing pitch accuracy)
δ = | A1-Y1-p.a21 | + | A2-Y2-p.a22 |
= | 4-5-(-1) .0.3 | + | 6-4-(-1) .0.7 |
= 0.7 + 2.7 = 3.4

  As described above, the total difference δ becomes the minimum value (δ = 1.9) under the condition 2a for reducing the feature index X1. Therefore, the analysis processing unit 44 specifies the analysis data QB that designates “decrease in vibrato depth” (acoustic feature = depth of vibrato, change direction = decrease) as a condition for bringing the singing voice V closer to the target (A1, A2). Is generated. As understood from the above description, the characteristic index Xn having a large coefficient anm in the relational expression Fm of the impression index Ym different from the target value Am (that is, the characteristic index Xn having a relatively large influence on the impression index Ym) is the impression. The index Ym is preferentially selected as the characteristic index Xn to be changed in order to bring the index Ym close to the target value Am. The user who confirms the analysis result (analysis data QB) by the analysis processing unit 44 on the display device 18 is a policy of “decrease the vibrato depth” in order to realize the “child-friendly and clean singing”. Can be grasped as the best.

  By the way, for example, in Japanese Patent Application Laid-Open No. 2008-20798, a difference between a reference value indicating an exemplary singing and a feature amount of a singing voice to be evaluated is sequentially evaluated, and “sounding is clear”, “clearly”, The technique of displaying the comment of singing instruction according to the result of evaluation is disclosed. However, the above technique only presents the user with improvements for approaching an exemplary song. That is, even if it sings so as to match the comment of the singing instruction, it only approaches an exemplary singing, and cannot approach a singing that can make the listener feel a specific auditory impression. According to the fifth embodiment, as can be understood from the above-described examples, there is an advantage that the user can grasp the optimum improvement point (acoustic feature) for bringing the singing voice V close to the target auditory impression. In addition, by improving the user's singing so as to approach the goal, application as a method of self-realization and health maintenance (psychotherapy and fitness) can also be expected.

<Sixth Embodiment>
FIG. 13 is a configuration diagram of an acoustic analysis device 100F according to the sixth embodiment. As illustrated in FIG. 13, the acoustic analysis device 100 </ b> F according to the sixth embodiment includes the same elements (feature extraction unit 22, impression identification unit 24, presentation processing unit 26, relational expression setting unit 40, target as in the fifth embodiment. The sound processing unit 46 is added to the setting unit 42 and the analysis processing unit 44). Extraction of N feature indexes X1 to XN by the feature extraction unit 22, specification of M impression indexes Y1 to YM by the impression specification unit 24, and setting of M related equations F1 to FM by the related equation setting unit 40 Is the same as in the first embodiment. Therefore, the sixth embodiment can achieve the same effect as that of the first embodiment.

  Similar to the fifth embodiment, the target setting unit 42 of the sixth embodiment sets a target value Am of each impression index Ym in accordance with, for example, an instruction from the user. The analysis processing unit 44 uses the analysis data QB for designating acoustic features (feature index Xn) to be changed in order to bring each impression index Ym specified by the impression specifying unit 24 for the singing voice V close to the target value Am, in the fifth embodiment. Generate in the same way as

  The sound processing unit 46 in FIG. 13 performs an acoustic process on the singing voice V that changes the acoustic feature specified by the analysis processing unit 44. Specifically, the acoustic processing unit 46 converges so that the acoustic feature specified by the analysis data QB generated by the analysis processing unit 44 changes (increases / decreases) in the direction specified by the analysis data QB. Acoustic processing is performed on the singing voice V collected by the sound device 16. That is, among the N characteristic indexes X1 to XN of the singing voice V, the characteristic index Xn (that is, the impression index) having a large coefficient (contribution to the impression index Ym) anm in the relational expression Fm of the impression index Ym different from the target value Am. The characteristic index Xn) that can effectively bring Ym close to the target value Am is preferentially changed by the acoustic processing by the acoustic processing unit 46.

  For specific acoustic processing executed on the singing voice V, a known acoustic processing technique corresponding to the type of acoustic feature to be changed is arbitrarily employed. For example, when the characteristic index Xn for bringing the impression index Ym related to clearness to the target value Am is “sense of noise”, the acoustic processing unit 46 performs acoustic processing (noise addition processing) for adding a noise component to the singing voice V. Run. Further, for example, when the analysis data QB designates “decrease in vibrato depth” as illustrated in the fifth embodiment, the acoustic processing unit 46 suppresses minute fluctuations in pitch in the singing voice V. An acoustic process is performed on the singing voice V. The singing voice V after processing by the acoustic processing unit 46 is reproduced from, for example, the sound emitting device 17 (speaker or headphones). Note that, instead of (or along with) reproduction of the singing voice V, it is also possible to generate a file of the singing voice V after processing by the acoustic processing unit 46.

  As understood from the above description, according to the fifth embodiment, it is possible to adjust the auditory impression of the singing voice V to a desired impression (auditory impression according to the target value Am). Particularly in the illustration of the fifth embodiment, since each target value Am is variably set in accordance with an instruction from the user, there is an advantage that the singing voice V having a desired auditory impression can be generated.

  The characteristic index Xn specified by the analysis data QB (hereinafter referred to as “priority index” for convenience) is sufficiently varied in the singing voice V (that is, the impression index Ym sufficiently approximates the target value Am). It may not be possible. For example, even if the analysis data QB specifies “increase in the depth of vibrato”, if the singing voice V does not include a section in which the pitch is maintained over a length of time that vibrato can be added, the priority index is The impression index Ym cannot be made sufficiently close to the target value Am by increasing a certain “depth of vibrato”. In the above case, the acoustic processing unit 46 sets the priority index in the order effective for increasing the impression index YM from the N feature indices X1 to XN of the singing voice V to the target value Am (ascending order of the total difference δ). The acoustic processing for the singing voice V is executed so that the feature index Xn positioned at the next position changes. According to the above configuration, each impression index Ym can be effectively brought close to the target value Am regardless of the characteristics of the singing voice V.

  Incidentally, for example, Japanese Patent Application Laid-Open No. 2011-09597 discloses a configuration in which a plurality of types of control variables applied to speech synthesis are set according to instructions from a user. However, with the above technique, there is a problem that it is difficult for the user to clearly understand which of the multiple types of control variables should be adjusted to achieve a desired audio impression. is there. In the sixth embodiment, the target value Am of each auditory impression is set in accordance with an instruction from the user. For example, even a user who does not have specialized knowledge about the speech synthesis control variable has a desired auditory impression singing voice. There is an advantage that V can be generated effectively (instruction by the user is facilitated).

<Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples may be appropriately combined.

(1) In each of the above-described forms, the auditory impression is specified for the singing voice V over the entire section of the music, but the auditory impression (M impressions) for each of a plurality of sections obtained by dividing the singing voice V on the time axis. It is also possible to specify the indices Y1 to YM) sequentially. When the auditory impression is sequentially specified for each section of the singing voice V, the presentation data QA exemplified in the second to fourth embodiments and the analysis data QB exemplified in the fifth and sixth embodiments are sung. A configuration may also be adopted in which updating is performed sequentially (in real time) for each section according to the auditory impression of each section of the voice V.

(2) In each embodiment described above, elements for analyzing the singing voice V picked up by the sound pickup device 16 (feature extraction unit 22, impression identification unit 24, presentation processing unit 26, information generation unit 32, target setting unit 42, An acoustic analysis apparatus 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G) including both an analysis processing unit 44 and an acoustic processing unit 46) and a related formula setting unit 40 for setting each related formula Fm is provided. Although illustrated, it is also possible to mount the relational expression setting unit 40 in a separate device from other elements.

  For example, as illustrated in FIG. 14, the acoustic analysis device 110 and the acoustic analysis device 120 that communicate with each other via the communication network 200 (for example, the Internet) can share the functions illustrated in the above-described embodiments. is there. The acoustic analysis device (relevant formula setting device) 110 uses the reference data group DR and the relationship description data DC to set the M related formulas F1 to FM in the same manner as in the first embodiment. Part 40. For example, the acoustic analysis device 110 is realized by a server device connected to the communication network 200. As illustrated in FIG. 14, the M related formulas F1 to FM set by the acoustic analysis device 110 (the related formula setting unit 40) are transferred to the acoustic analysis device 120 via the communication network 200. It is also possible to transfer the M relational expressions F1 to FM from the acoustic analysis device 110 to the plurality of acoustic analysis devices 120 in common. The acoustic analysis device 120 includes a feature extraction unit 22 and an impression identification unit 24, and sings in the same manner as in the first embodiment using M related expressions F1 to FM transferred from the acoustic analysis device 110. By analyzing the voice V, the auditory impression (M impression indices Y1 to YM) of the singing voice V is specified. In the acoustic analysis device 120, an information generation unit 32 similar to those in the second to fourth embodiments, a target setting unit 42, and an analysis processing unit 44 similar to those in the fifth and sixth embodiments may be installed. . In the configuration of FIG. 14, it is not necessary to cause the acoustic analysis device 120 to hold the reference data group DR and the relationship description data DC and to set each relational expression Fm, so that the configuration and processing of the acoustic analysis device 120 are simplified. There is an advantage that.

(3) In the second to fourth embodiments, the control data for controlling various devices can be set as the presentation data QA according to the auditory impression of the singing voice V. The control data is applied to, for example, control of an image (background image) displayed on the display device 18 during song singing, playback of accompaniment sounds played by a playback device (karaoke device), and control of effects such as lighting devices. Is done. It is also possible to change foods and drinks that can be ordered in a store such as a karaoke store according to the presentation data QA. It is also possible to apply the auditory impression (M impression indices Y1 to YM) of the singing voice V to singing evaluation (scoring). For example, a configuration in which a variable applied to singing evaluation is adjusted according to the presentation data QA, and a similarity (dissimilarity) between an impression registered in advance for each song and an auditory impression of the singing voice V is reflected in the evaluation result. A configuration (for example, a configuration in which the points are increased as the two are similar) is preferably employed.

(4) In the fifth embodiment and the sixth embodiment, each target value Am is set according to an instruction from the user, but the method of setting the target value Am is not limited to the above examples. For example, a configuration in which the target value Am (A1 to AM) is selected in advance for each piece of music and the target setting unit 42 selects the target value Am of the music that the user actually sings may be employed. Also, the target setting unit 42 can variably set each target value Am according to the attributes of the music sung by the user (main melody, genre, singer, etc.).

(5) In each form mentioned above, although the singing voice V which the user sang the music was illustrated, the analysis object is not limited to the singing voice V. For example, auditory impressions (M impression indicators) are analyzed by using each related expression Fm for voices such as conversation sounds, musical instrument performance sounds (musical sounds), and synthesized voices generated by voice synthesis technology (singing voices and conversational sounds). It is also possible to specify Y1-YM). Moreover, it is emitted from any sound system including sound played back at each point under a remote conference system that sends and receives sound between remote locations (for example, conversation sound at a conference) and sound emitting devices such as speakers. The auditory impression can be specified for the sound to be heard. As understood from the above description, the specific content (type) of the sound (analysis target sound) to be analyzed in the present invention, the principle of pronunciation, and the like are arbitrary.

100 (100A, 100B, 100C, 100D, 100E, 100F, 100G), 110, 120 ... acoustic analysis device, 10 ... arithmetic processing device, 12 ... storage device, 14 ... input device, 16 ... sound collecting Device: 18 ... Display device, 22 ... Feature extraction unit, 24 ... Impression identification unit, 26 ... Presentation processing unit, 32 ... Information generation unit, 40 ... Related expression setting unit, 42 ... Target setting unit 44 …… Analysis processing unit, 46 …… Sound processing unit.

(5) In each form mentioned above, although the singing voice V which the user sang the music was illustrated, the analysis object is not limited to the singing voice V. For example, auditory impressions (M impression indicators) are analyzed by using each related expression Fm for voices such as conversation sounds, musical instrument performance sounds (musical sounds), and synthesized voices generated by voice synthesis technology (singing voices and conversational sounds). It is also possible to specify Y1-YM). For example, in the analysis of the performance sound of a musical instrument, an impression index Ym such as light and dark and clearness can be specified as in the above-described embodiments. Moreover, it is emitted from any sound system including sound played back at each point under a remote conference system that sends and receives sound between remote locations (for example, conversation sound at a conference) and sound emitting devices such as speakers. The auditory impression can be specified for the sound to be heard. As understood from the above description, the specific content (type) of the sound (analysis target sound) to be analyzed in the present invention, the principle of pronunciation, and the like are arbitrary.

Claims (5)

A plurality of reference data in which an impression index indicating an auditory impression of a reference sound and a feature index indicating an acoustic feature of the reference sound are associated with each other, and a correspondence relationship between the auditory impression and a plurality of types of acoustic features are defined. Relational expression setting means for setting a relational expression that expresses the relationship between the impression index of the auditory impression and the feature index of each acoustic feature in the correspondence defined by the relational description data using the relation description data An acoustic analysis apparatus comprising: The acoustic analysis apparatus according to claim 1, wherein the relationship description data defines a correspondence relationship between the auditory impression and the plurality of types of acoustic features via a plurality of intermediate elements included in the auditory impression. The acoustic analysis apparatus according to claim 1, wherein the relational expression setting unit sets the relational expression for each of a plurality of types of auditory impressions. The acoustic analysis apparatus according to claim 1, wherein the relational expression setting unit acquires the reference data and updates a predetermined relational expression using the reference data. A feature extraction means for extracting a feature index of the sound to be analyzed;
A plurality of reference data in which an impression index indicating an auditory impression of a reference sound and a feature index indicating an acoustic feature of the reference sound are associated with each other, and a correspondence relationship between the auditory impression and a plurality of types of acoustic features are defined. The relational expression calculated using the relationship description data and expressing the relationship between the impression index of the auditory impression and the feature index of the plurality of types of acoustic features in the correspondence defined by the relationship description data, An acoustic analysis apparatus comprising: an impression specifying unit that calculates an impression index of the analysis target sound by applying the feature index extracted by the feature extraction unit.

Images