WO2019230236A1

WO2019230236A1 - Binaural sound processing apparatus, information processing apparatus, and program

Info

Publication number: WO2019230236A1
Application number: PCT/JP2019/016469
Authority: WO
Inventors: 龍太岡崎; 直輝齋藤; 岡本　直樹; 尚美北村
Original assignee: 株式会社資生堂
Priority date: 2018-05-28
Filing date: 2019-04-17
Publication date: 2019-12-05
Also published as: TW202005417A; JP7195314B2; JPWO2019230236A1

Abstract

A binaural sound processing apparatus is provided with: a storage device in which application identification information for identifying each of a plurality of applications and binaural sound source information in which a sound when each application is applied to the skin is binaurally recorded are stored in association with each other; a means for determining application identification information that identifies at least one application from among the plurality of applications; a means for selecting the binaural sound source information associated with the determined application identification information; and a means for outputting the selected binaural sound source information.

Description

Binaural sound processing device, information processing device, and program

The present invention relates to a binaural sound processing device, an information processing device, and a program.

In general, there are individual differences in the tactile sensation that an applied object (for example, cosmetics) applied to a human body gives to an applied person.
For example, consumers of cosmetics select cosmetics in consideration of not only the cosmetic effect obtained by using the cosmetics but also the tactile sensation (for example, the touch) when using the cosmetics.
On the other hand, cosmetic sellers promote the sale of cosmetics by appealing to consumers both of beauty effects and tactile sensations.

Japanese Patent Application Laid-Open No. 2003-186762 discloses a recipe for cosmetics according to a parameter input by a consumer in order to input a parameter related to the touch to support selection of the cosmetic in consideration of the touch. The technique which presents is disclosed.

However, the technique disclosed in Japanese Patent Application Laid-Open No. 2003-18662 cannot allow a consumer to experience tactile sensation. Therefore, the tactile sensation that the consumer feels when actually using the cosmetics corresponding to the presented recipe may be different from the tactile sensation that the consumer imagined when inputting the parameters.
In particular, in online shopping, which has been rapidly spreading in recent years, consumers cannot experience the touch of cosmetics.

That is, conventionally, the consumer cannot experience the feel of the coating before using the coating.

An object of the present invention is to allow a consumer to experience the feel of a coating before using the coating.

One embodiment of the present invention provides:
Means for storing the application identification information for identifying each of the plurality of applications and the binaural sound source information in which the sound when each application is applied to the skin is binaurally recorded;
Means for determining application identification information for identifying at least one application from a plurality of applications;
Means for selecting binaural sound source information associated with the determined application identification information;
Means for outputting the selected binaural sound source information;
It is a binaural sound processing device.

According to the present invention, it is possible to allow the consumer to experience the feel of the coating before using the coating.

It is the schematic which shows the structure of the speech processing system of this embodiment. It is a block diagram which shows the structure of the speech processing system of FIG. It is explanatory drawing of the outline | summary of this embodiment. It is a figure which shows the data structure of the applied material information database of this embodiment. It is a flowchart of the information processing of this embodiment. It is a figure which shows the example of a screen displayed in the information processing of FIG. 10 is an explanatory diagram of an outline of Modification 1. FIG. It is a figure which shows the data structure of the user information database of the modification 1. 10 is a flowchart of information processing according to Modification 1; It is a figure which shows the example of a screen displayed in the information processing of FIG. It is explanatory drawing of the outline | summary of the modification 2. FIG. 10 is a flowchart of information processing according to a second modification. It is explanatory drawing of the outline | summary of the modification 3. 10 is a flowchart of information processing according to Modification 3; It is explanatory drawing of the outline | summary of the modification 4. 10 is a flowchart of information processing according to Modification 4; It is a figure which shows the example of a screen displayed in the information processing of FIG. 10 is an explanatory diagram of an outline of modification example 5. FIG. 10 is a flowchart of information processing according to Modification 5;

Hereinafter, an embodiment of the present invention will be described in detail based on the drawings. Note that in the drawings for describing the embodiments, the same components are denoted by the same reference symbols in principle, and the repetitive description thereof will be omitted.

(1) Configuration of Voice Processing System The configuration of the voice processing system will be described. FIG. 1 is a schematic diagram illustrating a configuration of a voice processing system according to the present embodiment. FIG. 2 is a block diagram showing a configuration of the voice processing system of FIG.

As shown in FIGS. 1 and 2, the sound processing system 1 includes a binaural sound processing device 10, a sound output device 20, and a server 30.
The binaural sound processing device 10 and the audio output device 20 are connected by wired communication or wireless communication. The binaural sound processing apparatus 10 and the server 30 are connected via a network (for example, the Internet or an intranet) NW.
The user U can listen to the sound (for example, binaural sound) processed by the binaural sound processing device 10 via the sound output device 20.

The binaural sound processing device 10 is configured to generate a binaural sound signal.
The binaural sound processing device 10 is an example of an information processing device that transmits a request to the server 30. The binaural sound processing device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

The audio output device 20 is configured to output a binaural sound corresponding to the audio signal transmitted from the binaural sound processing device 10. The audio output device 20 is, for example, a headphone or an earphone.

The server 30 is an example of an information processing apparatus that provides the binaural sound processing apparatus 10 with a response corresponding to the request transmitted from the binaural sound processing apparatus 10. The server 30 is, for example, a web server.

(1-1) Configuration of Binaural Sound Processing Device The configuration of the binaural sound processing device 10 will be described with reference to FIG.

As shown in FIG. 2, the binaural sound processing device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14.

The storage device 11 is configured to store a program and data. The storage device 11 is, for example, a combination of a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage (for example, a flash memory or a hard disk).

The programs include, for example, the following programs.
An OS (Operating System) program An application program for executing information processing (for example, an application for binaural sound processing)

The data includes, for example, the following data.
-Database referenced in information processing-Data obtained by executing information processing (that is, information processing execution result)

The processor 12 is configured to realize the function of the binaural sound processing device 10 by starting a program stored in the storage device 11. The processor 12 is an example of a computer.

The input / output interface 13 is configured to acquire a user instruction from an input device connected to the binaural sound processing apparatus 10 and to output information to an output device connected to the binaural sound processing apparatus 10.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

The communication interface 14 is configured to control communication between the binaural sound processing device 10 and the server 30.

(1-2) Server Configuration The configuration of the server 30 will be described with reference to FIG.

As shown in FIG. 2, the server 30 includes a storage device 31, a processor 32, an input / output interface 33, and a communication interface 34.

The storage device 31 is configured to store programs and data. The storage device 31 is, for example, a combination of ROM, RAM, and storage (for example, flash memory or hard disk).

The programs include, for example, the following programs.
-OS program-Application program that executes information processing

The data includes, for example, the following data.
・ Database referenced in information processing ・ Results of information processing

The processor 32 is configured to realize the function of the server 30 by starting a program stored in the storage device 31. The processor 32 is an example of a computer.

The input / output interface 33 is configured to acquire a user instruction from an input device connected to the server 30 and output information to an output device connected to the server 30.
The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof.
The output device is, for example, a display.

The communication interface 34 is configured to control communication between the server 30 and the binaural sound processing device 10.

(2) Overview of Embodiment An overview of this embodiment will be described. FIG. 3 is an explanatory diagram of the outline of the present embodiment.

The binaural sound processing apparatus 10 stores binaural sound source information. Binaural sound source information is associated with an applied material that can be applied to the human body. The applied product is, for example, a cosmetic that can be applied to the skin. Examples of the cosmetics include makeup cosmetics, skin care cosmetics, hair care cosmetics, and the like. As for the form of cosmetics, solid, liquid, spray form etc. are mentioned, for example.
In the binaural sound source information, a binaural sound that is reproduced by vibration transmitted to the eardrum when each application is applied to the human body is recorded.
When the user U designates a desired application, the binaural sound processing device 10 outputs binaural sound source information associated with the designated application to the audio output device 20.
The audio output device 20 outputs the binaural sound recorded in the binaural sound source information output from the binaural sound processing device 10.

Such a binaural sound of the coating can stimulate the sense of hearing and the tactile sensation that accompanies the hearing, thereby allowing the user U to experience the tactile sensation when using the coating. In the present embodiment, by letting the user U hear the binaural sound of the desired application, it is possible to present a tactile sensation when using the desired application.

(3) Database The database of this embodiment will be described. The following databases are stored in the storage device 11.

(3-1) Applied Product Information Database The applied product information database of this embodiment will be described. FIG. 4 is a diagram showing a data structure of the applied product information database according to the present embodiment.

The applied material information database of FIG. 4 stores applied material information related to the applied material.
The application information database includes an “application ID” field, an “application name” field, a “recommended attribute” field, a “binaural sound source” field, and a “content” field. Each field is associated with each other.

In the “application ID” field, an application ID for identifying the application (an example of “application identification information”) is stored. The application ID is a main key of a record in the application information database (hereinafter referred to as “application information record”).

In the “application name” field, information (for example, text) relating to the name of the application is stored.

In the “recommended attribute” field, information related to user attributes for which the use of the applied product is recommended (hereinafter referred to as “recommended attribute information”) is stored. The “recommended attribute” field includes a “gender” field, an “age” field, and a “skin condition” field.

In the “Gender” field, information on the gender recommended to use the coating is stored.

In the “age” field, information on the age at which the use of the applied product is recommended is stored.

In the “skin state” field, information on the skin state where the use of the applied product is recommended is stored.

In the “binaural sound source” field, a binaural sound source file (an example of “binaural sound source information”) in which a vibration sound when applying a coating is recorded is stored.

In the “content” field, a content file of content related to the applied material is stored. The content includes, for example, at least one of the following.
・ Video content ・ Still image content ・ CG (Computer Graphics) content ・ Audio content

(4) Information processing The information processing of this embodiment is demonstrated. FIG. 5 is a flowchart of information processing according to this embodiment. FIG. 6 is a diagram illustrating an example of a screen displayed in the information processing of FIG.

As shown in FIG. 5, the binaural sound processing device 10 performs designation of application (S100).
Specifically, the processor 12 displays the screen P100 (FIG. 6) on the display.

The screen P100 includes operation objects B100a to B100b.
The operation objects B100a to B100b are objects that receive a user instruction for designating a coating material. An application ID is assigned to the operation objects B100a to B100b.

For example, when the user U operates the operation object B100a, the processor 12 specifies the application ID assigned to the operation object B100a. Thereby, designation | designated of a coating material is received.

After step S100, the binaural sound processing apparatus 10 executes binaural sound source selection (S101).
Specifically, the processor 12 refers to the application information database (FIG. 4), and obtains information on the “binaural sound source” field (that is, a binaural sound source file) associated with the application ID specified in step S100. read out.

After step S101, the binaural sound processing apparatus 10 executes binaural sound reproduction (S102).
Specifically, the processor 12 generates the binaural sound signal recorded in the binaural sound source file by opening the binaural sound source file read in step S101.
The processor 12 transmits the generated audio signal to the audio output device 20.
The audio output device 20 outputs a binaural sound based on the audio signal transmitted from the processor 12.

According to the present embodiment, the user U can experience a tactile sensation when using a desired application by listening to the binaural sound output by the audio output device 20.

(5) Modification A modification of the present embodiment will be described.

(5-1) Modification 1
Modification 1 will be described. The modified example 1 is an example in which a binaural sound corresponding to a combination of a coating material and a user attribute of the user U is output.

(5-1-1) Overview of Modification 1 An overview of Modification 1 will be described. FIG. 7 is an explanatory diagram of an outline of the first modification.

In the binaural sound processing apparatus 10 of FIG. 7, binaural sound source information is stored. The binaural sound source information is associated with a combination of an application and a user attribute (for example, an application recommended to be used from the user attribute of the user U).
When the user U designates a desired application, the binaural sound processing apparatus 10 outputs binaural sound source information associated with the combination of the designated application and the user attribute of the user U to the audio output device 20.
The audio output device 20 outputs the binaural sound recorded in the binaural sound source information output from the binaural sound processing device 10.

In the first modification, among the desired coating materials of the user U, by applying a binaural sound of the coating material to which the user attribute of the user U is associated, a coating material recommended for the user U is applied. A feeling of touch can be presented.

(5-1-2) Database A database according to Modification 1 will be described.

(5-1-2-1) User Information Database A user information database according to the first modification will be described. FIG. 8 is a diagram illustrating a data structure of the user information database according to the first modification.

The user information database in FIG. 8 includes a “user ID” field, a “user name” field, a “user attribute” field, and an “behavior history” field.
Each field is associated with each other.

In the “user ID” field, a user ID for identifying the user U (an example of “user identification information”) is stored. The user ID is a primary key of a record in the user information database (hereinafter referred to as “user information record”).

In the “user name” field, information (for example, text) related to the user name of the user U is stored.

In the “user attribute” field, user attribute information regarding the attribute of the user U is stored. The “user attribute” field includes a “gender” field, an “age” field, and a “skin condition” field.

In the “gender” field, information on the gender of the user U is stored.

In the “age” field, information about the age of the user U is stored.

In the “skin state” field, skin state information regarding the skin state of the user U is stored. The “skin state” field is stored in the user information database by at least one of the following methods.
The processor 12 stores the skin condition information input by the user U in the “skin condition” field.
The processor 12 stores the skin condition information transmitted from the device (for example, a smartphone) capable of acquiring the skin condition information via the communication interface 14 in the “skin condition” field.
The processor 12 stores the skin condition information transmitted from the wearable device worn by the user U in the “skin condition” field.

In the “behavior history” field, behavior history information related to the history of the behavior of the user U is stored. The “behavior history” field includes a “browsing history” field and a “purchase history” field.

In the “browsing history” field, browsing history information related to the browsing history of the website (for example, the URL (Uniform Resource Locator) of the website browsed by the user U) is stored.

In the “purchase history” field, purchase history information related to the purchase history of the product (for example, information on the application ID of the application purchased by the user U and information on the purchase date) is stored.

(5-1-3) Information Processing Information processing according to the first modification will be described. FIG. 9 is a flowchart of information processing according to the first modification. FIG. 10 is a diagram illustrating an example of a screen displayed in the information processing of FIG.

As shown in FIG. 9, the binaural sound processing device 10 executes acquisition of user information (S110).
Specifically, the processor 12 displays the screen P110 (FIG. 10) on the display.

The screen P110 includes an operation object B110 and a field object F110.
The field object F110 is an object that accepts input of user information.
The operation object B110 is an object that receives a user instruction for confirming user information input to the field object F110.

For example, when the user U inputs the user ID to the field object F110 and operates the operation object B110, the processor 12 refers to the user information database (FIG. 8) and sets the user ID input to the field object F110. Identifies the associated user information record.

After step S110, the binaural sound processing device 10 executes binaural sound source selection (S111).

In the first example of step S111, the processor 12 refers to the “recommended attribute” field of the application information database (FIG. 4), and includes user attribute information (for example, gender, The application information record corresponding to age and information on skin condition) is specified.
The processor 12 reads the information in the “binaural sound source” field (that is, the binaural sound source file) of the application information record.

In the second example of step S111, the storage device 11 stores an action history of the user U and a correlation model of the applied product that the user U likes. The input of the correlation model is the action history of the user U. The output of the correlation model is application information.
The processor 12 inputs the information of the “behavior history” field (for example, at least one information of the “browsing history” field and the “purchase history” field) included in the user information record specified in step S111 into the correlation model. Application information (for example, application ID) relating to an application recommended for use by the user U is predicted.
The processor 12 reads the information of the “binaural sound source” field (that is, the binaural sound source file) associated with the predicted application ID by referring to the application information database (FIG. 4).

After step S111, the binaural sound processing apparatus 10 performs binaural sound reproduction (S102), as in FIG.

According to the first modification, the user U can experience the tactile sensation when using the recommended application in consideration of his / her attributes by listening to the binaural sound output by the audio output device 20. Can do.

(5-2) Modification 2
Modification 2 will be described. The modified example 2 is an example in which a binaural sound optimized according to the movement of the user U is output.

(5-2-1) Overview of Modification 2 An overview of Modification 2 will be described. FIG. 11 is an explanatory diagram of an outline of the second modification.

As illustrated in FIG. 11, when the user U moves while listening to the binaural sound output from the audio output device 20, the binaural sound processing device 10 corrects the binaural sound according to the movement of the user U.
The audio output device 20 outputs the binaural sound corrected by the binaural sound processing device 10.

(5-2-2) Information Processing Information processing according to the second modification will be described. FIG. 12 is a flowchart of information processing according to the second modification.

As shown in FIG. 12, the binaural sound processing apparatus 10 executes steps S100 to S101 as in FIG.

After step S101, the binaural sound processing device 10 executes motion information acquisition (S120).
Specifically, the binaural sound processing device 10 is connected to a motion sensor (not shown) configured to detect a user's movement. The motion sensor includes, for example, at least one of the following.
An image sensor that detects an image of the user U An acceleration sensor that detects an acceleration of a part of the user U's body (for example, a hand) A gyro that detects an angular acceleration of a part of the user U's body (for example, the hand) Sensor ・ A geomagnetic sensor that detects geomagnetism

The processor 12 calculates the movement of the user U based on information detected by the motion sensor (for example, at least one of an image, acceleration, gyro, and geomagnetism).

After step S120, the binaural sound processing apparatus 10 performs binaural sound correction (S121).
Specifically, the storage device 11 stores correction parameters for each movement of the user U.
The processor 12 corrects the sound characteristics of the binaural sound using the correction parameter corresponding to the motion calculated in step S120. The voice characteristic includes, for example, at least one of the following.
・ Frequency (playback speed as an example)
-Sound pressure-Time difference between left and right binaural sounds (as an example, phase difference)

After step S121, the binaural sound processing device 10 executes step S102 as in FIG.

According to the second modification, the binaural sound processing device 10 corrects the binaural sound according to the movement of the user U. Thereby, it is possible to cause the user U to interactively and actively experience the tactile sensation when using the application according to the movement while listening to the binaural sound.
Further, the user U can listen to the binaural sound optimized according to his / her movement. Thereby, even if the user U moves while listening to the binaural sound, the tactile sensation when using the coating can be correctly transmitted to the user U.
More specifically, by correcting the frequency according to the speed at which the user U strokes his / her skin, an interactive tactile sensation according to the motion of stroking the skin can be presented.
By correcting the time difference between the left and right binaural sounds according to the position and direction in which the user U strokes his / her skin, an interactive tactile sensation according to the position and direction in which the user U strokes can be presented.
By correcting the sound pressure according to the strength with which the user U strokes his / her skin, an interactive tactile sensation according to the strength with which the user U strokes can be presented.
Since the actual sound is canceled by superimposing the opposite phase sound of the actual sound generated when the user U strokes his skin on the binaural sound, the binaural sound corresponding only to the tactile sensation can be presented.

(5-3) Modification 3
Modification 3 will be described. The third modification is an example in which a binaural sound optimized according to the environment of the user U is output.

(5-3-1) Overview of Modification 3 An overview of Modification 3 will be described. FIG. 13 is an explanatory diagram of an outline of the third modification.

As shown in FIG. 13, while listening to the binaural sound output from the audio output device 20, the binaural sound processing device 10 responds to the environment information related to the environment of the user U (that is, the binaural sound reproduction environment). Correct the binaural sound.
The audio output device 20 outputs the binaural sound corrected by the binaural sound processing device 10.

(5-3-2) Information Processing Information processing according to the third modification will be described. FIG. 14 is a flowchart of information processing according to the third modification.

As shown in FIG. 14, the binaural sound processing apparatus 10 executes steps S100 to S101 as in FIG.

After step S101, the binaural sound processing apparatus 10 executes acquisition of environment information (S130).
Specifically, the binaural sound processing device 10 is connected to an environment sensor (not shown) configured to detect environment information regarding the environment in which the user U exists. The sensor includes, for example, at least one of the following.
・ Temperature sensor that detects temperature ・ Humidity sensor that detects humidity ・ Sensor that detects the UV level to which user U is exposed

The processor 12 acquires environmental information detected by the environmental sensor via the input / output interface 13.

After step S130, the binaural sound processing apparatus 10 performs binaural sound correction (S131).
Specifically, the storage device 11 stores correction parameters for each environment information.
The processor 12 corrects the sound characteristics of the binaural sound using the correction parameter corresponding to the environmental information acquired in step S130. The voice characteristic includes, for example, at least one of the following.
・ Frequency (playback speed as an example)
-Sound pressure-Time difference between left and right binaural sounds (as an example, phase difference)

After step S131, the binaural sound processing device 10 executes step S102 as in FIG.

According to the third modification, the binaural sound processing device 10 corrects the binaural sound according to the environment where the user U exists (that is, the binaural sound reproduction environment). As a result, the user U can experience a tactile sensation according to the environment.

(5-4) Modification 4
Modification 4 will be described. Modification 4 is an example in which a binaural sound is output in accordance with the content related to the applied material.

(5-4-1) Overview of Modification 4 An overview of Modification 4 will be described. FIG. 15 is an explanatory diagram of an outline of the fourth modification.

As shown in FIG. 4, in the applied material information database, an applied material ID, a binaural sound source file, and a content file are stored in association with each other.
As illustrated in FIG. 15, when content (for example, image content specified by the user U) is selected, the binaural sound processing device 10 opens a binaural sound source file associated with the content file corresponding to the selected content. Thus, an audio signal corresponding to the binaural sound source file is generated.
The audio output device 20 outputs a binaural sound based on the audio signal generated by the binaural sound processing device 10.

(5-4-2) Information Processing Information processing according to the fourth modification will be described. FIG. 16 is a flowchart of information processing according to the fourth modification. FIG. 17 is a diagram illustrating an example of a screen displayed in the information processing of FIG.

As shown in FIG. 16, the binaural sound processing device 10 executes content specification reception (S130).
Specifically, the processor 12 displays the screen P130 (FIG. 17) on the display.

The screen P130 includes an operation object B130 and a field object F130.
The field object F130 is an object that receives a user instruction for designating the file name of the content file.
The operation object B130 is an object that receives a user instruction for confirming the file name specified in the field object F130.

When the user U specifies an arbitrary file name in the field object F130 and operates the operation object B130, the processor 12 acquires the file name specified in the field object F130.

After step S130, the binaural sound processing apparatus 10 performs binaural sound source selection (S131).
Specifically, the processor 12 reads the binaural sound source file associated with the content file corresponding to the file name acquired in step S130 with reference to the applied material information database (FIG. 4).

After step S131, the binaural sound processing apparatus 10 performs playback of content and binaural sound (S132).
Specifically, the processor 12 displays the screen P131 (FIG. 16) on the display by opening the content file corresponding to the file name acquired in step S130.

The screen P131 includes a display object A131.
On the display object A131, content corresponding to the content file (for example, a promotional video of the applied material) is reproduced.

The processor 12 generates an audio signal of a binaural sound recorded in the binaural sound source file read in step S131 so as to be synchronized with the content reproduced by the display object A131.
The processor 12 transmits the generated audio signal to the audio output device 20.
The audio output device 20 outputs a binaural sound based on the audio signal transmitted from the processor 12.

According to the fourth modification, the binaural sound processing device 10 presents a binaural sound in synchronization with the content designated by the user U. Thereby, it is possible to let the user U experience the tactile sensation when using the coated material introduced by the content while presenting the desired content.

In the fourth modification, when the binaural sound processing apparatus 10 receives the designation of the CG content in step S130, the binaural sound processing apparatus 10 changes the CG content in accordance with the user instruction in step S132, and changes the CG content to the changed CG content. A corresponding binaural sound may be reproduced.

As an example, when a CG content in a scenario in which a makeup professional applies a coating to an image of the user U's face is specified, the user U instructs the binaural sound processing device 10 to at least one of the following: User instructions can be given.
An instruction for designating a part (all or a part) to be coated with an application. An instruction for designating a coating application method (for example, thin coating or thick coating). The processor 12 responds to a user instruction. The CG content displayed on the display is changed, and the binaural sound source information is also changed according to the changed CG content.

In this case, the binaural sound processing apparatus 10 presents CG content and binaural sound according to the user instruction. Thereby, according to the user instruction | indication while experiencing CG content, the user U can be made to experience interactively and actively the tactile sense when using a coating material.

(5-5) Modification 5
Modification 5 will be described. The fifth modification is an example in which the binaural sound is corrected in accordance with the change with time of the components of the coating material during the reproduction of the binaural sound.

(5-5-1) Overview of Modification 5 An overview of Modification 5 will be described. FIG. 18 is an explanatory diagram of an outline of the fifth modification.

As shown in FIG. 18, the binaural sound processing device 10 corrects the binaural sound according to the lapse of time while listening to the binaural sound output from the audio output device 20.
The audio output device 20 outputs the binaural sound corrected by the binaural sound processing device 10.

(5-5-2) Information Processing Information processing according to Modification 5 will be described. FIG. 19 is a flowchart of information processing according to the fifth modification.

As shown in FIG. 19, the binaural sound processing apparatus 10 executes steps S100 to S101 as in FIG.

After step S101, the binaural sound processing apparatus 10 performs binaural sound correction (S140).
Specifically, the storage device 11 stores correction parameters corresponding to tactile sensations that change over time for each application ID.
The processor 12 identifies the correction parameter associated with the application ID identified in Step S100.
The processor 12 corrects the sound characteristics of the binaural sound using the specified correction parameter based on the reproduction time of the binaural sound. The voice characteristic includes, for example, at least one of the following.
・ Frequency (playback speed as an example)
-Sound pressure-Time difference between left and right binaural sounds (as an example, phase difference)

According to the modified example 5, the binaural sound processing device 10 corrects the binaural sound according to the reproduction time. Thereby, the user U can experience the time change of the tactile sensation when using the coating.

(6) Summary of this embodiment This embodiment is summarized.

The first aspect of this embodiment is
Application identification information (for example, application ID) for identifying each of the plurality of applications, binaural sound source information (for example, binaural sound source file) in which the sound when each application is applied to the skin is binaurally recorded, Means (for example, storage device 11) for storing
Means for determining application identification information for identifying at least one application from among a plurality of applications (for example, the processor 12 that executes the process of step S100);
Means for selecting binaural sound source information associated with the determined application identification information (for example, the processor 12 that executes the process of step S101);
Means for outputting the selected binaural sound source information (for example, the processor 12 that executes the process of step S102);
This is a binaural sound processing apparatus 10.

According to the first aspect, the binaural sound processing device 10 presents the user U with a binaural sound corresponding to the coated material. Thereby, before using a coating material, the user U (for example, consumer of a coating material) can be made to experience the tactile sensation of a coating material.

The second aspect of this embodiment is
The means for determining determines the application identification information in accordance with an instruction from the operator.
This is a binaural sound processing apparatus 10.

According to the second aspect, the binaural sound processing device 10 presents the user U with a binaural sound corresponding to a desired application. Thereby, before using a coating material, the user U (for example, consumer of a coating material) can be made to experience the touch feeling of a desired coating material.

Means for associating and storing binaural sound source information and user attribute information related to user attributes;
Means for acquiring user attribute information (for example, the processor 12 that executes the process of step S110) on the user attribute;
The means for selecting selects the binaural sound source information associated with the determined application identification information and the acquired user attribute information.
This is a binaural sound processing apparatus 10.

According to the third aspect, the binaural sound processing device 10 presents the user U with a binaural sound corresponding to the user attribute of the user U. Thereby, the user U (for example, the consumer of a coating material) can be made to experience the tactile sensation of the coating material suitable for the user U.

The fourth aspect of this embodiment is
Means for predicting a coated material recommended for use by the user (for example, the processor 12 that executes the process of step S111);
The means for determining the application object identification information is the binaural sound processing apparatus 10 that acquires the application object identification information of the determined application object.

According to the fourth aspect, the binaural sound processing device 10 presents the user U with the binaural sound of the application recommended for use by the user U. Thereby, the user U (for example, the consumer of a coating material) can be made to experience the touch feeling of the coating material recommended to use with respect to the user U.

The fifth aspect of this embodiment is
The means for determining the applied material determines the applied material based on the user's skin condition.
This is a binaural sound processing apparatus 10.

According to the fifth aspect, the binaural sound processing device 10 presents the user U with a binaural sound of an application recommended for use based on the skin state of the user U. Thereby, the tactile sensation of the application according to the skin condition of the user U can be experienced by the user U (for example, a consumer of the application).

The sixth aspect of this embodiment is
Based on the user's behavior history, it is provided with means for predicting an application recommended for use by the user (for example, the processor 12 that executes the process of step S111),
The means for selecting selects binaural sound source information associated with the application identification information of the predicted application.
This is a binaural sound processing apparatus 10.

According to the sixth aspect, the binaural sound processing device 10 presents the user U with the binaural sound of the application recommended for use based on the user U's action history. Thereby, the tactile sensation of the application according to the tendency of the user U's behavior can be experienced by the user U (for example, the consumer of the application).

The seventh aspect of this embodiment is
The action history is at least one of a browsing history of the website accessed by the user and a purchase history of the application purchased by the user.
This is a binaural sound processing apparatus 10.

According to the seventh aspect, the binaural sound processing apparatus 10 uses the binaural of the application recommended for use based on at least one of the browsing history of the user U's website and the purchase history of the application purchased by the user. Present a sound. Thereby, the user U (for example, the consumer of a coating material) can be made to experience the tactile sensation of the coating material according to at least one of the user's U behavior tendency and the purchasing tendency.

The eighth aspect of this embodiment is
Means (for example, a processor 12 that executes the process of step S120) for acquiring movement information related to the movement of the user who uses the application;
Means for correcting the selected binaural sound source information according to the acquired motion information (for example, the processor 12 that executes the process of step S121);
The output means outputs the corrected binaural sound source information.
This is a binaural sound processing apparatus 10.

According to the eighth aspect, the binaural sound processing device 10 corrects the binaural sound according to the movement of the user U while listening to the binaural sound. Thereby, it is possible to cause the user U to interactively and actively experience the tactile sensation when using the application.
Moreover, even if the user U moves while listening to the binaural sound, the tactile sensation when using the coating can be correctly transmitted to the user U.

The ninth aspect of this embodiment is
The means for acquiring motion information acquires motion information from at least one of an image sensor, an acceleration sensor, a gyro sensor, and a geomagnetic sensor.
This is a binaural sound processing apparatus 10.

According to the ninth aspect, the tactile sensation when using the application can be correctly transmitted to the user U according to the motion information specified from at least one of the image, acceleration, angular acceleration, and geomagnetism.

The tenth aspect of this embodiment is
Means for acquiring environment information relating to the user's environment (for example, the processor 12 that executes the process of step S130);
Means for correcting the selected binaural sound source information according to the acquired environment information (for example, the processor 12 that executes the process of step S131);
The output means outputs the corrected binaural sound source information.
This is a binaural sound processing apparatus 10.

According to the tenth aspect, the binaural sound processing device 10 corrects the binaural sound according to the environment where the user U exists (that is, the binaural sound reproduction environment). Thereby, the user U can experience a tactile sensation according to the environment.

The eleventh aspect of this embodiment is
Means for associating and storing the moving image content relating to the applied material and the binaural sound source information;
The means for selecting selects binaural sound source information associated with the video content to be played,
Correct binaural sound source information according to the playback time of the video content.
This is a binaural sound processing apparatus 10.

According to the eleventh aspect, the binaural sound processing device 10 presents a binaural sound in synchronization with the content designated by the user U. Thereby, it is possible to let the user U experience the tactile sensation when using the coated material introduced by the content while presenting the desired content.

The twelfth aspect of this embodiment is
Means for associating and storing the CG content relating to the coating and the binaural sound source information;
The means for selecting selects binaural sound source information associated with the reproduced CG content,
Binaural sound source information is changed in accordance with an operator instruction received during playback of CG content.
This is a binaural sound processing apparatus 10.

According to the twelfth aspect, in response to an instruction (for example, a user instruction) given to the user U while experiencing the CG content, the tactile sensation when using the application is interactively and actively performed. It can be experienced.

The thirteenth aspect of this embodiment is
A means for predicting a temporal change in composition of the coating identified by the determined coating identification information with reference to a prediction model for predicting a temporal change in composition when each coating is applied to the human body ,
Means for time-converting the selected binaural sound source information based on the predicted time change;
The means for outputting outputs time-converted binaural sound source information.
This is a binaural sound processing apparatus 10.

According to the thirteenth aspect, the binaural sound processing device 10 corrects the binaural sound according to the reproduction time. Thereby, the user U can experience the time change of the tactile sensation when using the coating.

The fourteenth aspect of this embodiment is
This is a program for causing a computer (for example, the processor 12) to realize any one of the above means.

(7) Other variations

The storage device 11 may be connected to the binaural sound processing device 10 via the network NW. The storage device 31 may be connected to the server 30 via the network NW.

The binaural sound processing device 10 and the audio output device 20 may be integrally configured. In this case, the binaural sound processing device 10 and the audio output device 20 are connected by an electric circuit instead of wired communication or wireless communication.

The server 30 may execute at least a part of the steps executed by the binaural sound processing device 10. In this case, the binaural sound processing device 10 transmits an information processing request to the server 30.
The server 30 performs information processing according to the request transmitted from the binaural sound processing device 10 and transmits a response including the result of the information processing to the binaural sound processing device 10.
The binaural sound processing device 10 transmits an audio signal to the audio output device 20 based on the information processing result included in the response transmitted from the server 30.

At least one of the databases stored in the storage device 11 may be stored in the storage device 31.

As mentioned above, although embodiment of this invention was described in detail, the scope of the present invention is not limited to said embodiment. The above-described embodiment can be variously improved and changed without departing from the gist of the present invention. Moreover, said embodiment and modification can be combined.

1: Audio processing system 10: Binaural sound processing device 11: Storage device 12: Processor 13: Input / output interface 14: Communication interface 20: Audio output device 30: Server 31: Storage device 32: Processor 33: Input / output interface 34: Communication interface

Claims

Means for storing the application identification information for identifying each of the plurality of applications and the binaural sound source information in which the sound when each application is applied to the skin is binaurally recorded;
Means for determining application identification information for identifying at least one application from a plurality of applications;
Means for selecting binaural sound source information associated with the determined application identification information;
Means for outputting the selected binaural sound source information;
Binaural sound processing device.
The means for determining determines the application identification information in accordance with an instruction from an operator.
The binaural sound processing apparatus according to claim 1.
Means for associating and storing the binaural sound source information and user attribute information relating to user attributes;
Means for obtaining user attribute information relating to user attributes;
The selecting means selects the determined applied object identification information and binaural sound source information associated with the acquired user attribute information.
The binaural sound processing apparatus according to claim 1.
Means for determining an application recommended for use by the user;
The means for determining the application object identification information obtains the application object identification information of the determined application object,
The binaural sound processing apparatus according to claim 3.
The means for determining the applied material determines the applied material based on the skin condition of the user.
The binaural sound processing device according to claim 4.
Based on the user's behavior history, comprising means for predicting the application recommended for use by the user,
The means for selecting selects binaural sound source information associated with the application identification information of the predicted application.
The binaural sound processing device according to any one of claims 1 to 5.
The behavior history is at least one of a browsing history of a website accessed by the user and a purchase history of a coating purchased by the user.
The binaural sound processing device according to claim 6.
Means for obtaining movement information relating to the movement of the user using the application;
Means for correcting the selected binaural sound source information according to the acquired motion information;
The means for outputting outputs the corrected binaural sound source information.
The binaural sound processing device according to any one of claims 1 to 7.
The means for acquiring the movement information acquires the movement information from at least one of an image sensor, an acceleration sensor, a gyro sensor, and a geomagnetic sensor.
The binaural sound processing apparatus according to claim 8.
Means for obtaining environmental information about the user's environment;
Means for correcting the selected binaural sound source information according to the acquired environment information;
The means for outputting outputs the corrected binaural sound source information.
The binaural sound processing device according to any one of claims 1 to 9.
Means for associating and storing the moving image content relating to the applied material and the binaural sound source information;
The means for selecting selects binaural sound source information associated with the video content to be played back,
Correcting the binaural sound source information according to the playback time of the video content;
The binaural sound processing device according to any one of claims 1 to 10.
Means for associating and storing CG (Computer Graphics) content related to the coating material and the binaural sound source information;
The means for selecting selects binaural sound source information associated with the CG content to be reproduced,
Changing the binaural sound source information in response to an operator instruction received during playback of the CG content;
The binaural sound processing device according to any one of claims 1 to 11.
Means for predicting a temporal change in the composition of the coating product identified by the determined coating product identification information with reference to a prediction model for predicting a temporal change in the composition when each coating material is applied to the human body. Prepared,
Means for time-converting the selected binaural sound source information based on the predicted temporal change;
The outputting means outputs the time-converted binaural sound source information;
The binaural sound processing device according to any one of claims 1 to 12.
A program for causing a computer to realize each means according to any one of claims 1 to 13.