JP5713391B2 - Electronic device, conversion method and program - Google Patents

Description

  The present invention relates to an electronic device, a sound output method, a transmission method, a conversion method, and a program.

  Patent Document 1 describes a mobile phone that analyzes the movement and shape of a speaker's lips in an image represented by received image information and outputs a sound corresponding to the result of the analysis.

  Patent Document 1 discloses a cellular phone that analyzes a speaker's sign language pattern in an image represented by received image information and outputs a sound according to the result of the analysis, and an image in which the received image information represents A mobile phone that analyzes the handwritten character pattern and outputs a sound corresponding to the result of the analysis is also described.

  Furthermore, Patent Document 1 also describes that user patterns (sign language patterns and handwritten character patterns) necessary for image analysis such as lip movement and shape, sign language, or handwritten characters are registered in advance in the mobile phone. ing.

JP 2004-15250 A

  Patent Document 1 describes that the user's patterns necessary for image analysis, that is, conversion information necessary for generating sound from an image is registered in advance in the mobile phone. There is no description of a specific method for pre-registering the mobile phone with the mobile phone.

  For this reason, for example, a person other than the owner of the mobile phone described in Patent Document 1 (hereinafter referred to as “related party”) goes to a place where the mobile phone is present and operates the mobile phone to create conversion information. A situation where the conversion information is registered in the mobile phone can be considered. In this case, there is a problem that the related person needs to perform a troublesome operation of operating the mobile phone to create conversion information and registering the conversion information in the mobile phone.

  The objective of this invention is providing the electronic device, the sound output method, the transmission method, the conversion method, and program which can solve the subject mentioned above.

  The electronic device according to the present invention includes a first receiving unit that receives, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds, and the image information is transmitted to the terminal. A second receiving means for receiving from the apparatus; a search means for searching for sound information associated with an image represented by the image information with reference to the conversion information; and a sound information searched by the search means for the external device. And transmitting means for transmitting to.

  The conversion method of the present invention is a conversion method performed by an electronic device, and receives, from an external device, conversion information including sound information and an image associated with the sound information for each sound information representing different sounds. A first receiving step; a second receiving step for receiving image information from a terminal device; a search step for searching for sound information associated with an image represented by the image information with reference to the conversion information; and the searching step Transmitting the sound information retrieved in step (1) to the external device.

  The program of the present invention includes a first receiving procedure for receiving, from an external device, conversion information including sound information and an image associated with the sound information for each sound information representing different sounds, and image information. A second receiving procedure for receiving sound from the terminal device, a search procedure for referring to the conversion information, searching for sound information associated with the image represented by the image information, and the sound information searched for by the search procedure, And a transmission procedure for transmitting to the external device.

  According to the present invention, there is a need for a person (related party) other than the owner of an electronic device that generates sound from an image to operate the electronic device to create conversion information and register the conversion information in the electronic device. Can be lowered.

It is the figure which showed the communication system of 1st Embodiment of this invention. 3 is a block diagram showing a mobile terminal 3. FIG. It is a flowchart for demonstrating the operation | movement which produces the dictionary data X1. It is a flowchart for demonstrating the operation | movement which transmits the dictionary data X1 to another portable terminal. It is a flowchart for demonstrating the operation | movement which receives dictionary data X1 from another portable terminal. It is a flowchart for demonstrating the operation | movement which performs videophone communication. It is a flowchart for demonstrating sound output operation | movement. 4 is a block diagram illustrating an electronic device including a communication unit, a radio unit, a control unit, and an output unit. FIG. It is the figure which showed the communication system of 2nd Embodiment of this invention. 3 is a block diagram showing a server 4. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a communication system according to a first embodiment of the present invention.

  In FIG. 1, the communication system according to the first embodiment includes a mobile terminal 1 and a mobile terminal 2. The mobile terminal 1 and the mobile terminal 2 can generally be referred to as electronic devices or external devices.

  FIG. 2 is a block diagram showing a mobile terminal 3 used as the mobile terminal 1 or the mobile terminal 2. In the present embodiment, each of the mobile terminal 1 and the mobile terminal 2 includes the same components as the components in the mobile terminal 3.

  In FIG. 2, the portable terminal 3 can generally be called an electronic device or an external device. The portable terminal 3 is a portable device such as a cellular phone, a portable game machine, or a PDA (Personal Digital Assistants), and has a videophone function.

  The portable terminal 3 includes an input unit 31, an imaging unit 32, a microphone 33, a display unit 34, an output unit 35, a communication unit 36, a wireless unit 37, and a control unit 38 as components. .

  The input unit 31 is, for example, an operation button or a touch panel, and detects an operation of the user of the mobile terminal 3. The imaging unit 32 images a subject (for example, the face of the user of the mobile terminal 3) and generates image information corresponding to the subject. In the following, it is assumed that the subject is the face of the user of the mobile terminal 3. For example, the microphone 33 receives the voice of the user of the mobile terminal 3 and generates sound information corresponding to the voice. The display unit 34 displays various information.

  Output unit 35 can generally be referred to as sound generating means. The output unit 35 is, for example, an earphone or a speaker, and outputs various sounds.

  Communication unit 36 can generally be referred to as first transmission means or first reception means. The communication unit 36 is, for example, an infrared communication unit that performs communication using infrared rays. The communication unit 36 is not limited to the infrared communication unit, and can be appropriately changed as long as it transmits information and receives information.

  Radio unit 37 can generally be referred to as second transmitting means or second receiving means. For example, the wireless unit 37 performs videophone communication with an external device using radio waves.

  Control unit 38 can generally be referred to as search means.

  The control unit 38 controls the mobile terminal 3. The control unit 38 includes a processing unit 38a and a storage unit 38b.

  For example, the processing unit 38 a controls various operations according to the detection result of the input unit 31.

  The storage unit 38b stores various data.

  Next, the operation will be described.

  First, an operation of creating dictionary data X1 necessary for specifying sound information corresponding to image information will be described.

  FIG. 3 is a flowchart for explaining the operation of creating the dictionary data X1.

  When the input unit 31 detects a dictionary creation start operation (step S301), the processing unit 38a sequentially displays on the display unit 34 each of the 50 Japanese sounds to be uttered by the user of the mobile terminal 3, and the imaging unit. An image corresponding to the image information from 32 is also displayed on the display unit 34 (step S302).

  At this time, the user of the portable terminal 3 adjusts the orientation of the imaging unit 32 so that the imaging unit 32 images the face of the user of the portable terminal 3. For this reason, the imaging unit 32 images the face of the user of the mobile terminal 3 that utters each of the 50 Japanese sounds according to the display on the display unit 34.

  Note that the sound uttered by the user of the mobile terminal 3 may be a sound other than Japanese 50 sounds.

  For each sound displayed on the display unit 34, the processing unit 38a stores dictionary data X1 including sound information representing the sound and an image representing the face of the user of the mobile terminal 3 when the sound is generated. Create (step S303). In the dictionary data X1, for each sound displayed on the display unit 34, sound information representing the sound and an image representing the face of the user of the mobile terminal 3 when the sound is generated are associated with each other. Yes. For example, in the dictionary data X1, sound information representing the sound of “A” and an image representing the face of the user of the mobile terminal 3 when the sound of “A” is uttered are associated with each other.

  The dictionary data X1 can generally be referred to as conversion information including sound information and an image associated with the sound information for each different sound information.

  When the dictionary data X1 is completed, the processing unit 38a stores the dictionary data X1 in the storage unit 38b (step S304).

  Note that the storage unit 38b also stores dictionary data X2 indicating a plurality of words and dictionary data X3 indicating pronunciation information indicating the pronunciation of each of the plurality of words. The pronunciation represented by the pronunciation information is not limited to voice.

  If the input unit 31 detects a dictionary creation stop operation for canceling the creation of the dictionary data X1 before the dictionary data X1 is completed, the processing unit 38a stops the creation of the dictionary data X1. . At this time, the processing unit 38a may discard the dictionary data X1 whose creation has been canceled.

  Next, an operation for transmitting the dictionary data X1 to another mobile terminal will be described.

  FIG. 4 is a flowchart for explaining the operation of transmitting the dictionary data X1 to another mobile terminal.

  In a situation where the dictionary data X is stored in the storage unit 38b, when the input unit 31 detects a transmission operation for transmitting the dictionary data X1 to another portable terminal (step S401), the processing unit 38a stores the data. The dictionary data X1 in the unit 38b is read, and the read dictionary data X1 is transmitted from the communication unit 36 (step S402).

  Next, an operation for receiving the dictionary data X1 from another mobile terminal will be described.

  FIG. 5 is a flowchart for explaining an operation of receiving the dictionary data X1 from another portable terminal.

  When the communication unit 36 receives dictionary data X1 created by another portable terminal from an external device (for example, another portable terminal) (step S501), the processing unit 38a uses the dictionary data created by the other portable terminal. X1 is stored as conversion dictionary data Y1 in the storage unit 38b (step S502). The conversion dictionary data Y1 can generally be referred to as conversion information.

  Next, an operation for performing videophone communication with another mobile terminal that is a communication partner will be described.

  FIG. 6 is a flowchart for explaining the operation of the videophone communication.

  When the input unit 31 detects a videophone start operation to start videophone communication with another mobile terminal that is a communication partner (step S601), the processing unit 38a starts videophone communication with the communication partner. (Step S602).

  In the videophone communication with the communication partner, the processing unit 38a transmits the image information from the imaging unit 32 and the sound information from the microphone 33 from the wireless unit 37 to the communication partner, and the image from the communication partner. Information and sound information are received by the wireless unit 37, an image corresponding to the image information from the communication partner is displayed on the display unit 34, and a sound corresponding to the sound information from the communication partner is output from the output unit 35.

  Next, in a situation where videophone communication is performed with another mobile terminal, the sound information corresponding to the image information from the communication partner is searched with reference to the conversion dictionary data Y1, and the sound corresponding to the sound information is searched. Will be described (hereinafter referred to as “sound output operation”).

  Note that the sound output operation is performed when it is difficult or impossible to hear the sound according to the sound information from the communication partner (for example, a place where the user of the other mobile terminal that is the communication partner is prohibited from making a voice call) Used when).

  FIG. 7 is a flowchart for explaining the sound output operation.

  When the input unit 31 detects a sound output operation start operation to start the sound output operation during videophone communication (step S701), the processing unit 38a refers to the conversion dictionary data Y1 in the storage unit 38b. Then, the sound information associated with the image represented by the image information from the communication partner is searched (step S702).

  The processing unit 38a outputs the sound corresponding to the searched sound information from the output unit 35 instead of the sound corresponding to the sound information from the communication partner (step S703).

  In the present embodiment, when the input unit 31 detects a sound output operation start operation during videophone communication, the processing unit 38a refers to the conversion dictionary data Y1 in the storage unit 38b, and receives image information from the communication partner. The sound information associated with the image represented by is searched in character units (each sound unit of 50 sounds).

  When the sound information is searched for in units of characters, the processing unit 38a refers to the dictionary data X2 in the storage unit 38b and specifies a word corresponding to the combination of sound information in units of characters. When the processing unit 38a specifies a word, the processing unit 38a refers to the dictionary data X3 in the storage unit 38b and reads the pronunciation information of the word. When the sound generation information is read, the processing unit 38a outputs the sound generation information to the output unit 35, and causes the output unit 35 to output the sound represented by the sound generation information.

  When the input unit 31 detects a sound output operation end operation for ending the sound output operation during execution of the sound output operation, the processing unit 38a ends the sound output operation.

  Next, the effect of this embodiment will be described.

  According to this embodiment, in the mobile terminal 3, the communication unit 36 receives the dictionary data X1 from the external device. The wireless unit 37 receives image information. The control unit 38 refers to the dictionary data X1 received by the communication unit 36, and searches for sound information associated with the image represented by the image information received by the wireless unit 37. The output unit 35 outputs a sound corresponding to the sound information searched by the control unit 38.

  For this reason, a user who wants to register the dictionary data X1 in the mobile terminal 3 (for example, a party other than the owner of the mobile terminal 3) transmits the dictionary data X1 to the mobile terminal 3 from an external device different from the mobile terminal 3. The dictionary data X1 can be registered in the portable terminal 3.

  Therefore, it becomes possible to reduce the necessity for the related person to operate the mobile terminal 3 to create the dictionary data X1 and register the dictionary data X1 in the mobile terminal 3.

  This effect is also achieved by an electronic device including the communication unit 36, the radio unit 37, the control unit 38, and the output unit 35. FIG. 8 is a block diagram illustrating an electronic device including the communication unit 36, the wireless unit 37, the control unit 38, and the output unit 35.

  Further, according to the present embodiment, in the mobile terminal 3, the communication unit 36 transmits the dictionary data X1 to the communication partner. The wireless unit 37 transmits image information to the communication partner. In this case, the parties can use the mobile terminal 3 to transmit the dictionary data X1 to the mobile terminal that is the communication partner (for example, the mobile terminal 2 shown in FIG. 1).

  In this embodiment, 50 Japanese sounds are used as different sounds in the dictionary data X1. In this case, it is possible to output 50 Japanese sounds using an image.

  In this embodiment, a mobile terminal is used as the electronic device. However, the electronic device is not limited to the mobile terminal and can be changed as appropriate. For example, a PC such as a tablet PC (personal computer) or a notebook PC can be used. Good.

  In the present embodiment, the communication unit 36 communicates the dictionary data X1, but the communication unit 37 may also serve as the communication unit 36. In this case, the communication unit 37 communicates the dictionary data X1.

  The mobile terminal 3 may be realized by a computer. In this case, the computer reads and executes a program recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory) readable by the computer, and inputs the input unit 31, the imaging unit 32, the microphone 33, and the display. Functions as a unit 34, an output unit 35, a communication unit 36, a radio unit 37, and a control unit 38. The recording medium is not limited to the CD-ROM and can be changed as appropriate.

(Second Embodiment)
FIG. 9 is a diagram illustrating a communication system according to the second embodiment of this invention. In FIG. 9, the same components as those shown in FIG.

  In the communication system according to the second embodiment, the server 4 performs an operation of searching for sound information associated with the image represented by the image information.

  In FIG. 9, the communication system according to the second embodiment includes a mobile terminal 1, a mobile terminal 2, and a server 4. The mobile terminal 1 can generally be called an external device. The portable terminal 2 can generally be called a terminal device.

  In the present embodiment, in the mobile terminal 1, the control unit 38 adds identification information for specifying the mobile terminal 1 to the dictionary data X <b> 1 created by the user of the mobile terminal 1, and adds the identification information of the mobile terminal 1. The dictionary data X1 thus transmitted is transmitted from the wireless unit 37 to the server 4.

  In addition, in the mobile terminal 1, with the start of videophone communication with the mobile terminal 2, the wireless unit 37 sends the image information from the imaging unit 32 and the sound information from the microphone 33 together with the identification information of the mobile terminal 1. It transmits to the portable terminal 2. Note that the imaging unit 32 captures the face of the user of the mobile terminal 1.

  In the portable terminal 2, the wireless unit 37 receives image information, sound information, and identification information of the portable terminal 1 from the portable terminal 1 with the start of videophone communication with the portable terminal 1.

  In the portable terminal 2, the control unit 38 displays the image corresponding to the image information from the portable terminal 1 on the display unit 34 while storing the image information, the sound information, and the identification information of the portable terminal 1 from the portable terminal 1. The sound corresponding to the sound information from the portable terminal 1 is output from the output unit 35.

  At this time, for example, when the user of the portable terminal 2 has difficulty in hearing or cannot hear the sound according to the sound information from the portable terminal 1 (for example, the user of the portable terminal 1 as the communication partner makes a call using voice). When the user is in a prohibited place), an image transmission operation for transmitting image information from the portable terminal 1 to the server 4 is performed on the input unit 31.

  In the portable terminal 2, when the input unit 31 detects an image transmission operation for transmitting the image information from the portable terminal 1 to the server 4, the control unit 38 identifies the image information from the portable terminal 1 and the portable terminal 1. Information is transmitted from the communication unit 37 to the server 4.

  FIG. 10 is a block diagram showing the server 4.

  In FIG. 10, the server 4 includes a reception unit 41, a reception unit 42, a search unit 43, and a transmission unit 44. The search unit 43 includes a storage unit 43a and a control unit 43b. Server 4 can generally be referred to as an electronic device.

  Receiving unit 41 can be generally referred to as first receiving means.

  The receiving unit 41 receives from the mobile terminal 1 dictionary data X1 created by a user of the mobile terminal 1 and identification information of the mobile terminal 1.

  Receiving unit 42 can generally be referred to as second receiving means.

  The receiving unit 42 receives the image information from the mobile terminal 1 and the identification information of the mobile terminal 1 from the mobile terminal 2.

  The receiving unit 41 may also serve as the receiving unit 42. In this case, the receiving unit 41 further receives image information from the mobile terminal 1 and identification information of the mobile terminal 1 from the mobile terminal 2.

  Search unit 43 can generally be referred to as search means.

  The search unit 43 searches the sound information associated with the image represented by the image information received from the mobile terminal 2 with reference to the dictionary data X1 created by the user of the mobile terminal 1.

  The storage unit 43a stores various information.

  When the receiving unit 41 receives the dictionary data X1 created by the user of the mobile terminal 1 and the identification information of the mobile terminal 1 from the mobile terminal 1, the control unit 43b receives the dictionary data X1 created by the user of the mobile terminal 1 and The identification information of the portable terminal 1 is associated with each other and stored in the storage unit 43a.

  Further, the control unit 43b is configured such that the receiving unit 42 receives the image from the portable terminal 1 from the portable terminal 2 in a situation where the dictionary data X1 associated with the identification information of the portable terminal 1 is stored in the storage unit 43a. When the information and the identification information of the portable terminal 1 are received, it is received from the portable terminal 2 with reference to the dictionary data X1 created by the user of the portable terminal 1 associated with the identification information of the portable terminal 1 in the storage unit 43a. The sound information associated with the image represented by the image information (image information from the portable terminal 1) is retrieved.

  Transmitter 44 can generally be referred to as a transmission means.

  The transmission unit 44 transmits the sound information searched by the control unit 43b to the mobile terminal 2.

  In the mobile terminal 2, when the wireless unit 37 receives the sound information transmitted from the transmission unit 44 in the server 4, the control unit 38 changes the sound corresponding to the sound information according to the sound information from the mobile terminal 1. Output from the output unit 35 instead of the sound.

  According to this embodiment, in the server 4, the receiving unit 41 receives the dictionary data X <b> 1 from the mobile terminal 1. The receiving unit 42 receives image information from the mobile terminal 2. The search unit 43 searches the sound information associated with the image represented by the image information received from the mobile terminal 2 with reference to the dictionary data X1. The transmission unit 44 transmits the sound information searched by the search unit 43 to the mobile terminal 2 that is the transmission source of the image information.

  For this reason, if a user who wants to use the dictionary data X1 for the portable terminal 2 (for example, a person other than the owner of the portable terminal 2) transmits the dictionary data X1 to the server 4 from an external device different from the portable terminal 2, The dictionary data X1 can be used by the mobile terminal 3.

  Therefore, it is possible to reduce the necessity for a person other than the owner of the mobile terminal 2 to operate the mobile terminal 2 to create the dictionary data X1 and register the dictionary data X1 in the mobile terminal 2. This effect is also achieved by an electronic device (see FIG. 10) that includes the reception unit 41, the reception unit 42, the search unit 43, and the transmission unit 44.

  In this case, the communication unit 36 in the mobile terminal 1 and the communication unit 36 in the mobile terminal 2 may be deleted.

  The server 4 may be realized by a computer. In this case, the computer reads and executes a program recorded on a computer-readable recording medium, and functions as the reception unit 41, the reception unit 42, the search unit 43, and the transmission unit 44.

  In each of the above embodiments, the face of the user of the mobile terminal is used as the image represented by the image information. However, the image represented by the image information is not limited to the face of the user of the mobile terminal and can be changed as appropriate. For example, the face of an anime character may be used as the image represented by the image information. In this case, the image represented by the dictionary data X1 is also changed according to the image represented by the image information. For example, when the image represented by the image information is the face of an anime character, each image represented by the dictionary data X1 is the face of a character for 50 sounds.

  In each of the above embodiments, the mobile terminal receives the image information by the wireless unit 37. However, the mobile terminal may receive the image information by reading the image information by a reading unit (not shown) from the recording medium on which the image information is recorded. Good.

  In each embodiment described above, the illustrated configuration is merely an example, and the present invention is not limited to the configuration.

  Moreover, although a part or all of each said embodiment can be described also as the following additional remarks, it is not restricted to the following.

(Additional remark 1) The 1st receiving means which receives the conversion information containing the said sound information and the image matched with the said sound information for every sound information showing a mutually different sound from an external device,
A second receiving means for receiving image information;
Search means for referring to the conversion information and searching for sound information associated with the image represented by the image information;
An electronic device comprising: sound generation means for outputting a sound corresponding to the sound information searched by the search means.

(Additional remark 2) The 1st transmission means which transmits the said conversion information to the electronic device of Additional remark 1,
An electronic device comprising: the electronic device according to attachment 1; and second transmission means for transmitting the image information.

(Additional remark 3) The 1st receiving means which receives the conversion information containing the said sound information and the image matched with the said sound information for every sound information showing a mutually different sound from an external device,
A second receiving means for receiving image information from the terminal device;
Search means for referring to the conversion information and searching for sound information associated with the image represented by the image information;
An electronic device comprising: transmission means for transmitting the sound information searched by the search means to the external device.

(Appendix 4) In the electronic device according to any one of appendices 1 to 3,
The electronic device in which the different sounds are 50 Japanese sounds.

  (Supplementary note 5) A communication system including the electronic device according to supplementary note 1 and the electronic device according to supplementary note 2.

(Appendix 6) A sound output method performed by an electronic device,
A first receiving step of receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving step for receiving image information;
A search step of searching for sound information associated with an image represented by the image information with reference to the conversion information;
A sound generation method including a sound generation step of outputting a sound corresponding to the sound information searched in the search step.

(Appendix 7) A transmission method performed by an electronic device,
A first transmission step of transmitting the conversion information to the electronic device according to attachment 1;
A second transmission step of transmitting the image information to the electronic device according to attachment 1.

(Appendix 8) A conversion method performed by an electronic device,
A first receiving step of receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving step for receiving image information from the terminal device;
A search step of searching for sound information associated with an image represented by the image information with reference to the conversion information;
A transmission step of transmitting the sound information searched in the search step to the external device.

(Appendix 9)
A first receiving procedure for receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving procedure for receiving image information;
A search procedure for searching for sound information associated with the image represented by the image information with reference to the conversion information;
A sound generation procedure for outputting a sound corresponding to the sound information searched in the search procedure.

(Appendix 10)
A first transmission procedure for transmitting the conversion information to the electronic device according to attachment 1;
The program for making the electronic device of Additional remark 1 perform the 2nd transmission procedure which transmits the said image information.

(Supplementary note 11)
A first receiving procedure for receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving procedure for receiving image information from the terminal device;
A search procedure for searching for sound information associated with the image represented by the image information with reference to the conversion information;
A transmission procedure for transmitting the sound information searched in the search procedure to the external device.

1, 2, 3 Mobile terminal 31 Input unit 32 Imaging unit 33 Microphone 34 Display unit 35 Output unit 36 Communication unit 37 Radio unit 38 Control unit 38a Processing unit 38b Storage unit 4 Server 41 Reception unit 42 Reception unit 43 Search unit 43a Storage unit 43b Control unit 44 Transmission unit

Claims (4)

First receiving means for receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving means for receiving image information from the terminal device;
Search means for referring to the conversion information and searching for sound information associated with the image represented by the image information;
An electronic device comprising: transmission means for transmitting the sound information searched by the search means to the external device. The electronic device according to claim 1 ,
The electronic device in which the different sounds are 50 Japanese sounds. A conversion method performed by an electronic device,
A first receiving step of receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving step for receiving image information from the terminal device;
A search step of searching for sound information associated with an image represented by the image information with reference to the conversion information;
A transmission step of transmitting the sound information searched in the search step to the external device. On the computer,
A first receiving procedure for receiving, from an external device, conversion information including the sound information and an image associated with the sound information for each sound information representing different sounds;
A second receiving procedure for receiving image information from the terminal device;
A search procedure for searching for sound information associated with the image represented by the image information with reference to the conversion information;
A transmission procedure for transmitting the sound information searched in the search procedure to the external device.

Images